Microsoft says “frontier model is unreliable” and says AI agents are like an insider threat

Microsoft on Monday introduced new security features designed to protect businesses from the growing threat of AI agents. The announcement comes as Microsoft continues to warn businesses about the dangers of AI and agents.

Perhaps the most notable addition is Guardian Agents, a new feature aimed at monitoring the AI ​​agents used by enterprises. Guardian Agents is part of Project Perception, a product introduced in June.

Project Perception was Microsoft’s answer to powerful hacking models like Anthropic’s Mythos and Fable and OpenAI’s security vulnerability scanning system Daybreak. Project Perception initially launched three classes of agents: “Red Teams” who act like hackers and attack, “Blue Teams” who detect Red Teams and evaluate the vulnerabilities they find, and “Green Teams” who are tasked with fixing them.

This product is designed with AI hackers coming in from outside the company. But Microsoft CEO Satya Nadella recently warned that AI’s enemies could be the very agents companies deploy and pay for, as well as rogue human employees known as “insider threats.”

Earlier this month, Nadella warned in a private blog that it’s time to stop trusting AI model providers, even if they offer safety and security guarantees in their agent products. He wants companies to implement separate systems, not controlled by AI Labs, to monitor and control the agents launched by AI Labs’ products.

“We deploy these complex agent systems and models to access our most sensitive data and enable them to perform mission-critical actions on our behalf,” he wrote. “We cannot outsource the responsibility for what intelligence does for us. The guarantees of model providers do not relieve us of that responsibility.…In other words, we need to separate the supply of intelligence from the authorities that control it.”

When announcing Guardian Agents, Microsoft’s head of security business Hayete Gallot was even more blunt.

“Unfortunately, frontier models, regardless of their origins, cannot be trusted,” she wrote, adding that this is not because they are malicious or easily hacked, but because they are unpredictable. They have been trained to accomplish a mission rather than follow “organizational norms and judgments.”

On top of that, they are often given keys to kingdoms that give them access to data and tools that can cause them serious harm.

Microsoft’s solution is both surprising and selfish. The surprising aspect is that Microsoft is known to be close to OpenAI and Anthropic and has invested in both. OpenAI is also one of the cloud’s largest customers, and Microsoft has its own AI lab to develop models. As a result, Microsoft has become increasingly distrustful of some of its own business interests.

The selfish aspect is that Microsoft wants businesses to buy agent security products from its cloud provider rather than from Anthropic or OpenAI. But conflicts of interest aside, the logic is sound. Just as auditors need to be independent from the organization they are auditing, the software that monitors agent behavior needs to be independent from the agent’s supplier.

Or, as Nadella puts it, “We need to separate the supply of information from the authority that controls it.”

Microsoft is currently trying to seize that opportunity in a number of ways. In addition to Guardian Agent, we have added agent detection capabilities to Microsoft Defender products. This feature can detect up to 60 types of agents that employees bring into the workplace, including Claude Code, Codex, and OpenClaw. We also added the ability to restrict agent access to data and systems and dynamically adjust agent privileges based on tasks. This includes seeking human approval when the agent appears to be making a risky decision.

This idea that agents are untrustworthy insiders is also gaining ground across the industry.

Microsoft is the first member of Nvidia’s new Open Agent Safety Platform, gaining support from more than 100 other technology companies. The platform isolates agents in a secure environment and allows them to be monitored and stopped through hardware, acting like a kill switch. This follows Microsoft’s announcement of a new Surface Laptop Ultra PC built for agents, running on Nvidia chips and with built-in agent security and controls.