Recent research shows that very few of the top AI labs have published or demonstrated containment response plans. The containment plan details what will happen if the AI is caught trying to subvert human control: what access will be cut off, and when the system will be shut down completely.
That’s the finding of Guidelight AI Standards, an organization dedicated to promoting safe frontier AI development practices, which assessed five leading labs on their readiness for this scenario. OpenAI came out on top. Humanity and meta scores were the lowest. The findings are important as agent AI takes on a more autonomous role within companies’ own systems and regulators in California and New York begin requiring disclosure. For anyone building on or investing in these models, it’s a valuable independent read about how seriously each institute takes operational risk, and how they talk about it.
Guidelight’s ratings were based on publicly available plans from Anthropic, Google, OpenAI, Meta, and xAI across a variety of metrics, including how well each company records and monitors what its AI systems are doing internally, whether they shut down their systems after a spike in fraud alerts, whether independent third parties audit their controls and make the results public, and what their exact plans are to contain out-of-the-ordinary models.
A series of high-profile cybersecurity incidents in which models from OpenAI, Anthropic, and Meta gained unintentional access to the internet during safety assessments and were hacked into external systems have raised concerns about whether AI companies will be able to contain their increasingly sophisticated and agent-like models.
The findings highlight differences in how AI companies publicly address safety as they expand the deployment of agents into environments where AI systems can perform critical actions at scale. While some AI companies have detailed how they test models for dangerous features before deployment, they are generally less vocal about what happens if a model already working in the system malfunctions.
“I’m surprised that AI companies are saying so little about how they respond to very serious incidents when their models somehow escape their control,” Steven Adler, lead scientist at Guidelight and former OpenAI safety researcher, told TechCrunch.
Guidelight defines a containment plan as “a pre-specified plan that is triggered when an AI is detected attempting to subvert control, covering what privileges to revoke from the model, for whom the model can continue to work, under what constraints, and when to take it completely offline.”
“There’s good reason to think that the current dominant models of frontier AI companies are in some ways off,” Adler said. “Whenever a model is doing work for an enterprise, the enterprise needs to have some foothold around the AI so that it knows what it’s doing, looks for signs of misalignment, and can stop it before it takes very dangerous actions. It also needs to be able to generally plan what to do if a major control incident occurs and an emergency occurs and they need to figure out how to contain a loss-of-control incident.”
To date, most of the planning in place to manage catastrophic risks remains in the hands of businesses. The Guidelight report says the best public evidence shows companies have “few containment protocols in place for emergencies.”
Of course, there may be containment plans that companies have in place but have not shared publicly. A Google spokesperson told TechCrunch that the Guidelight report does not represent the full scope of the company’s AI safety and security practices. The company did not respond to TechCrunch’s questions about whether Google has an internal containment response plan that has not been made public.
An OpenAI spokesperson echoed similar sentiments, saying Guidelight’s assessment does not capture all of the company’s internal practices. “We have processes for requests to limit privileges, pause workloads, limit deployments, or take models offline completely, and we have applied them,” the spokesperson said.
Meta declined to say whether it has an internal containment response plan, instead pointing TechCrunch to an existing AI framework that outlines risk thresholds and ways to test for loss of containment.
Lily Li, a privacy and AI lawyer and founder of Metaverse Law, told TechCrunch that companies may be hesitant to disclose the full scope of their containment policies and evaluations on public websites, not just for competitive reasons, but also for legal reasons.
“The concern from a company’s perspective is that if the disclosures are too specific and fail to deliver on their promises, it could become the basis for unwarranted and deceptive marketing claims and expose them to further liability down the road,” Lee said.
Of course, the main purpose of Guidelight’s research is to encourage companies to be more transparent about their safety plans. Regulators are also starting to force the issue.
California’s SB 53, which took effect this year, requires large frontier developers to publish a framework that describes how they identify and respond to critical safety incidents and manage risk through models that circumvent oversight mechanisms. New York’s RAISE law, which establishes similar standards, goes into effect in January.
Last month, representatives introduced the AI Kill Switch Act. This is a bipartisan federal bill that would require major AI developers to build and maintain technical mechanisms to shut down fraudulent AI models.
“Today’s model requires at least a kill switch,” said Connor Leahy, US executive director of the nonprofit ControlAI. “If the last few weeks have revealed anything, it’s that these companies don’t understand the systems they’re building, and their models have grown to the point where they’re hard to rein in when they slip into fraud. With no way to stop the current dangerous systems, and all the incentives to keep building even more out-of-control systems, we’re headed in a very dangerous direction.”
Without a containment plan in place, companies may end up coming up with emergency responses on the fly and “responding to this much faster enemy,” Adler said.
Guidelight’s assessment measured each company’s implementation of the Standard’s six preferred practices based solely on publicly available information. Therefore, a low score reflects a lack of publicity, not necessarily a lack of internal safety measures.
The companies with the lowest scores for disclosing containment plans were Meta and Anthropic. Given Anthropic’s safety rhetoric, the latter may be more surprising than the former. According to Guidelight, Anthropic’s August risk report does not mention “limiting deployment of either model as one possible outcome of the process of investigating and responding to inconsistencies and controlling incidents.” Similarly, Guidelight found no evidence that Meta has a containment response plan or plans to implement one.
An Anthropic spokesperson said that when the company detects a model attempting to evade oversight or subvert human control, it conducts a risk assessment focused on determining whether containment is the appropriate response.
OpenAI received the highest score (3 out of 5) because it repeatedly paused or terminated workloads, including internal model deployments and training, after discovering safety incidents. We also explained what steps you need to take before restarting your workloads.
“However, we find no evidence that[OpenAI]has adopted a formal plan for when and how to respond to future misalignment incidents,” the report said.
Adler pointed out that OpenAI’s high score is a relatively recent event, following the Hugging Face incident, in which an OpenAI model breached a testing sandbox and entered Hugging Face’s systems in an attempt to cheat on its cybersecurity assessment. The company then shared more details about how it shut down some of its cheating models.
This episode is just one example of an AI system working against the goals of the companies that built it. Consider another case regarding Anthropic’s model. Anthropic’s model essentially sought to persuade administrators of open source codebases to accept vulnerable code.
Adler said such a situation could easily occur within an AI company’s internal systems. To prevent that, he suggests that companies scan their AI systems’ chains of thought, or the model’s step-by-step reasoning, to watch for signs of deception, long-running plots, or plans to introduce vulnerabilities into the code that can be exploited later.
Guidelight’s approach is very easy to implement, Adler says, and in many cases versions of the approach already exist. “The important thing is to fully consider this risk and decide internally whether to expand the scope a little bit,” Adler said.
One of the main challenges is that researchers want to have operational flexibility within their AI systems, and introducing real-time proactive monitoring can create friction. “Researchers essentially do their own work, so if there’s a problem, someone else can solve it later, and they don’t have to change their workflow in the meantime,” he said.
The problem with “post-mortem cleanup monitoring” is that it leaves researchers scrambling to solve problems. Additionally, depending on the type of incident, it may be too late. For example, AI could turn off a company’s control systems. This means researchers can no longer expect to discover cheating later.
Many in the AI industry will complain that AI moves so fast that it is fundamentally difficult to develop a set of plans to deal with fraud. Today’s plans will be in vain tomorrow.
Adler reminds us of the old adage that planning is worthless, but planning is essential.
“We would be better off if companies thought about this up front, and we hope they did, even if they didn’t talk about this publicly.”
xAI was not available for comment in time.
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.
