After calling for a slowdown in the industry over the weekend along with Anthropic’s Dario Amodei and Elon Musk, OpenAI chief Sam Altman has released his most detailed comments yet on how the AI safety framework will work.
Safety concerns about the technology have reached a fever pitch since the Anthropic researcher quit last week, with those building the technology warning that they believe it “could wipe out humanity by the end of the decade” and prompting other employees at the institute and rival OpenAI to warn of the catastrophic risks.
Since then, AI company leaders have shown rare solidarity, with both Mr. Altman and Mr. Musk on Saturday endorsing Mr. Amodei’s essay urging AI companies to slow down the rate at which they improve their cutting-edge models.
AI stocks fell on Monday as investors digested the comments. President Donald Trump on Sunday dismissed CEOs’ warnings, saying the slowdown was unnecessary and would jeopardize America’s lead over China in AI.
Sam Altman reveals two ways AI could go ‘very bad’
“We welcome a federal framework that sets consistent safety requirements for frontier AI,” Altman said in a post on X shortly after midnight Monday, adding that “no amount of U.S. competitive pressure should justify recklessness.”
Altman warned of two situations in which advances in AI could go in a “very bad” direction. One is the loss of “control of the future to AI” and the concentration of too much power in the hands of a single individual or company.
Meanwhile, lawmakers in Washington are scrambling to respond to calls for safeguards.
This all comes as Anthropic and OpenAI prepare for what is expected to be a historic IPO. In an interview with Fortune magazine published Saturday, Altman ruled out going public in 2026.
Amodei’s 3-step proposal
Many of the concerns about AI safety revolve around models that develop the ability to improve their own performance, a technique known as recursive self-improvement (RSI).
“Since almost the summer of this year, AI has made tremendous and rapid progress, primarily due to improvements in AI’s ability to build the next generation of AI,” Amodei wrote in his essay. “If left unchecked, these systems can be beyond our ability to understand and control them, so we need to be very careful when tracking them.”
Amodei proposed a three-phase plan aimed at slowing the pace of development without “sacrificing commercial advantage or America’s lead in AI.”
The plan includes giving each frontier AI company “employee-like access” to external evaluators, which Anthropic is currently working on, he said. Amodei also called on leading AI research institutes to establish common safety standards, limit the rate of unchecked AI progress, and coordinate efforts globally.
In a short X post on Saturday, Altman said he agreed with Amodei that AI companies should “keep pace with the frontier.” He added: “Committing to having independent evaluators with the same access rights as employees is a great idea and we intend to do the same.”
“It’s a good idea (and we’re excited about ideas like independent auditors) to have consistent rules that allow us to manage frontier risk and maximize returns,” Altman said in a post Monday. But he added: “When we say ‘set the pace,’ we don’t mean ‘stop.’ Progress has been rapid and will continue to be so.”
“The cost of pacing is worth it,” he concluded. “No amount of U.S. competitive pressure should justify recklessness or put capability ahead of coordination and oversight.”
“We need help from governments, international coordination. But first we should do what we can ourselves.”
international cooperation
Coordinating China’s competing AI developers, AI security measures and industry slowdown is likely to be a major challenge.
The United States and China remain embroiled in a battle for AI supremacy, with tensions rising as Chinese models become more sophisticated and adopted globally.
Amodei said Sunday that the “toughest dilemma” about his proposal is what would happen if adversaries choose not to take the same action.
“The longer term would be to work together to put speed limits on the rate at which AI advances,” Amodei said on CBS News’ “Sunday Morning.”
“I think that’s a very difficult thing to do because the incentive to move forward and the military advantage that comes from it is so great. And to be honest, I don’t know if it’s possible, but we should try.”
Anthropic CEO’s essay has drawn criticism in China, with the state newspaper Global Times writing on Monday that “Amodei’s proposal portrays China’s legitimate development in the field of AI as a threat and aims to further stir up conflict between China and the United States in this field.”
China’s Foreign Ministry said on Monday that the CEOs’ comments were “fear-mongering”.
