human Days after CEO Dario Amodei rocked the industry with his call for a concerted slowdown, the company is sharing three new metrics that it says could help artificial intelligence companies monitor the pace of development.
The company said in a blog post on Thursday that it shared a methodology for measuring AI-driven research and development, AI agent monitoring, and compute allocation within Anthropic and encouraging other organizations to do similar efforts. The metric is based on a three-stage deceleration plan announced by Amodei on Saturday, but provides few details about what the actual rollout will include.
“As the world looks to explore frontiers, we should do everything possible to minimize the gap between what frontier laboratories know and what the public knows,” Anthropic said in a post Thursday. “This means better measuring developments in AI, reporting it publicly, and giving society the opportunity to decide how this information is used.”
Amodei’s weekend call for slowdowns received support from industry leaders, including: OpenAI CEO Sam Altman said: space x CEO Elon Musk and Google DeepMind Chairman Demis Hassabis followed a stark warning from researchers about AI’s increasing potential for harm. Amodei said his plan aims to slow the rate of improvement in model capabilities “without sacrificing commercial advantage or America’s lead in AI.”
Regarding the first metric, Anthropic said it determined that its Claude model was “not operating fully autonomously” for some of the research and development work it measured.
The second metric involves building systems to monitor and intervene in the actions taken by AI agents. They found that at any given time, approximately 30,000 agents were performing research and engineering work across the most used internal platforms.
For the third metric, Anthropic measured a “snapshot” of how all compute was used from July 13th to July 20th. The company said it found that about 6% of the compute used for AI research and development was allocated to safety. The company said about 12% of the computing allocated to “AI-driven” research and development was devoted to safety.
Anthropic said these metrics are great for showing how models are built and complement functional assessments that show “what a model can do.” Overall, Anthropic said third parties outside the institute should have a “starting point” to assess the pace of AI development.
“We want to model that transparency by publishing these measurements, and we will continue to do so,” Antropic said.
Spotlight: Why AI Labs Demands Slowdown
