Anthropic will provide employee-level access to six researchers from METR, a nonprofit organization that tests cutting-edge AI models, who will be able to independently evaluate Anthropic’s models, safety practices and risk management, the company said in a blog post Thursday.
The deal comes as incidents of AI agents acting against instructions have been made public every week or two since late August. The incident has sparked calls for independent checks on AI companies.
“We want to be publicly accountable for the decisions we are making,” Anthropic said in a blog post. “We aim to bridge the gap between what evaluators understand about the systems of leading AI companies and what the companies themselves understand.”
Under the new arrangement, the METR team will have an Anthropic-issued company laptop, office space, and access to internal tools roughly equivalent to those used by Anthropic’s own risk assessment team. “Humanity employees are encouraged to share information with their fellow employees working on adjustment risks,” the company said.
Mr. Anthropic likened the arrangement to banking, similar to when regulators have resident inspectors inside large banks.
METR plans to publish its own assessments of Anthropic’s risks, incidents and practices, but the company waived its right to block them.
“Anthropic cannot refuse publication, but may request redactions of information that falls into narrow, predefined categories, such as those related to customer data, attorney-client privilege, or information about model architecture,” the company said. “METR is free to publicly characterize our edits and say whether or not we believe the edits are legitimate, as long as we do so without revealing the information that was redacted,” Antropic added.
METR is the only rating agency at the top of the three-tier system that Anthropic is building. Lower-tier evaluators, such as the biosecurity nonprofit SecureBio, conduct narrower-scope reviews and must be approved by Anthropic before publication. The middle tier can capture more information and publish it without approval, while the top tier adds METR employee-level access.
Thursday’s announcement fulfills a pledge Anthropic CEO Dario Amodei made in a September essay in which he proposed incorporating outside evaluators within the AI lab as a first step toward slowing the technology’s development. OpenAI soon pledged to do the same, and the companies, along with Google, Meta, Nvidia, and xAI, later signed a White House agreement pledging an independent review.
If you make a purchase through links in our articles, we may earn a small commission. This does not affect editorial independence.
