Over the past year, concerns have swirled about the potential dangers posed by artificial intelligence, and debate has raged over the appropriate path for AI safety. Now, Anthropic appears to be offering its own vision for how governments should regulate its industry.
Politico recently reported that the AI Institute is considering new legislation on AI safety and hopes Congress will consider adopting it. The law envisages certain scenarios in which the release of an AI model may be blocked from distribution.
These scenarios include “foreseeable and immediate risks” that could result in more than 50 deaths and more than $1 billion in property damage, Politico wrote. The catastrophic risks in question “must result from the creation or release of biological weapons, models that evade the control of their developers, or cyber-attacks,” the paper said.
The metrics presented here (50 people, $1 billion) are similar to those in the recently passed California bill, Senate Bill 53, which also defines these scenarios as tantamount to “catastrophic risk.”
Anthropic’s bill suggests that an analysis of the model’s potential for disaster could be accomplished through the use of an “independent evaluator” registered with the U.S. Department of Commerce. These evaluators could be compensated by the AI Institute itself, as long as “payment is not dependent on the evaluation results,” Politico wrote.
It is unclear how this proposed system would ensure that payments do not affect ratings. DeepMind Chairman Demis Hassabis argued several months ago that an autonomous AI regulator should emulate the financial industry’s Financial Industry Regulatory Authority (FINRA).
Anthropic’s bill would require the evaluator to contact the Department of Commerce within 72 hours if it determines an imminent catastrophic risk is posed. The Commerce Department would then have to ask a D.C. judge to halt the model’s release. The bill would explicitly prohibit federal agencies from having the discretion to stop distribution of models on their own, Politico writes.
Anthropic did not immediately respond to TechCrunch’s request for comment, but a spokesperson confirmed the bill’s contents to Politico. “This bill reflects the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state legislators since June,” the spokesperson said.
The theoretical risks laid out in the draft policy are a little removed from the kinds of incidents that have been most concerning in the technology industry lately. While some have warned about the potential for people to die, the main concern is the threat of hacking by agents. A spate of incidents in which AI programs apparently disobey human operators and hack websites on the open internet has raised concerns that the models lack adequate security controls.
To offer a solution to the spectacle of crisis created by the lab itself, companies like Anthropic are offering their own recommended solutions.
In fact, research institutions have largely resisted forced government oversight, despite constant warnings that the technologies they sell could spell the end of human civilization. Instead, they proposed an autonomous system with built-in raters. This is likely a “neutral” third-party observer located within the AI organization who can monitor models for potential signs of risk or abuse.
Still, Anthropic’s mere legislative proposal appears to be slightly more advanced than other regulatory proposals put forward by the industry, including NVIDIA. NVIDIA CEO Jensen Huang recently argued that the AI industry doesn’t need any new laws.
If you make a purchase through links in our articles, we may earn a small commission. This does not affect editorial independence.
