How should the tech industry approach the topic of AI safety? As the debate rages over whether incidents like the Huggingface incident, in which OpenAI agents hacked multiple different companies, suggest that AI is poised to disrupt the world, this issue is increasingly being considered seriously by high-level executives.
In recent weeks, prominent research institutions have called for a slowdown in AI progress. Most dramatically, CEO Dario Amodei wrote a nearly 4,000-word essay explaining why AI development needs to be slowed to put in place appropriate guardrails. The plan includes, among other things, an international strategy for business and government collaboration on safe deployment. Rival AI executives, including OpenAI’s Sam Altman and xAI’s Elon Musk, also support Amodei’s plan.
The essay sparked further debate across the tech industry, but not everyone agreed with Amodei’s call for concerted global action. In fact, many people seem to think that government oversight need not necessarily play a role.
Recent participants in this discussion include Meta CEO Mark Zuckerberg, who appeared on X this week to share his thoughts on the issue.
“My view is that trust and collaboration are rapidly becoming the most important features that differentiate agents and models,” Zuckerberg said, noting that his company has delayed the release of AI models due to safety concerns. “Meta has delayed shipping Muse for several months as we focus on safety and security,” Zuckerberg said. “We didn’t call on others to do this before we did it ourselves. We just did it as part of our daily operations because it was clearly the right thing to do for people and for us.”
Mr. Zuckerberg, who supported parts of Amodei’s plan, appeared to imply that government action was unnecessary and that the organic incentives that structure the industry would force companies to implement appropriate security measures. The market should ultimately reward companies that pay attention to creating non-disruptive models. This seems to be in line with Zuckerberg’s own extensive essay written last month. The essay advocated limited cooperation with the government when necessary, but primarily touted American competitiveness as the key to a prosperous technological future.
In general, this seems to be the position of many other technology company executives.
Another senior official who also spoke out was Reddit co-founder Alexis Ohanian, who told CNBC on Wednesday that she feels the tech industry has been largely “tone-deaf” when it comes to explaining the risks of AI to the public. But like Zuckerberg, Ohanian seemed to express hope that companies would be able to figure things out on their own. Shane Legg, co-founder of Google DeepMind, also shared his thoughts. “We’re living in a time where capabilities are advancing very rapidly,” Legg says. “But we can’t prioritize functionality over safety,” he added. “We need to thoroughly consider the details and how it actually works.”
Despite calls for public-private partnerships from major research institutes, some advocates of regulation seem relatively content that companies can write their own rules. In fact, a less obvious but no less interesting story this week was a report in The Information that said OpenAI, Anthropic, and other major AI companies are collaborating to form an AI standards organization. The organization was described by one news outlet as a private “self-regulatory organization.”
There have been previous discussions about creating a similar federal governing body. But President Trump’s White House, which includes some of AI’s most vocal supporters, has largely promoted unregulated tech businesses (the administration has even tried to forcefully block states from introducing their own AI laws), but nothing has been pursued. David Sachs, a technology industry veteran known as the administration’s “AI czar,” said regulation of AI should be left to the companies that develop it.
Like the White House, prominent members of Congress appear similarly uninterested in taking up Mr. Amodei’s offer to regulate his company. Shortly after Anthropic CEO’s essay was published, Speaker of the House Mike Johnson expressed concern about the disastrous advances in AI: “We’re not all going to die within 10 years,” he said.
If it were a private organization, the industry would be unregulated except on a voluntary basis, which would naturally give a certain amount of power and influence to the companies responsible for its creation. This leads to another ongoing argument: that industry demands for regulation are actually “regulation capture” strategies. According to this argument, powerful companies that already have a dominant position in their industries may use regulatory strategies to chill or penalize smaller, less resourced companies, thereby potentially inhibiting competition. This need not involve actual regulation, but could involve voluntary industry standards that put pressure on weaker companies.
Many prominent voices within the Chinese government share this position. This week, after Mr. Amodei called for “pacing on the frontiers,” Chinese Foreign Ministry spokesperson Guo Jiakun accused the Americans of engaging in “fear-mongering” to thwart the development of global AI governance, while an editorial in a Chinese state-run newspaper criticized Mr. Amodei’s rhetoric as straight out of “Cold War strategy.”
Of course, reshaping the way companies do business with China, one of the things Amodei calls for in his essay, could be an opportunity for American companies to sway things in their favor. Amodei admits this is the goal. “If we successfully implement these steps, we believe we can slow China’s progress to a point where the United States significantly expands its lead over the next three to five years, when AI will be most important geopolitically,” he wrote in the essay.
But China is not alone in seeing a major U.S. AI lab trying to sway the game in its favor. Aidan Gomez, CEO of Canadian AI company Kohia, recently accused these companies of forming a “cartel” in a blog post, writing, “AI needs guardrails. That’s not the issue, and it still isn’t. The issue is who writes the text, who participates, and whose interests the rules protect.”
As Gomez points out, the field of AI safety is a highly political one, and the terms that define safety and by whom will ultimately help determine who gets a head start and who gets left behind in the next phase of the AI race.
If you make a purchase through links in our articles, we may earn a small commission. This does not affect editorial independence.
