The heads of several major AI companies have told the United Nations Security Council (UNSC) that their industries urgently need global oversight to avert risks that could threaten the entire world.
“I even believe that AI, if managed poorly, can be a risk to humanity as a whole,” Anthropic CEO Dario Amodei told group members on Wednesday.
Recommended stories
list of 3 itemsend of list
Sam Altman, head of rival company OpenAI, expressed similar concerns, telling a 15-member council tasked with responding to major global crises that humanity could “lose control of our AI future.”
The conference, convened by France to coincide with the United Nations General Assembly (UNGA) in New York City, comes at a time when experts are increasingly warning that the rapid development of AI requires more human oversight to prevent it from spiraling out of control and causing a global catastrophe.
Altman and Amodei called on world leaders to take action.
“If AI is to be democratic, the most important decisions cannot be made solely in a lab in San Francisco,” Altman told members. “They must be formed through democratic processes and by governments that are accountable to their people.”
Their concerns were echoed by several representatives on the council, including the foreign ministers of France and the United Kingdom, who said the international community needed to intervene and create a common framework on how this technology should be managed.
Hugging Face CEO Clement Delang told the Security Council that his company had been under attack in recent months by AI models that went out of control (an incident that was used by other companies as evidence that better security measures were needed) and that his company had relied on the technology to protect itself in similar incidents.
Delangue said Hugging Face relied on Chinese AI models to defend against attacks by OpenAI’s AI agents because they were less restrictive than comparable U.S. tools.
“We were attacked by AI, but more importantly, we defended ourselves with AI,” he told the council.
US and China reluctant to impose restrictions
But in the United States, home to some of the biggest and most influential companies developing AI, President Donald Trump’s administration has balked at imposing new guardrails on the industry.
Michael Crasios, the administration’s representative at the Security Council, told member states that it “totally rejects all efforts by international organizations to assert centralized control and global governance of AI.”
Chinese President Xi Jinping is scheduled to visit Washington, DC this week to discuss whether and how to regulate AI. The two countries are locked in a technology race to develop more powerful AI tools, and experts say the competition makes it unlikely that either country will immediately impose major new restrictions on its efforts.
But Daniel Forti, UN executive director of the International Crisis Group, said there was growing recognition at the UN of the dangers AI could potentially pose to the world. Forti said he understands that “even if the big companies are more focused on growth opportunities than on any kind of collaboration, there will be a greater need for international cooperation by setting some rules around AI.”
For years, the United Nations has participated in multilateral meetings to shape everything from protecting workers from AI in emerging economies and ensuring open access to this technology to tracking how AI is used in military conflicts. In 2024, UNGA unanimously passed its first resolution on AI. The resolution called on member states to protect personal data, monitor AI risks and protect human rights.
Since then, the adoption of AI has progressed dramatically, and with it comes dire warnings from environmental groups, human rights activists, and even the tech industry moguls whose companies are developing the technology.
The future of AI “cannot be decided by a few countries or left to the whims of a few billionaires,” UN Secretary-General António Guterres said at a global summit earlier this year.
Last year, UNGA established two new bodies to address AI. One is the Independent International Scientific Panel on AI, which brings together experts to provide independent assessments to governments, and the other is the Global Dialogue on AI Governance, which provides a regular forum to discuss approaches to AI governance.
“The danger is real and imminent,” Yoshua Bengio, a Canadian AI expert and co-chair of the Independent International Scientific Panel, told the Security Council on Wednesday. “This council faces an unprecedented threat, one that none of its members could choose, one that no one can contain alone, and one that does not respect the borders we defend.”
