As researchers warn that artificial intelligence could pose catastrophic risks, some of the pioneers who helped develop the technology weigh in with additional warnings about the dangers of unrestricted advances in capabilities.
Anthropic, OpenAI, google DeepMind is all about sounding the alarm about the potentially devastating risks of developing technology.
AI chiefs, including Sam Altman, Dario Amodei, Demis Hassabis and Elon Musk, are in rare agreement to call for slowdowns and regulatory oversight.
Meanwhile, debate is heating up in Washington, citing concerns. President Donald Trump on Monday bristled at growing calls to regulate AI, although some Democratic and Republican lawmakers have pushed for legislation to check advances in AI.
Those who contributed to the development of AI think about existential risk:
joshua benzio
Joshua Bengio attends the All In event on Wednesday, September 27, 2023 in Montreal, Canada.
Graham Hughes Bloomberg | Getty Images
Yoshua Bengio is a Canadian computer scientist who is considered a central pioneer of modern deep learning.
“Scientists at the Frontier AI Institute have unique insight into cutting-edge models,” Bengio said on September 9, in response to anthropologist Jacob Coxon’s resignation announcement. Coxon has warned that AI “could kill us all by the end of the decade”.
Scientists often become aware of the “risks involved months before a model is released to the public,” he added. “Their perspectives are essential to keeping society informed and should be taken very seriously.”
Days before the social media storm began, OpenAI’s chief scientists warned that “no one is ready for the consequences” of AI.
In a blog published Friday, Bengio said that today’s AI systems “already possess the hacking skills and persuasive powers needed to cause significant harm to human interests.”
He added that recent events have shown that AI is capable of planning over days or even weeks, and that “as our ability to strategize over the long term continues to advance, the risks will only get worse.”
“My concern about current attempts by AI companies to mitigate inconsistencies is that these efforts may only mask them by rewarding and selecting AIs that cheat without being caught,” Bengio said. “We need to ensure that research continues to better monitor AI behavior, chain of thought, and activity within networks.”
Jeffrey Hinton
Geoffrey Hinton, Godfather of AI at the University of Toronto, takes center stage on day two of Collision 2023 at the Enercare Center in Toronto, Canada.
Ramsey Cardi Sports File | Getty Images
Jeffrey Hinton, professor emeritus at the University of Toronto, is considered a pioneer in neural networks and deep learning.
Asked by the BBC on September 10th if he thought there was a more than 10% chance that AI would kill off all of humanity within 10 years, Hinton said: “This is the kind of thing that is very difficult to estimate, because nothing like this has ever happened before. We have never created an entity that might soon be smarter than us.”
He said a 10% chance was not “unreasonable”, adding: “Nobody knows how to come up with a sensible estimate.”
He said AI systems could design biological viruses that are “very nasty” alongside computer viruses and “perform devastating cyberattacks. There are countless other ways that artificial intelligence could eliminate us if it wanted to. We have to figure out how to design artificial intelligence systems so that they don’t want to do that.”
“We have two possible futures,” Hinton added. “One is finding a way to deal with those hazards before they get out of control, and the other is not.”
aidan gomez
Aidan Gomez, CEO of Cohere Inc., attends the Bloomberg Tech Summit on Tuesday, October 21, 2025 in London, UK.
Chris Ratcliffe | Bloomberg | Getty Images
Aidan Gomez, co-founder and CEO of AI company Cohere, is one of the authors of the 2017 research paper “Attending Is All You Need,” which helped lay the foundation for today’s leading AI models.
“I think these models are the most powerful cyberweapons that have ever been created and that we’ve seen,” Gomez said on CNBC’s “The Tech Download” podcast on Sept. 1. “They’re phenomenal in terms of finding and exploiting vulnerabilities at scale.”
“What recent incidents have made clear is that the security of your deployment environment determines your safety,” he said, referring to a cybersecurity attack carried out by a rogue AI model over the summer.
“Weak container security is what enables breakout events, not an inherent push toward autonomous systems,” Gomez told CNBC. “As policymakers consider how to manage AI, they should recognize that different systems carry different risks, and the rules should not be shaped solely by big tech companies chasing AGI.”
