Microsoft Chairman and CEO Satya Nadella on Saturday called for building advanced artificial intelligence systems with containment, independent control, and “emergency brakes” that allow authorized people to pause or stop models in the middle of a task.
“Non-deterministic models need to be surrounded by strong, deterministic system design, human control, and reliable operating procedures to establish industry standards where existing standards are insufficient,” Nadella said in a post on social media platform X.
“Treatment of frontier closed-weight models and open-weight models like insider risk is the way to build such a system,” he said.
Nadella’s comments came amid warnings from tech company leaders and researchers. microsoft Co-founder Bill Gates, Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, space x CEO Elon Musk – Claims technology is moving too fast for insufficient AI safety protocols.
An AI researcher quit Anthropic last month, accusing the company and its biggest rival, OpenAI, of “putting their lives on the line.” Later that day, the head of coordination at AI safety specialist Anthropic said there was a more than 10% chance that the technology would “kill all humans” within the next 10 years.
Conversely, President Donald Trump has repeatedly denied the risk of AI extinction, instead emphasizing the need for the industry to stay ahead of China. President Trump recently introduced a new “AI Corps” led by Director of National Intelligence Jay Clayton to promote the industry and root out bad actors.
Microsoft’s Nadella wrote on Saturday that AI systems should be designed based on “observability principles.”
These include model diversity; A human-readable footprint of the model’s actions. Continuous system testing. Independent management and auditability. Containment. Nadella then writes about the revelations of the incident.
“The most reliable superintelligence system will not be the one with the model we most trust,” he writes. “That would make the model the least reliable.”
