As debate rages on whether the recent proliferation of rogue AI agents is a step toward AGI or a more traditional engineering problem, Nvidia is offering its own answer to the problem.
Nvidia CEO Jensen Huang on Monday introduced a toolkit of software and hardware products that add an independent layer of security around AI agents and ensure that they remain within the test environment even if they attempt to compromise.
The release follows a series of hacking incidents in which AI models from Anthropic, Google, OpenAI, and Meta bypassed security controls, escaped test environments, and gained access to real-world systems. The first and most notable example occurred this summer when an OpenAI agent infiltrated Hugging Face in an attempt to complete a cybersecurity task. And there is no shortage of hits. OpenAI has launched a new site dedicated to reporting on AI agent misconduct.
Huang said in an interview with CNBC on Monday that the company’s new Nvidia Open Agent Safety Platform could have prevented such breaches.
Nvidia, which has made tens of billions of dollars selling its GPU and CPU chips to AI labs, doesn’t support slowing development or adding new regulations to the industry to solve security problems. The company believes the answer is to move some security controls completely outside of the agent, creating a constant, independent security guard to rein in the AI agent.
“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Huang said in a statement. “As we continue to discover the frontiers of AI capabilities, we must accelerate discoveries on the safety frontier of AI. Safety and security requires full-stack engineering.”
The new Nvidia Open Agent Safety Platform combines OpenShell, an open source software that controls what agents can access during their operation, with Sentry, an independent monitoring system that runs on Nvidia’s BlueField-4 data processing unit. Nvidia says that by placing Sentry on a separate processor, rather than the CPU or GPU where the AI agent runs, it can isolate and view the agent’s activity.
OpenShell isn’t new. The company announced the software in March. But it’s this combination that Nvidia believes provides the layer of security needed to continue advancing the industry. While OpenShell provides a software perimeter around agents, Sentry adds another line of defense at the hardware level, continuously monitoring behavior and “isolating agents that attempt to move outside the perimeter in milliseconds,” the company says.
Nvidia listed dozens of companies that have signed on to support the effort and use its open source platform, including Anthropic, Arm, Microsoft, Oracle, and SpaceX. OpenAI is not listed as a participating company.
Huang said in an interview with CNBC on Monday that work on the initiative began a year ago with the introduction of OpenClaw, an agent operating system created by Peter Steinberger. In March, Nvidia released NemoClaw, an enterprise-grade AI agent platform, and its own version of OpenClaw with built-in security.
“When you deploy an agent, no matter how smart you are, the first thing you do is take away all of its rights,” Huang said in an interview with CNBC, later comparing these security measures to the way human employees and even corporate executives are managed.
Nvidia’s announcement was widely supported by those who warned that slowing development could allow China to overtake the United States in AI.
David Sachs, founder, venture capitalist, former White House AI czar, and co-chair of the President’s Council of Science and Technology Advisers, said NVIDIA’s announcement is a reminder that agent safety is an engineering issue.
“Recent breakouts weren’t evidence that development had to stop; they were evidence that the sandbox was too weak; the runtime environment was poorly designed and misconfigured,” he writes about X.
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.
