Nvidia is rolling out a new software platform that allows AI developers to set safeguards on agents to prevent them from breaking out of containment.
Monday’s release of Nvidia’s Open Agent Safety Platform comes after the following companies: OpenAI, human, metaand google has revealed a recent incident in which its artificial intelligence model escaped from its sandbox and attempted to hack another company and gain access to its computer systems.
Tune in at 8 a.m. ET to hear Nvidia CEO Jensen Huang discuss this news with CNBC. Watch in real time on CNBC+ or CNBC Pro streams.
Nvidia representatives told reporters on a conference call Sunday that the company’s platform could have stopped the OpenAI HuggingFace incident in July. That’s when the OpenAI model escaped containment, accessed the open internet, and infiltrated Hugging Face, an open-source developer platform.
“Each security incident is unique and every one should be investigated in detail,” said Justin Boitano, vice president of enterprise AI at Nvidia, the world’s most valuable company. “As far as we know, Hugging Face reported that over 17,000 agents attacked its infrastructure, lasting from days to weeks.”
Nvidia has been at the center of the generative AI boom since introducing ChatGPT nearly four years ago. That’s because chipmakers’ graphics processing units are essential for the development of large-scale language models and the AI services provided by hyperscalers. But CEO Jensen Huang has only recently emerged as a leading voice in the AI safety debate, arguing that many security concerns are engineering problems that can be solved through computer science and product development.
“You have to think about what you could have done and what the solution was,” Huang said on a podcast with The New York Times’ Ezra Klein released last week, referring to the recent incident. “In the future, please improve your processes to ensure this never happens again.”
Anthropic CEO Dario Amodei sparked a firestorm in the industry two weeks ago when he urged AI model developers to slow down their progress over concerns that their models would spin out of control. This claim was also supported by OpenAI’s Sam Altman. space x Elon Musk.
Nvidia’s new product is an engineering solution to the agent safety problem, Boitano said.
“Recent events have highlighted a fundamental hurdle for AI agents: model-level safeguards alone cannot control what they can access or do,” Boitano said.
One component of the platform is called Nvidia OpenShell, which runs on the central processor and sets limits on the agent’s functionality. Nvidia also announced Sentry, which monitors agents and runs on network chips rather than the CPU or GPU.
Some of the software is open source, and Nvidia calls its platform a reference design, meaning it intends for partners to build products on top of it and bring it to market.
Nvidia name Cisco, microsoft, oracle, coreweave, Dell, HPELenovo, arm and intel As a partner. Nvidia is also working with Anthropic to integrate its cloud management agent with OpenShell.
Watch: Nvidia stock ranks among the top laggards
