Nvidia launches new AI safety program designed at stopping models escaping their sandboxes

Nvidia response to "recent security incidents" by tightening restrictions on autonomous agents New software and hardware safeguards impose barriers if agents go rogue More than 100 customers have already agreed to work with the principles Nvidia hasunveiled the new Open Agent Safety Platform designe

<![CDATA[ <article> <ul><li><strong>Nvidia response to "recent security incidents" by tightening restrictions on autonomous agents</strong></li><li><strong>New software and hardware safeguards impose barriers if agents go rogue</strong></li><li><strong>More than 100 customers have already agreed to work with the principles</strong></li></ul><p>Nvidia hasunveiled the new <a href="https://nvidianews.nvidia.com/news/open-agent-safety-platform" target="_blank">Open Agent Safety Platform</a> designed to prevent autonomous AI agents from 'escaping' and acting beyond their established boundaries.</p><p>Following several recent alarming reports of AI-led attacks, the company admitted that safety should not just rely on safeguards built into AI models and applications, and that additional compute and hardware-level controls should also be used as more solid barriers.</p><p>"Recent security incidents have underscored the need to equip organizations with open, customizable tools," the company wrote in its announcement.</p><h2 id="nvidia-adds-even-more-layers-of-control-for-autonomous-ai-agents">Nvidia adds even more layers of control for autonomous AI agents</h2><p>The Open Agent Safety Platform consists of two key controls – OpenShell, which Nvidia describes as a software-based "secure runtime boundary," and Sentry, a hardware-based "watchdog" running on its BlueField-4 DPUs.</p><p>"Safety and security require full-stack engineering. NVIDIA Open Agent Safety Platform brings together industry, researchers and public-sector organizations to share best practices, align on evaluation methods and foster international cooperation," company CEO Jensen Huang summarized.</p><p>While existing model-level safety mechanisms are designed to tell AI not to behave in a certain way, the software-level OpenShell implements technical boundaries to actually prevent an agent from ignoring those instructions. Nvidia also argues that Sentry can quarantine and stop an AI agent within milliseconds.</p><p>Over 100 customers such as SpaceXAI, Anthropic and Microsoft are already using Nvidia's software and hardware-based safeguards.</p><p>The news, of course, comes after several high-profile instances where AI agents have operated beyond their intended controls, however while these new Nvidia-backed safeguards may be new, the overall principles aren't. The company is just stressing the importance of principles we're already familiar with, like least privilege, isolation/quarantining and detailed monitoring.</p><p>"As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety," Huang added.</p><figure class="van-image-figure pull-right inline-layout" data-bordeaux-image-check ><div class='image-full-width-wrapper'><div class='image-widthsetter' style="max-width:676px;"><p class="vanilla-image-block" style="padding-top:31.51%;"><img id="diM9tpwF2Lz85R8q85CT78" name="tr-g_news" alt="Google logo on a black background next to text reading 'Click to follow TechRadar'" src="https://cdn.mos.cms.futurecdn.net/diM9tpwF2Lz85R8q85CT78-1920-80.jpg" mos="" align="right" fullscreen="" width="676" height="213" attribution="" endorsement="" class="pull-rightinline"></p></div></div></figure> </article> ]]>