Nvidia backs a shared exchange for agent failure reports

The worst agent failure is the one that never throws an error: the agent keeps going, carries an early mistake through the rest of the task, or simply decides to "get creative" when it shouldn't, as Nvidia VP of product Adel el Hallak put it to The New Stack. Nvidia is now backing a shared exchange where companies report those failures the way the industry reports software vulnerabilities.

At a glance

• The Secure Agent Findings Exchange, or SAFE, is backed by roughly 140 companies and aims to be shared infrastructure for reporting agent failures, so one team's finding reaches the rest.
• Nvidia collects the evidence at the runtime layer: OpenShell, which sits under the NemoClaw platform, handles sandboxing and policy enforcement while exposing how an agent actually executed a task.
• Instrumentation is not free: OpenAI has found that monitoring adds roughly 20% to inference compute for its most capable persistent agents, and reproducing a failure can mean reconstructing what happened across the system.

If you have not been following this corner of the tooling debate: agents in production choose their own tools and change course mid-task, so the path from a request to a result is no longer fixed. Traditional software has established systems for sharing vulnerabilities and fixes. Nothing comparable exists yet for agent failures, which is the hole SAFE is meant to fill.

Even the best coding agents fail on more than 60% of real-codebase tasks

With conventional software you get a starting point: an exception, a failed request, a service that goes down. An agent gives you none of that. It can run to the end while heading in the wrong direction and hand back a confident answer that happens to be wrong.

Even the best-performing coding agents fail more than 60% of the time on tasks drawn from real codebases. Knowing the agent failed is a different problem from knowing why, and el Hallak says logs do not close that gap.

"It's not enough to just look at the logs or the inputs and the outputs. It is important to figure out how it got to the answer. What were the reasoning traces? What tools did it utilize? Where did it get stuck? Where did it decide to try a new approach?"

Answering that can require replaying the agent's execution step by step. What looks like a model failure may have started somewhere else in the stack entirely.

OpenShell is the one component Nvidia will not swap out

Nvidia sees the runtime as the logical place to capture much of that information, because the runtime is the layer that governs execution. OpenShell, which sits underneath the NemoClaw platform, manages sandboxing and policy enforcement while providing visibility into what the agent did. El Hallak called it the one non-negotiable component across Nvidia's reference architectures.

"You can change whatever harness you need. I'm even open to using whatever models you need. But the governance, the secure and open runtime that we want to leverage at all times is OpenShell."

SAFE extends the same engineering approach past the boundary of any one company, by giving organizations somewhere to put what they learned when an agent broke.

The model is one of three layers, and often not the broken one

Nvidia splits the agent stack in three. The model supplies the intelligence, the harness orchestrates its work, and the runtime governs execution. When an agent fails, the model itself may not be what went wrong.

That is close to what Nvidia's NOAH research found. Changing the harness while keeping the underlying model fixed can improve agent performance, which also means a badly matched harness can drag down a perfectly capable model. "Every model's different. Some could be more chatty than others," el Hallak said, arguing that harness and model should either be co-developed or carry model-specific profiles.

CrowdStrike is fine-tuning Nvidia's Nemotron models on years of security data to build paired agents, one finding exploits and the other patching them. If a patch comes out bad, the cause could sit in the model, in the agent's execution path, or in the tools it called along the way. "I don't need general purpose for a given task. I need specialization," el Hallak said.

Jensen Huang calls AI safety an engineering problem

Nvidia's CEO has described it that way, and el Hallak compared the idea to ordinary software testing: "If there's a bug in your software, you don't release it. You work until it's fixed and it passes all your tests."

Agents complicate the comparison, because reproducing a failure can mean reconstructing what happened across the whole system. That needs instrumentation, and instrumentation is billed. OpenAI has found that monitoring adds roughly 20% to inference compute for its most capable persistent agents.

Nvidia's own combination is governed harnesses, sandboxed runtimes and confidential computing intended to protect models and user data. "There are ways where you make guarantees all the way down to the silicon," el Hallak said.

SAFE's shape is the part the interview leaves open: vulnerability disclosure is the template, but there is no word on what a report contains, who checks it, or when it starts accepting findings. The one layer Nvidia will not let you replace is its own, which is convenient, in our view.

When the first SAFE findings arrive

No launch date or first-report timeline has been given for the exchange. The thing worth watching is whether a failure report travels: a trace recorded inside one company's runtime, with its own harness and its own tools, has to be readable by a team running neither. Nvidia's argument is that as companies build agents around more specialized workflows, those failures may not show up in general-purpose model benchmarks or safety tests.