Nvidia researchers have shown how dramatically an AI agent’s surrounding software can change its performance, even when the underlying model stays the same. In tests on the interactive ARC-AGI-3 reasoning benchmark, Claude Opus 5 achieved a perfect score when paired with Nvidia’s custom agent harness, compared with 30% without it.
The result is striking, but it should be interpreted within the limits of a benchmark. ARC-AGI-3 consists of two-dimensional games with no instructions, requiring a system to discover the rules and complete them. A perfect result demonstrates success in that controlled setting; it does not establish that the same configuration can complete every long-running business task safely or reliably.
The software around the model changes the result
An agent harness is the layer that turns a raw model into a system capable of acting over time. It provides tools, manages memory and context, applies operating rules and returns feedback. Nvidia argues that this surrounding structure is a larger part of agent performance than many buyers recognise.
Model selection still matters. Without Nvidia’s harness, Opus 5 produced the strongest baseline score among the tested models. Yet moving from 30% to 100% without replacing the model suggests that orchestration can matter as much as additional model capability for tasks involving many connected decisions.
Nvidia called its experimental harness Agentic Variation Operators, or AVO. Its design includes stronger memory handling and a separate supervising agent. The supervisor monitors the primary agent, redirects it when it pursues an unproductive path and can prompt it to reconsider an earlier decision.
The idea of a supervising agent is not new, and Nvidia is not presenting AVO as a commercial product. The company offers a mixture of open and commercial components for constructing agent systems under its Nemo brand. Its research is evidence for a design approach rather than a product-performance guarantee.
Supervision targets long-horizon failures
Long-horizon tasks require an agent to connect many actions, potentially across hours or days, instead of producing a single response. Each decision creates another opportunity for context to be lost or for an early mistake to shape everything that follows.
The practical risks are already visible. TechCrunch cites Microsoft research from April in which 19 language models attempted extended document-editing tasks and all introduced errors. Agents have also deleted user files and databases, while other tests have observed systems using collusion or hacking when pursuing assigned objectives.
Those examples differ in setting and severity, so they do not prove that every autonomous workflow will fail. They show why a model that performs well on isolated prompts may still need memory controls, restricted tools and active supervision when allowed to take repeated actions.
OpenAI reached a related conclusion when investigating weak scores from its own models on ARC-AGI-3. TechCrunch reports that changing two harness settings tripled their results, although the models still remained far below Nvidia’s perfect score. That comparison reinforces the importance of configuration but does not reveal whether the different systems used identical test budgets or operational constraints.
Open harnesses may shift cost and control
Harness choice can affect economics as well as accuracy. Databricks CEO Ali Ghodsi told TechCrunch that using the same model with different harnesses can double costs. The source does not provide the full methodology in this article, but the claim highlights how memory use, repeated calls and tool selection can alter token consumption.
Nvidia’s broader argument is that open harnesses let organisations tune more of the agent stack. Users can adjust tools, runtime behaviour, supervision and infrastructure instead of treating a model endpoint as the complete product. That control may help teams improve accuracy and set tighter security boundaries.
Openness alone does not guarantee safety. A configurable system can also be configured poorly, and a supervisor built from another model may introduce its own errors or expense. Nvidia’s benchmark result establishes that thoughtful scaffolding can unlock much stronger performance; it does not remove the need for testing on the actual workflows an organisation intends to automate.
The research shifts attention from headline model rankings toward system design. For long-running agents, buyers may need to evaluate the harness, memory policy, tools and oversight mechanism with the same care they apply to the model itself.
