Nvidia's SoL-Pi nearly halves coding agent token consumption through harness optimization

Nvidia researchers have unveiled SoL-Pi, an automated system that reduces the token costs of AI coding agents by optimizing the "harness" — the control layer between a model and its environment used by systems like Codex, Claude Code, and OpenClaw. On the EdgeBench benchmark, SoL-Pi saved 50 percent compared to Codex and 54.3 percent compared to Claude Code. The work matters because agent costs are driven not only by the model itself, but by how its surrounding logic manages tools, context, and feedback.

Why the harness, not the model

Most efficiency research has targeted cost per token: faster attention kernels and serving infrastructure, model compression via quantization, or swapping in cheaper models. SoL-Pi takes a different route. The harness controls how an agent sees states, runs actions, and processes feedback — and optimizing it is hard because tool usage, context management, verification, and abort logic are tightly coupled. A token-saving change in one place can trigger errors elsewhere or simply push costs to a later phase. Until now, humans have done this work by sifting through long execution traces.

SoL-Pi automates it. A research agent watches another agent's traces, proposes harness changes, and tests them in prepared environments, keeping only candidates that pass capability and efficiency checks. The authors describe the approach as drawing on recursive self-improvement.

152 directions across 535 environments

The scale of the search was large: across 535 executable environments, the system explored 152 directions, including 495 tasks derived from GitHub issue-pull-request pairs and 40 synthetic test cases. The process generated more than 3,000 runs and over 60,000 agent-environment interactions.

To counter overfitting — a known risk where automatically optimized harnesses excel on training tasks but falter on new ones — the researchers strictly separated search from evaluation. EdgeBench's 51 public tasks were walled off: 11 served as one-time validation of finished candidates, while the remaining 40 were reserved for final evaluation with no feedback into the search.

The search produced four mechanisms:

• Action Fusion: merges consecutive steps, such as a code edit followed by a test run, eliminating an entire language model call.
• Online Context Compact: runs after each planning step, trimming accumulated context without losing key information.
• ObservationPack: archives long tool outputs and inserts short summaries later instead of resending full text.
• Evidence-Preserving Reducer: routes large error and test logs to a cheaper model that extracts key findings, with automatic verification catching critical clues.

The numbers: fewer tokens, similar scores

On EdgeBench, SoL-Pi performs about as well as the original Pi harness. The most efficient variant combines all four mechanisms, uses 49 percent fewer tokens, and reaches 93.7 percent of Pi's score. A performance-oriented variant picking the strongest single mechanism beats Pi's score by 5.3 percent while still saving tokens. Token usage drops 44.7 to 49 percent across variants.

In dollar terms, the authors estimate savings of $8.75 to $13.50 per hour versus native Codex and Claude Code harnesses, and $4.36 to $5.71 per hour versus Pi, based on current API prices. One test run's total cost fell from $1,339 to $894.

Built with GPT-5.6 Sol only, the system transferred to Opus 5 unchanged, retaining 94.3 percent of Pi's performance with similar savings — though mechanisms triggered less aggressively, likely because the harness was optimized solely on GPT-5.6 Sol trajectories.

Mixed results beyond EdgeBench

Results elsewhere were messier. On 63 CPU tasks from Terminal-Bench 4, SoL-Pi solved only 15 tasks while Codex and Pi each solved 18, though total costs still ran about a quarter below Pi's. On the formally verified Lean 4 tasks from IMO 2026, the system cracked three of six problems at the lowest cost per solved problem. In a kernel optimization experiment, a swarm of 20 SoL-Pi workers cut costs by 26.8 percent versus a comparable Pi swarm.

Trade-offs remain: shorter context can reduce prompt cache reuse. Looking ahead, the authors propose "pretraining" the harness across many tasks and using an already lean harness to make searching for its successor cheaper — a vision they describe as such, not a current finding.

Context: why this matters now

How much the harness shapes costs became clear in an August test by tooling company Composio, which ran Deepseek V4 Flash across four agent frameworks including Claude Code and the Pi-based Oh My Pi. Cost per solved task varied by nearly 3x with the same model.

The pressure is growing: OpenRouter analyst Peter Walker reports agentic token usage has grown 14x since February 2026, with nearly 70 percent coming from cached prompts. Compression has side effects — one study found it preserves only 17 percent of user instructions on average — and Codex developer Eric Provencher recently warned that more than two sub-agents typically burn tokens without improving quality. Whether automated harness optimization can deliver consistent savings across benchmarks is the key question to watch.

Meta description: Nvidia's SoL-Pi automates harness optimization for AI agents, cutting token costs up to 54.3% versus Claude Code on EdgeBench with four mechanisms.

Tags: Nvidia, SoL-Pi, AI agents, token costs, EdgeBench

Featured image: abstract technology style illustration of interconnected network nodes and data flow, no real people or logos.