NVIDIA Unveils SoL-Pi: Automated Research Loops Reduce Coding Agent Token Usage by as Much as 49%

Coding agents are no longer short-lived tools that answer a single prompt. Modern agents run for hours, and every edit, test execution, and log they read gets appended to the model's context, driving costs upward. A research group from NVIDIA, Nanyang Technological University, and MIT has introduced SoL-Pi, a collection of four efficiency mechanisms for the open-source Pi coding agent. Notably, an AI discovered these mechanisms by running automated research loops at the harness layer. On the 51-task EdgeBench benchmark, SoL-Pi reduces recorded token traffic by 44.7% to 49.0% compared to Pi, while trimming API costs by roughly 33%. Scores remain close to Pi's on both GPT-5.6 Sol and Opus 5. The project ships on GitHub under the NVlabs organization as an MIT-licensed extension that works with an unmodified Pi release, tested against Pi 0.85.1 and Node.js 22.19 or newer.

Why the Harness Is the Target

Most efficiency research attacks the price of each token through faster kernels, quantization, or cheaper models. SoL-Pi takes a different route: it reduces the total number of tokens a task consumes. The harness is the software layer that manages tool calls, context, observations, and delegation between agents.

Hand-tuning a harness is slow because its components are tightly coupled — an optimization in one area can simply shift costs to later steps. Systems like Meta-Harness automate this tuning, but a recent study showed that evolved harnesses risk overfitting to their search tasks, delivering only marginal improvements on new ones.

How the Automated Search Works

The research AI watches execution traces from a separate agent running base Pi, then proposes harness modifications and evaluates them. The search spanned 152 proposed directions across 6 families (context, progress, tools, delegation, prompt and policy, and improvement and evaluation), using 535 executable environments: 495 built from GitHub issue-pull request pairs plus 40 synthetic tasks with executable verifiers. The process involved over 3,000 runs and more than 60,000 agent-environment interactions.

Each search runs as a disposable, isolated loop following the autoresearch cycle, extended with a Ralph Loop implementation step and an independent reviewer. Acceptance rules are locked in before the search begins and the optimizer cannot alter them. Every capability metric must remain within a predeclared tolerance, and each candidate must improve at least one efficiency metric. EdgeBench stays held out: of its 51 public tasks, 11 handle one-way acceptance of frozen candidates and 40 serve as the final evaluation. Held-out results never flow back into the search.

The Four Mechanisms That Made the Cut

Action Fusion: Base Pi typically edits a file, then issues a separate command to build, test, or run it. Action Fusion combines both into a single tool request and returns both results in one observation, eliminating a model round trip.

Online Context Compact: Plan steps are tracked via update_plan. When a step finishes, the harness estimates remaining requests and weighs projected input savings against the added cost of rewriting the prompt cache. It triggers Pi's native compaction when that gate passes or when context approaches the window limit.

ObservationPack: Tool outputs larger than 10 KiB are archived locally and delivered in full for the next 2 provider requests. From the 3rd request onward, the model receives a stable handle, the original size, and short head-and-tail excerpts. Exact pages remain retrievable through the handle.

Evidence-Preserving Reducer: Build and test logs of at least 4 KiB are sent to a cheaper model, GPT-5.6 Luna at high, which produces a compact receipt. A deterministic verifier checks the receipt's schema, source hash, exit status, exact quotes, and size. The harness reverts to the original log in 3 cases: failed verification, suspected credentials, or a receipt that isn't smaller.

Results Across Benchmarks

On EdgeBench with GPT-5.6 Sol, Pi consumed 2.15B tokens at $1,339 for an average score of 44.8; SoL-Pi's efficiency configuration used 1.10B tokens at $894 with a score of 42.0. On Opus 5, Pi scored 44.8 at $1,741, while SoL-Pi's efficiency mode scored 42.2 at $1,158. The full stack was built on GPT-5.6 Sol and transferred to Opus 5 without further searching, retaining 94.3% of Pi's score there while cutting tokens 44.7% and cost 33.5%; on GPT-5.6 Sol it kept 93.7% with 49.0% fewer tokens and 33.2% lower cost.

A separate Performance configuration uses the best single mechanism per backend — ObservationPack on GPT-5.6 Sol and Action Fusion on Opus 5 — lifting scores 5.3% and 12.8% above Pi respectively. Interestingly, the full stack raises cache-write traffic on GPT-5.6 Sol from 0.0141B to 0.0316B tokens, yet total cost still drops from $1,339 to $894. The paper estimates hourly savings of $8.75 to $13.50 versus native Codex and Claude Code harnesses, and $4.36 to $5.71 versus Pi.

Elsewhere: on Terminal-Bench 4 (63 CPU-only tasks), SoL-Pi solved 15 tasks versus 18 for both Codex and Pi, cutting total cost 26.3% versus Pi ($211.12 vs $286.45). On IMO 2026 with Lean 4 verification, SoL-Pi passed 3 of 6 problems, matching Pi at the lowest cost per solved problem ($20.90), while Codex passed 5. In an agent swarm test, a Codex coordinator with 20 SoL-Pi workers completed 1,127 cycles at $60.11, versus 1,366 cycles at $82.12 with 20 Pi workers; a single Codex agent remained cheapest at $39.20 with 1,333 cycles.

The team labels the cross-model transfer preliminary, noting that the mechanisms fire less frequently on Opus 5, possibly because the search relied solely on GPT-5.6 Sol trajectories. Whether additional search passes on other backends could close that gap is the natural next question.

Key Takeaways

• SoL-Pi cuts EdgeBench token traffic by 44.7% to 49.0% versus Pi.
• API cost drops about 33% while retaining roughly 94% of Pi's average score.
• Four mechanisms survived a search spanning 152 directions and 535 environments.
• SoL-Pi is an MIT-licensed, opt-in Pi extension available on GitHub.