Opera: A Language-Based Critic Framework for Long-Horizon Coding Agents

Headline: Opera Turns Coding Agent Feedback Into Persistent Checks And Higher Resolve Rates

Meta description: Opera makes coding agent feedback persistent, using typed operators and audits to raise resolve rates on Terminal-Bench 2.1, SWE-Bench Pro and DeepSWE v1.1.

Tags: Opera, coding agents, test-time critic, benchmark results

Featured image: abstract technology style, no real people, no company logos, no text

Background: The Feedback Loop That Coding Agents Miss

For long-horizon coding agents, the hard part is not only writing code. It is keeping track of what has been corrected, what still needs attention, and whether a fix actually changed the underlying problem. Opera coding agent feedback addresses that gap by treating each correction as a persistent note rather than a one-off message. In the latest reported results, that design shows measurable effects on resolve rates across Terminal-Bench 2.1, a SWE-Bench Pro subset, and DeepSWE v1.1, with gains of up to 12.4, 15.0, and 8.9 points across four policy models.

This matters because agent tasks unfold over many steps. A correction that points out a failed test or a mismatched interface may be accurate at the moment, but if the agent does not carry the correction forward, the same issue can reappear. The critic's value therefore depends on when it speaks, what it says, and whether the agent's later actions prove the issue was resolved.

Development: Turning Corrections Into Persistent Notes

Opera's process can be read as a timeline inside a single coding run. First, it decides when to review the work. The reported design uses periodic and event-driven triggers, so it does not only wait for a fixed interval; it can also react to notable moments in the agent's execution. Second, it diagnoses issues with typed operators. This gives corrections a more structured form than free text alone, while still remaining verbal feedback that the agent can act on.

Third, Opera audits feedback against visible evidence before sending it. That step is important for reliability: a critic that comments without checking the observable state may add noise. Fourth, it tracks the agent's subsequent actions to separate real fixes from mere compliance. An agent may edit code, rerun commands, or acknowledge the note, but the critic's role is to determine whether the diagnosed problem actually persists.

As a test-time critic, Opera is used while the agent is working. In the reported evaluations, it outperformed competitive critic baselines on all three benchmarks. It also helped when a policy critiques itself, suggesting that a separate critic can add structure even when the agent is already capable of self-review.

Latest Progress: Benchmark Gains And Near On-Policy Training Data

The most concrete numbers come from the latest reported test-time results. Across four policy models, Opera improved resolve rates by up to 12.4 points on Terminal-Bench 2.1, 15.0 points on a SWE-Bench Pro subset, and 8.9 points on DeepSWE v1.1. The reported figures describe increases in the share of tasks resolved.

The second development is training-data related. Opera's guided rollouts also serve as near on-policy training data. Near on-policy means the examples are close to the behavior the model is expected to produce, but not necessarily generated by that exact model. When Qwen3.5-9B was fine-tuned on these rollouts, held-out SWE-Bench Pro performance rose by 10.2 points without using a critic at inference. Held-out performance refers to evaluation on examples that were not part of training.

That result matters because it suggests the critic's process can improve the underlying policy, not just the final output during review. The reported comparison also notes that this fine-tuning matched the gain from fine-tuning on a stronger model's rollouts. The difference appeared when the evaluation harness changed: unlike the stronger-model rollout approach, the Opera-guided fine-tuned model preserved performance when switching from OpenHands to Terminus-2.

Current Stage: A Critic That Turns Feedback Into Evidence

The current stage is defined by reported benchmark and training results, not by a described deployment timeline. The numbers show two connected gains: better test-time correction and better training data. For readers, the incremental value is the shift from treating agent feedback as a transient comment to treating it as a persistent, auditable note that must be checked against later evidence.

The next thing to watch is how these benchmark gains translate into more varied coding tasks, longer sessions, and different harnesses. The OpenHands to Terminus-2 comparison is a concrete clue that transfer across execution environments may be a key test for critic systems. The reported gains across four policy models and three benchmarks establish the scale of the result, while the 10.2-point held-out SWE-Bench Pro improvement shows why the same feedback loop may also matter after training is finished.