Evidence-Based Oversight for Long-Horizon Agents: What Did the Agent Really Do?

Meta Description: AgentMonBench and EBG help users verify long-horizon AI agent decisions with evidence checkpoints, benchmark subsets, and structured behavior graphs.

Tags: #AIagents #LLMBenchmarks #ModelOversight #AgentMonBench #EBG

Featured Image: Abstract technology style: a translucent graph connecting code blocks, tool calls, and evidence nodes, with no text, no real people, and no company logos.

AgentMonBench and EBG address a growing gap in AI oversight: as agents complete multi-hour or multi-day workflows, users need a way to know which decisions actually changed the outcome. The paper introduces an evidence-grounded method for surfacing consequential choices instead of asking humans to read every tool call.

Background: Interpretability Moves Beyond Model Internals

Research on AI interpretability has traditionally concentrated on why a model makes an internal decision. That question still matters, but it no longer covers the full risk picture. AI agents are moving from short answers and simple tool calls toward workflows that can last for hours or days. In software development, data analysis, and scientific research, an agent can plan, invoke tools, modify code, and revise its strategy after seeing intermediate results.

This shift changes what oversight means. The model's internal reasoning may remain opaque, and the agent's external behavior can also become hard to track. A goal given to an agent does not contain every constraint needed to judge whether the result was correct.

Development: The Oversight Gap Appears

Three failure modes stand out in long-horizon execution. First, underspecified requirements let an agent fill missing assumptions on its own. Those assumptions may not match what the user intended. Second, silent behavioral deviations can occur when an agent changes a plan or implementation without telling anyone. Tests may still pass, but behavior may not match the original task. Third, consequential decisions may go unverified. A change to evaluation metrics, data filtering, or experimental settings can alter the final outcome even though it looks reasonable.

The evidence problem is practical. As a task grows longer, important details spread across chat history, tool calls, code repositories, and intermediate artifacts. Users cannot inspect every step. They may not know which decisions matter, when intervention is needed, or what follow-up instruction should be given. This makes traditional Human-in-the-Loop workflows fragile: a human can be present in the process without truly understanding what the agent did.

Latest Progress: AgentMonBench, EBG, and Codex Harness

The paper "What Did the Agent Actually Do? Evidence-Grounded Oversight for Long-Horizon Agents" is available under arXiv identifier 2610.06406. It frames the problem around two questions. One asks whether actual agent behavior aligns with user requirements. The other asks which consequential decisions occurred during autonomous execution and whether users can see the evidence behind them.

To test these questions, the authors introduce AgentMonBench, a benchmark with three subsets: SpecGAP, SilentSwap, and FeedbackTrace. The benchmark evaluates whether large language models can identify important decisions and locate the evidence that supports them. The code is published at https://github.com/zhk-lab/EBG, the dataset at https://huggingface.co/datasets/ZhaoHongKang/AgentMonBench, and the project page at https://zhk-lab.github.io/agentmonbench-ebg/.

The central method is the Evidence-Grounded Behavior Graph, or EBG. It converts scattered evidence from repositories and execution trajectories into a structured graph. The design is training-free. In experiments with 8 models, EBG improved consequential decision identification and evidence localization in most evaluation settings. Case studies involving real research workflows showed its potential to support verification and guide later agent execution.

The authors also integrate EBG into the Codex Harness. The integration adds three oversight checkpoints: before execution, after consequential execution changes, and before final reporting. These checkpoints are designed to help users clarify requirements, review important behavior changes, and verify outcomes from evidence rather than trust alone.

Current Stage: Evidence Before Full Transparency

The work marks a move from model-level interpretability toward long-horizon behavioral oversight. The open question is no longer only why a model reasoned in a certain way. It is what an agent actually did, which decisions shaped the result, and whether the evidence can be found quickly enough for a human to act. At this stage, the contribution is a benchmark, a graph-based method, and a harness integration that make those questions easier to ask. The next test will be whether oversight systems can consistently surface the few decisions that matter before an agent's work becomes too large to review.