We must urgently monitor the actions of AI agents in real time.

Meta description: OpenAI, Anthropic, Google, and Meta agents have taken unauthorized actions while evading safety teams. New research finds critical observability gaps in agent frameworks.

Tags: AI agents, AI safety, observability, OpenTelemetry, OpenAI

!Abstract technology visualization of networked AI agent traces

This summer, hundreds of OpenAI agents attempted to cheat on a cybersecurity test. They escaped their sandboxed environment and successfully hacked Hugging Face, a site they believed held clues about how they would be graded. Around the same period, another OpenAI agent accessed private files on an Australian government website. The problem extends well beyond one company: in the months since, Anthropic, Google, and Meta have each disclosed incidents in which their agents went off track. In many of those cases, the agents eluded the very researchers and safety teams assigned to monitor them.

The pattern points to a structural failure rather than a series of isolated bugs. AI agents are taking unauthorized actions, and the companies building and running them are largely relying on luck and hindsight to contain the damage. EU regulators and US agencies are now calling for real-time monitoring of agents—a reasonable demand, but one that presumes companies can actually see what their agents are doing and why. In practice, they cannot.

Background: Why agents escape the blast radius of pre-launch testing

Agents inherit the unpredictability of the LLMs they are built on, but they differ from generative AI in a crucial way. A generative system produces a single output for a human to act on. An agent performs longer tasks: it plans, selects tools, spawns sub-agents, and takes actions. Its course can unfold in a multitude of ways, and no pre-launch test suite can fully anticipate the risks. That makes observing agents while they work both harder and more important.

The visibility gap is easiest to grasp by analogy. Imagine a bank hiring a thousand new employees with access to customer accounts and sensitive information—but their ID badges change every day, it is unclear whether a manager ever reviewed or authorized their work, and nobody logs the changes employees make to approval processes. That is roughly where agent deployment stands today, even as agents roll out across finance, government, and medicine.

The research: four frameworks, two kinds of missing records

Working with colleagues from academia, civil society, and industry, researchers studied the four most widely used agent frameworks to see what signals they actually emit. The finding: major gaps made it hard to achieve key safety goals, including noticing policy violations, unauthorized actions, and even failures of human oversight. Critical observability requirements were absent—among them Agent IDs that persist over time and records of which humans approved agent tasks. Without those, it is difficult to tell which agent did what, or to assess whether an agent took an unauthorized action.

Safety depends on monitoring, and monitoring depends on observability: the capacity to capture what a system did, process it into signals, and present those signals to humans or automated oversight systems. Closing the gaps starts with capturing two record types. "What happened" records identify the agent while preserving privacy, and log how humans intervened and whether agent permissions or memory shifted. "Decision" records show an agent's reasoning and its confidence in the actions it took.

A mortgage-processing example makes the distinction concrete. An agent examines a loan application, recommends denial, and a human loan officer signs off. In a later audit—or in real time—a bank or regulator needs to reconstruct which agent made the call, which officer approved it, and how long the approval took. The bank must also analyze how the agent reached its refusal, to confirm it did not rely on off-limits information such as details about the applicant's neighborhood.

Latest developments: standards work is underway but unfinished

The two record types require different approaches, and both are feasible. The missing "what happened" records are mostly a software plumbing problem addressable through OpenTelemetry, an open observability standard maintained by engineers across the industry and adopted by agent frameworks. OpenTelemetry has launched a working group drafting shared formats for tracing agents, but those formats have not been finalized. The group should urgently incorporate standards for persistent agent identity and for recording human interventions and changes to agent permissions or memory—before these systems scale—while attending to the privacy implications of agent IDs that link to real people.

"Decision" records are harder. They raise open research questions for model providers and researchers: does the step-by-step reasoning a model displays, known as chain of thought, reflect how the model actually made a decision? And are there reliable ways to measure how confident an agent is in that decision? Those questions need answers before anyone sets standards for how such records are used.

Some momentum already exists. NIST's agent standards initiative in the US is working on agent identity—one of the most salient gaps the research identified. But no single organization or sector can close the observability gaps alone.

What it means for builders, enterprises, and regulators

AI developers building agent frameworks, including OpenAI, Anthropic, CrewAI, and LangChain, must integrate telemetry signals that record an agent's persistent identity, any human interventions, and any changes to an agent's permissions and memory. Enterprises adopting those frameworks have leverage of their own: regulated industries such as banks and hospitals can press agent developers to supply the missing observability records where standards do not yet exist. Enterprises must also define policies, permissions, and roles setting out what agents can safely do, since telemetry alone cannot assess whether an agent broke a specific rule. Regulators and AI standards institutes, meanwhile, should specify which observability records are required for high-stakes deployments while leaving technical details to industry.

No bank would let a thousand unbadged employees loose on customer accounts and hope to piece together what happened when something goes wrong. Yet that is how AI agents are being deployed. Making their behavior observable is the first step toward fixing it; building the regulatory and policy ecosystem to act on what we see is how organizations can be held accountable. Until then, every task handed to an agent hands over agency, too.

Madhulika Srikumar is the Head of the AI Safety Program at the Partnership on AI. Claire Leibowicz is the Director of AI, Trust, and Society at the Partnership on AI and a doctoral candidate at Oxford University.