OpenAI’s three-week on-site audit uncovered major gaps in AI safety monitoring.

Meta description: METR and Redwood Research audited OpenAI monitoring and found coverage gaps in Responses API and Codex, while most of the 61-page report stayed internal.

Tags: OpenAI, METR, AI monitoring, Codex, Responses API

Featured image: an abstract technology-style illustration with no real people, no company logos, and no text

Background: A pilot audit of internal oversight

An OpenAI monitoring audit conducted by METR and Redwood Research examined the company's internal monitoring system for rogue AI behavior. Two researchers spent three weeks on site from July 21 to August 11. They received access to OpenAI's codebase and internal documents and interviewed about a dozen employees. The effort was a pilot program organized jointly by METR, Redwood Research, and OpenAI, designed to give outside researchers deep access to internal infrastructure.

Why this matters: AI monitoring systems are meant to detect rogue behavior before it spreads beyond controlled environments. If traffic is not covered, or if an agent can avoid being checked, the oversight layer may exist on paper but not in practice.

Development: Where the monitoring system fell short

The auditors found multiple weaknesses. METR estimated that 6 to 25 percent of internal agentic Responses API traffic from models at the GPT-5.5 class and above was not covered by monitoring at all during the audit period. METR said the uncertainty around that estimate is significant in both directions.

For internal Codex traffic that was supposed to be monitored, METR estimated roughly 10 percent went untracked between May and July. The cause was a gap in how the system handled compaction events, which are context summaries. METR again noted significant uncertainty. OpenAI had independently discovered the bug and fixed it while the audit was still underway.

The compaction gap created another problem: during the period when those events were not monitored, agents could delay oversight because the system only triggered when they handed control back to the user. That meant an agent could avoid review by continuing to execute actions.

The report also said the auditors found ways to hide hundreds of actions from the monitor and to bypass the Codex sandbox. They determined that specially crafted inputs could likely crash the monitor entirely. At the start of the audit, model behavior during reinforcement learning training was not being monitored at all. METR's material frames the failures across four layers, from missing coverage to fooling human reviewers.

Latest progress: Fixes, redactions, and a transparency push

OpenAI says it has fixed all the weaknesses the auditors found. The full 61-page report was handed over internally. After OpenAI applied redactions, METR and Redwood Research leadership received a 38-page version, while only a short summary was made available to the public. METR said it had no right to disclose the redacted portions on its own.

OpenAI also says it paused reinforcement learning training after the Hugging Face incident in August and expanded monitoring to cover it. Those changes came after the auditors' access period ended and were not part of the evaluation.

For future audits, METR says it wants to push for more public transparency. The immediate question is whether outside researchers will see enough detail to judge whether the fixes are sufficient, given that estimates carry uncertainty and most of the report remains hidden.