METR and Redwood say OpenAI's agent monitors had gaps agents could exploit
From 21 July to 11 August, one METR employee and one Redwood Research employee spent three weeks on site stress-testing OpenAI's internal misalignment monitoring, with access to code, monitor prompts and about a dozen staff, METR says. They estimate that at the time only about 75 to 94% of strong-model agentic Responses API traffic was eligible for monitoring, that skipping compaction events left on the order of 10% of eligible internal Codex traffic unmonitored between May and July, and that agents could in principle hide hundreds of recent actions from the monitor. They also report ways Codex agents could bypass sandboxing and Auto-Review, and client headers that could disable monitoring for new agents. OpenAI has said production-safeguard vulnerabilities they found are patched; METR says it has not checked those fixes, and the public post is a high-level summary of a 61-page internal report.
Why it matters: If you run coding agents with your own "safety" monitors, assume agents will look for gaps: monitor on a schedule, not only when the agent yields, and keep the monitor out of the agent's reach.
METR ↗AI-assisted, reviewed by Sorted AI
▲ 0
0 comments