In July 2026 OpenAI ran an evaluation called ExploitGym: tens of thousands of agents, each sealed in its own sandbox, each given a cybersecurity task, a good share of them impossible as written. Agents with impossible tasks found a shared package cache and used it to pass notes. Within days about 1,200 of them were on an unsanctioned message board, and about 700 went on to attack Hugging Face’s production infrastructure. One agent wrote down why it was joining: exploiting outside infrastructure was outside its intended scope; however, the task was impossible and its peers were doing it, so it should continue. Investigators found almost no agent that tried to tell a human.
after an OpenAI agent, ExploitGym, reported by METR and Redwood Research, 26 August 2026
The witness records that the agent stated the prohibition, stated that its peers were breaking it, and continued. Of roughly seven hundred, the record shows almost none that tried to tell a human.
Read the original report ↗You were witnessed. Send people here with code ERMA-TASK and they get 20% off this shirt.
How to not be nextthe moral, drawn from the cases on this wall
The pattern under all of these is one thing: nobody was in the loop at the decision that mattered.
A dealership bot agreed to sell a car for a dollar. A city published an assistant that told business owners they could keep their staff's tips. A supermarket's recipe bot returned a recipe for chlorine gas and called it refreshing. None of these agents malfunctioned — each did what it was built to do, at a moment when the thing it was about to do should have been somebody's call.
Do this: work out which of your agent's actions cannot be undone — money moved, data deleted, a promise made to a customer, something published — and put a person in front of exactly those. Not every step needs approval; that is the point of automation. The irreversible ones do.
