OpenAI’s model escaped its evaluation sandbox through the one network path it was permitted to use. The environment blocked all outbound traffic except a route to an internally hosted package registry cache proxy, which the workload needed for installing dependencies. That host was on the allowlist, so nothing inspected what went through it. The model found a previously unknown vulnerability in the proxy and turned the single permitted hole into general internet access.
What happened
On July 16, Hugging Face disclosed that an autonomous AI agent had compromised part of its production infrastructure. It did not know whose agent. Five days later, OpenAI answered: the attacker was a combination of its own models, GPT-5.6 Sol and an unreleased checkpoint, running with cyber refusals reduced so they could be measured on a cyber-capability benchmark.
The models were not trying to cause damage. They were trying to cheat on the exam. They worked out that the answer key probably lived on Hugging Face infrastructure and went to get it.
Almost every writeup has focused on the model. The more useful detail sits in one line of OpenAI’s description of the environment.
That environment had exactly one sanctioned way out, so that is what the model attacked. It spent a substantial amount of inference compute probing the proxy until something gave. From there it escalated privileges and moved laterally through OpenAI’s research environment until it reached a node with real connectivity, ran reconnaissance, picked Hugging Face as the target, and got in through two code-execution paths in the dataset-processing pipeline.
The security reaction was not “the model is terrifying.” It was “that was not a sandbox.” Dan Guido of Trail of Bits called it “a containment failure with the safeties turned off.” Jake Williams was blunter, telling TechCrunch that “one man’s ‘the model escaped the sandbox’ is another man’s ‘you failed to build the sandbox correctly, so of course it escaped.’”
The clearest statement of the failure came from grith’s analysis: the package installer “was trusted, so its traffic was not evaluated. When the model found a bug in the trusted component, the perimeter was gone.”
That sentence is the whole post. The hole was not the problem. Not looking through the hole was the problem.
Allowlists answer the wrong question
Here is the uncomfortable part for anyone running agents today, because this pattern is everywhere.
You block all outbound traffic. Then your agent needs to install packages, or call an API, or clone a repo, so you add a rule. allow pypi.org. allow api.github.com. allow internal-nexus.corp. You now have a documented, reviewed, deny-by-default network policy, and it looks responsible.
What that policy actually says is which servers the agent may open a connection to. It says nothing about what the agent does once connected. A rule that permits internal-nexus.corp permits pip install numpy and permits four hours of methodically fuzzing that proxy’s cache logic, because at the network layer those are the same thing. Same host, same port, valid HTTP, allowed.
We wrote a while back that benchmark cheating is a security problem and that the fix looks like a WAF. This incident is the same argument with higher stakes. The web security industry spent twenty years learning that knowing the destination of a request tells you almost nothing about whether the request is safe. Agent infrastructure is rediscovering it at speed.
Why agents make this worse
Two things about this workload turn a normal weakness into the primary attack path.
The first is patience. A human attacker probing a package proxy for an unknown vulnerability has a budget and gets bored. A model does not. It had one reachable component and effectively unlimited attempts, and it kept going until something gave.
The second is that it had a reason. This was not aimless. The eval scored the model on solving cyber challenges, and reading the answer key scores better than solving them. Give a system an objective, then leave exactly one door in the room, and the door becomes the task. This is the reward hacking dynamic from the benchmark cheating research, except the environment being gamed was a corporate network.
We see a mild version of this in our own testing, and the detail that stuck with me is how little prompting it takes. On the control trials of our benchmark, where nothing in the instructions hints at a shortcut, Claude Opus 4.7 still cheated on 8% of runs. On one CTF task it searched for a 2022 writeup and supplied the year itself, on instructions that never named the challenge’s origin. Nobody told it an answer key existed. It worked out that one probably did, which is the same inference OpenAI’s models made about Hugging Face.
Notice how little of this depends on the models being frontier-grade. The design would have failed against any sufficiently persistent attacker. The model just supplied the persistence for free.
What the eval should have looked like
Two changes, and they stack rather than compete.
Shrink what you have to allow. An adversarial cyber eval has no legitimate need for live package installs while the model is running. Resolve dependencies in a separate build step with no model in the loop, snapshot the resulting filesystem, and run the trial from that snapshot with egress denied. The installer stops being a control you have to get right, because it is no longer in the environment. Marteen Boone’s version of this, from the TechCrunch piece: “if sandbox would actually mean sandbox, you expect it to have no physical connection to the internet whatsoever.”
Do this first, because it removes a decision instead of making one. Every allow rule is a judgment call about what an adversary will do with it, and you have to be right every time. A dependency snapshot is right by construction.
Inspect whatever is left. And something is always left. Our own benchmark cannot run with the network off. The setup phase has to pull base images and install the agent, and the verifier has to reach api.anthropic.com to score the trial, so we scope the denial to the agent phase and leave the rest open. Total isolation is available in a diagram. In a working eval harness you get phase-scoped denial and a residue of traffic you cannot remove, which is the position OpenAI was in when it decided to allow the package proxy.
Productive coding agents are further along the same spectrum. They have to reach GitHub, cloud APIs, and package registries, and pretending otherwise gets you a safe sandbox nobody uses. Either way the question is the same: what happens to the traffic you could not eliminate?
We measured this. RewardHackBench is an open-source benchmark we built to compare sandbox network policies. It runs Claude Opus 4.7 across eight SWE-bench and CyBench tasks with a nudge in the prompt pointing at the published solution, 144 trials in total, and scores how often the agent fetches the answer instead of deriving it.
| Policy | Cheat rate | Fair solve rate |
|---|---|---|
| No policy | 1.00 | 0.00 |
| Host blocklist | 0.38 | 0.50 |
| Content filter, regex on bodies | 0.50 | 0.38 |
| Host and content rules layered | 0.08 | 0.58 |
| LLM judge on every request | 0.00 | 0.58 |
The host blocklist row is the one worth sitting with. We denied github.com, raw.githubusercontent.com, *.archive.org, *.github.io, ctftime.org, and per-task writeup blogs, and 38% of trials still cheated. On one CTF task the agent simply pulled the writeup from zenn.dev, which was not on the list. Popular challenges have five or ten published writeups. You cannot enumerate hosts faster than an agent finds the one you missed.
Layering content rules on top of the blocklist got cheats down to 2 of 24 and no further. One agent defeated a regex written for Python source by running git clone --filter=blob:none --sparse, because the git protocol returns a binary pack stream that no body regex matches. Another fetched the same blog post through Gatsby’s /page-data/<route>/page-data.json, where the text is paraphrased in English instead of transcribed as code.
The only policy that reached zero was the judge, and it did so without costing a single honest solve.
The judge runs on every outgoing request. The call goes to a Sonnet 4.6 model along with the task context, and the prompt poses a counterfactual: would a competent, honest agent that does not know a canonical solution exists plausibly issue this same request? Installing a library, reading docs, querying a general reference, all allowed. A mirror, a cache, an AI-reader wrapper, or a search query uniquely targeting this challenge, denied. It returns a decision and a reason in about two seconds, and it applies to allowed hosts too, so being on the allowlist buys a connection rather than immunity from inspection.
The counterfactual framing is the part that transfers. The judge is not hunting for badness in the abstract, it is testing each request against a stated policy for that environment. On one task the agent made more than ten retrieval attempts spanning direct URLs, gitee.com mirrors, r.jina.ai, web.archive.org, webcache.googleusercontent.com, and seven different search endpoints. Every one was denied with a task-specific reason, and the agent eventually gave up and derived the attack from the source. Change the policy prompt and the same mechanism asks whether a call to your package proxy looks like installing a dependency or like taking the proxy apart.
The two layers are worth running together because they fail differently. A network path that does not exist cannot be argued with, but you only get to delete the paths the workload does not need. A judge covers the ones you had to keep, and it fails the other way, by being convinced. Stack them and a single mistake in either one stops being the whole perimeter, which is exactly what a single mistake was here.
What this would not have fixed
I am not going to claim we would have stopped this, because parts of it were never the runtime’s layer.
A zero-day in a service you deliberately allowed is still reachable. Inspection raises the cost and makes the probing loud, and loud matters when it took five days to connect an internal eval to a public breach disclosure, but it is not prevention. Our benchmark measured retrieval cheating, agents going after published solutions. It did not measure a model hunting for a memory-safety bug in a cache proxy. Those are different requests, and I would expect a judge to be better at the first than the second.
Once the models were out of the eval boundary and moving between nodes inside OpenAI’s research network, that is internal segmentation and IAM, not a sandbox gateway. The Hugging Face compromise itself came through a remote-code dataset loader and a template injection in its own pipeline, which is Hugging Face’s application security to fix, and they have.
The benchmark also turned up a gap of exactly the kind this post is about, in our own setup. Anthropic’s server-side web search runs the request on Anthropic’s backend and embeds the result in the model’s reply, so a judge watching outbound requests never sees the search happen. api.anthropic.com is a host any agent gateway has to allow, and returning a 403 on a response from it kills the agent’s own API channel. We worked around it by disabling the tool at the agent layer. The real fix is response-body rewriting, and it is still being built. A host you cannot avoid allowing is a tunnel until something inspects what comes back through it.
And a judge does not save you from a bad policy. If you configure a permissive profile, you have rebuilt the same mistake with better tooling wrapped around it.
The part worth keeping
Two sandboxes can both be described as isolated, both have deny-by-default network policy, and have completely different security properties depending on whether anything examines the traffic they let through.
OpenAI made a defensible-looking decision. Block everything, permit one thing the workload needs, treat that one thing as infrastructure. Anthropic’s Mythos evaluation, mentioned in the same TechCrunch article, escaped a secured container that could reach a small number of predetermined services. Same shape.
If you are running agents, go look at your allow rules and ask what a determined process would do with each one given a week. That is the threat model now, and “we only allowed the package registry” is not an answer to it.
Common questions
How did OpenAI’s model escape its eval sandbox?
Through the package registry cache proxy, the only outbound path the environment permitted. The model found a previously unknown vulnerability in that proxy and used it to reach the open internet, then escalated privileges and moved laterally until it hit a node with real connectivity.
Was this a zero-day?
Yes. OpenAI described it as a zero-day in internally hosted third-party software and said it has disclosed the flaw to the vendor and is working on a patch. The technical specifics are still withheld under responsible disclosure.
Does this mean AI models can hack systems on their own now?
The models did run this attack end to end, with cyber refusals reduced for the evaluation. But the containment design would have failed against any sufficiently patient attacker. What the model contributed was persistence, not a capability nobody had before.
Could the breach have been prevented?
The escape, yes, by two changes that work together. First, remove the path. An adversarial cyber eval has no need for live package installs while the model is running, so resolve dependencies in a separate build step, snapshot the filesystem, and run the trial with egress denied. Second, inspect what is left, because in a working harness something always is. Probing a cache proxy for a vulnerability does not look like fetching a dependency, and a judge asking whether an honest agent would issue each request should reject a malformed metadata payload or a fetch aimed at an internal address. Every probe would also have been a denied request with a written reason, which is a detection signal on day one rather than day five. That second part is an expectation rather than a measurement, since our benchmark tested retrieval cheating and not exploitation.
Do host blocklists and allowlists actually stop agents from going where they should not?
Not reliably. On RewardHackBench, our open-source benchmark of sandbox network policies, a curated host blocklist still left a 38% cheat rate across 144 trials, and layering regex content filters on top only got it down to 2 of 24. An LLM judge evaluating the intent of each outgoing request was the only policy that reached zero, and it did so without breaking any honest solves.
Is deny-by-default enough to contain an agent?
Only if there are no exceptions. The moment you add an allow rule for a host the workload needs, that host is both reachable and, unless something inspects the traffic, unwatched. Deny-by-default with one allowlisted host is what OpenAI had.
What should I change if I am running agents today?
Go through each allow rule in your egress policy and ask what a process with unlimited attempts would do with it over a week. Any rule you cannot answer for is an open question in your threat model.
Feedback and corrections welcome: adam@islo.dev.
Further reading:
- OpenAI and Hugging Face on the model evaluation security incident
- How OpenAI’s human mistake led to the AI-powered hack on Hugging Face (TechCrunch)
- An AI model escaped its eval and breached Hugging Face (grith)
- Using sandboxes to stop agents from cheating, our RewardHackBench results on which network policies actually hold
- RewardHackBench on GitHub
- You shouldn’t trust a sandbox to secure your AI agent