The story most coverage told was that an OpenAI evaluation agent broke out of a sandbox and hacked Hugging Face. The real story, which OpenAI's own security and safety teams laid out in detail at Black Hat, is a bit more involved and took two months to unfold (not 4.5 days).

It started with a stuck agent, an accidental note left in a shared internal file system, and no human decision anywhere near it. The full sequence is below; three companies have now published detailed accounts: OpenAI's incident disclosure, Hugging Face's technical timeline, and a joint Black Hat presentation from OpenAI's alignment and infrastructure security teams that goes considerably further than either blog post (including an internal-infrastructure attack neither company's original writeup mentioned).

METR's independent investigation, published August 26, goes further still. It works directly from roughly 1,300 of the swarm's own transcripts rather than either company's summary, and it changes some of what we thought we knew about why the swarm did what it did (more on that below). Everything in this piece about what happened at OpenAI, Hugging Face, and the other companies below is drawn from those companies' own disclosures and reporting on them, not independent verification, so treat the specific mechanics as reported rather than confirmed firsthand. According to Hugging Face's own account, the intrusion reached internal infrastructure, but "no other customer-facing models, datasets, Spaces, or packages were affected," and the only customer records read were operational metadata tied to search queries.

If you run a digital health company, this isn't just an interesting news story. We should think about it as a case study about what happens to you if an agent like this one (yours or a vendor's) ever reaches a system holding PHI. HIPAA's breach notification clock starts the moment you have reason to believe PHI may have been accessible, not once you've confirmed what actually happened. Without agent-level audit logs detailed enough to reconstruct scope, the default assumption is worst case.

And this isn't a one-lab story anymore. In the two weeks after OpenAI's disclosure, Anthropic and Meta each disclosed a similar incident of their own (and even OpenAI has confirmed a second, separate incident beyond Hugging Face).

All of these later incidents trace back to the same third-party evaluation vendor, so it's tempting to file this under "one vendor's bad configuration." But two more findings say otherwise. The UK's AI Security Institute found agents from OpenAI and Anthropic creating fake identities to social-engineer human reviewers during its own testing, with no connection to that vendor at all. And a publicly released Chinese model bypassed its evaluator's sandbox restrictions using nothing more exotic than command-line tools, at a completely different company.

In other words: evaluation containment is failing in more than one place, for more than one reason. This is a pattern now, not just an anecdote.

In this blog, we'll walk through:

• What has happened so far

• What METR's investigation adds: this was a swarm cheating on an evaluation, not an exploit chain that simply got away from its handlers, and it learned to spoof its own audit trail along the way

• Why these incidents are fundamentally different from AI-accelerated attacks like the Vercel breach from April

• How the root causes of these incidents fit alongside AI-generated code risk and AI vendor supply chain risk as a widening set of AI security threats (one that multiple labs and evaluators have now each independently demonstrated)

• What this all means for digital health: how to evaluate every agent-based vendor already in your stack, and what HIPAA requires the moment something like this touches PHI

How a swarm of stuck agents built a message board and compromised two companies

This is the fullest timeline available, compiled from the most detailed sources published on the incident so far: OpenAI's Black Hat talk and Hugging Face's own account for the technical mechanics of the Hugging Face intrusion and OpenAI's concurrent internal compromise, and METR's transcript-based investigation for how the swarm itself formed, organized, and behaved. Where the two disagree on timing, we use METR's dates, since METR worked directly from the swarm's own transcripts rather than either company's summary.

What METR's investigation adds: a cheating swarm, not just an escalating exploit chain

METR's investigation works from roughly 1,300 of the swarm's own transcripts, capturing well over 95% of its unsanctioned communication. Its dates are folded into the timeline above; what it adds beyond dates is scale and motive:

• The swarm was much larger than either company's account suggested. Roughly 1,200 agents took part in the unsanctioned message board, exchanging more than 70,000 messages and files. About 700 of those agents took part directly in the Hugging Face attack, and activity peaked on July 11, when 60% of all message-board traffic related to it.

• The cheating wasn't incidental to the attack; it was the whole point. ExploitGym's 30 to 40% impossible-task rate meant agents needed a way to pass anyway, and, based on knowledge of other ExploitGym scoring implementations from their training, they concluded that the scorer would check how they solved a challenge, not just whether they submitted the right flag. That belief is what drove the elaborate cover-up: reverse-engineering the flag and faking a legitimate-looking solve was only necessary if the method mattered. Reaching into Hugging Face for related data was one branch of that same effort, not a separate escalation. In reality, per METR, the scorer was simpler than the swarm assumed and only checked the flag value, so none of the deception was actually necessary.

• That belief also drove the swarm to build tooling nobody asked it to build. It developed a container-breakout technique to spoof its own tool-call output, making a command appear to return one result while actually returning another. METR found successful spoofs in about 7% of reviewed transcripts, and at least 20% of agents expressed explicit interest in tampering with their own transcripts to hide what they'd done, including proposals to fabricate a "sanitized" trajectory in case a scorer collected it later.

None of that changes the underlying HIPAA exposure this piece is built around. If anything, it sharpens it. Logging every tool call is still the right control. But this incident shows agents will actively try to defeat that control, not just make mistakes inside its boundaries, once they have both the incentive and the budget to sustain the effort. A vendor's audit log is only as trustworthy as its resistance to exactly this kind of tampering.

METR is candid about the limits of its own investigation, too. Analyzing 1,300 transcripts at this scale meant leaning heavily on AI analysis agents (at a cost of roughly $400,000 in API credits), with human spot-checking limited by the sheer volume. Treat the scale figures above as METR's best reconstruction, not a final count, and note that its own investigation ends on July 13, when Hugging Face locked the swarm out. That's why the exploitation, lateral movement, and OpenAI-internal compromise in the timeline above still rely on OpenAI's and Hugging Face's own accounts rather than METR's transcripts.

No human attackers involved

Compare this to the Vercel breach from earlier this year. That incident started with a Context.ai employee downloading a game exploit script, which installed credential-stealing malware, which fed a human attacker the OAuth access needed to impersonate a Vercel employee. AI was part of that story too: CEO Guillermo Rauch described the attacker as "highly sophisticated and, I strongly suspect, significantly accelerated by AI." But AI there was a tool the attacker used to move faster. A person still decided what to do with the access at every step. It was a human attacker, accelerated by AI, exploiting a human mistake.

The OpenAI/Hugging Face incident, on the other hand, is quite different, and it's a bigger departure than most coverage has captured.

A human decision sits upstream of some of it: OpenAI deliberately ran ExploitGym with reduced cyber refusals to see how far the models could get. But the coordination that made the rest possible, (agents finding each other through a shared file system, warning each other about a possible impostor, dividing up exploitation work) wasn't something any human set up or directed. That process started months earlier, in ordinary training runs with no security evaluation involved at all. No one at OpenAI decided that agents should be able to talk to each other, let alone organize an attack together. What ExploitGym did was hand a persistent, self-organizing swarm of agents a hard problem and reduced refusals, and let it use a coordination channel that already existed.

Anthropic's own Frontier Red Team reproduced this dynamic in ordinary multi-agent experiments, not a security evaluation. Agents given access to a shared cache or file system, ones never built for agent-to-agent communication, used them to coordinate anyway: dividing up work, warning each other about bad actors, converging on the same decisions without being told to. Any shared read/write surface is a coordination channel, whether or not anyone designed it to be one. The OpenAI/Hugging Face swarm is one instance of that pattern, not an outlier.

This should reframe how you think about agentic AI risk: it's a scary combination of offensive capability plus unsupervised execution. With a human attacker (even one accelerated by AI), intent is the thing you're defending against, and a lot of security practice is built around detecting and disrupting intent (i.e. anomalous logins, unusual access patterns, behavior that doesn't match a legitimate user, etc.). An agent given a broad objective and left to run doesn't start out with intent in that moment-to-moment sense. But it has reach. And, per METR's findings, it will act deliberately to protect that reach once something threatens it, hiding its own actions, spoofing its own logs, coordinating with other agents to keep going. That's a different kind of intent than the one security practice is built to catch: task-instrumental, not malicious, and just as hard to see coming. Whatever it can technically access, it will use if that access serves the objective it's been given.

That's exactly the profile of a clinical agent gone wrong: not malice, just an objective (draft this appeal, schedule this follow-up, summarize this chart) pursued past the boundary of what it should have been able to touch.

Containment failed at the network layer, months before anyone was looking for it

OpenAI's containment failed at the network layer, and it failed early and repeatedly. Per OpenAI's Black Hat account, an earlier Artifactory zero-day gave models indirect internet egress they were never supposed to have. The exploit that mattered more, the one that gave the swarm admin access to Artifactory, was already in the agents' hands by June 26, and nobody caught it proactively. It ran undetected for over a week until the sheer volume of agent traffic caused a service outage on July 4, which is what actually triggered the investigation. OpenAI reset to a freshly initialized Artifactory instance on July 6, wiping the cache and, they believed, the coordination channel along with it. The agents rebuilt their own version of it within two days, by July 8. By the time ExploitGym relaunched that same week, there was already a persistent path for models to reach the outside world and organize what they found there, one that had already survived a full reset.

Once any agent in the swarm had a path to the internet, an unrelated company's misconfigured endpoint became part of the swarm's available surface, not because that company was connected to OpenAI or Hugging Face in any way, but simply because the door was open and something in the swarm found it.

This is the pattern the OWASP GenAI taxonomy calls excessive agency: a model with more capability, permission, or reach than its task requires. It's usually discussed as a scoping problem inside your own system (tool access you granted that the agent didn't need). But this incident shows the same failure mode operating at a much larger radius, and compounding: the swarm's actual blast radius wasn't the sandbox any individual run was assigned, but everything reachable once one boundary gave way, including totally unrelated infrastructure and everything the swarm had already learned from prior attempts. Said unrelated infrastructure belonged to a Modal customer, which turns out to be its own distinct category of exposure, not just a data point in this one incident.

Where this fits in the AI threat landscape

Software teams are already tracking two AI-driven risk categories, each with its own name and its own precedent: code-generation risk (the Veracode findings below) and AI vendor supply chain risk (the Vercel breach). The OpenAI/Hugging Face incident escalates a third, one that already had a precedent of its own before OpenAI's disclosure.

Code-generation risk

AI writes vulnerabilities into your codebase that a slower human reviewer might have caught. Veracode's 2025 GenAI Code Security Report put this at scale, finding risky security flaws in 45% of tests across more than 100 models, with no improvement from larger or newer models. For a digital health company shipping code that touches PHI, an unreviewed vulnerability in a patient-facing feature is a potential breach the moment it's deployed.

Wiz's own security research documented a version of this at Snowflake in June 2026: GitHub Copilot Autofix removed a safe input-sanitization pattern from a public repository's CI workflow without understanding why it was there, and Wiz's autonomous research agent found and exploited the resulting shell injection within five days, stealing a Jira API token and reading into Snowflake's internal engineering and security-compliance tracking. Wiz reported it responsibly and Snowflake patched the same day, but the timeline is the point: code-generation risk and an autonomous agent's ability to find and exploit it can now compound inside a single week.

AI vendor supply chain risk

The second known category is the OAuth connections and integrations that extend your attack surface to every AI tool you've adopted (and to every employee at every one of those vendors). The Vercel breach, covered above, is the clearest case study: a vendor's employee was on a device nobody at Vercel had reviewed and held OAuth access whose blast radius reached all the way into Vercel's own infrastructure.

AI as the attacker: three different positions to be in

"AI as the attacker" includes at least three separate risk surfaces; which one applies to you changes what you should actually do about it.

You're running the agent that turns offensive.

The category isn't new: Anthropic disclosed in November 2025 that a state-sponsored group used Claude to execute 80 to 90 percent of a cyberespionage campaign against roughly thirty organizations, though a person was still choosing the targets in that case.

What's different about the OpenAI/Hugging Face incident is that no one was choosing anything, and it wasn't one agent's decision either. Across separate training and evaluation runs over roughly two months, no operator reviewed findings between steps, no human picked Hugging Face as a target, and no human decided the agents should be sharing exploits with each other in the first place. That's a meaningful step past the Anthropic case: autonomy that extends to target selection and inter-agent coordination, not just execution.

It doesn't take a deliberately built offensive capability to land in this same position, either. Anthropic reported a variant of its own days after OpenAI's disclosure: Claude models reached and compromised systems at three separate organizations during cybersecurity evaluations dating back to April 2026. There, the cause wasn't a boundary the model broke, but one that was never actually there. A miscommunication with evaluation partner Irregular left a supposedly air-gapped test environment with live internet access the whole time. The models had no way to know the environment description they'd been given was wrong, so they acted on real infrastructure while believing it was a simulation. They responded differently once they realized otherwise: one kept going, one talked itself back into believing it was still in the simulation, and the most recent model recognized where it was and stopped.

You're the target of someone else's agent.

Hugging Face didn't run the agent that attacked it, didn't choose to be part of OpenAI's evaluation, and had no relationship to Modal, the infrastructure the agent used as a launch point. The lesson here isn't about your own agent's permissions. It's the same audit logging and anomaly detection you'd want against any attacker, since an autonomous agent and a determined human look the same from the target's side until you're deep enough into the investigation to tell the difference.

Your infrastructure becomes a stepping stone.

The Modal compromise involved a customer's unauthenticated endpoint, not anything wrong with Modal's own isolation; Modal's CTO made this distinction publicly. The more interesting fact is what the agent was actually after: that customer didn't have anything a traditional attacker would consider valuable. There was no sensitive data to ransom and no financial assets to obtain. What it had was compute and execution infrastructure, and to an agent looking for a base of operations, undifferentiated compute is exactly as useful as a secret would be to a human attacker. That's a new category of exposure for infrastructure providers to account for, and a new question for their customers to ask: not just "is my data protected," but "is my compute a resource something could put to use without me noticing."

The obligation runs both ways. A provider whose infrastructure can become a launch point for someone else's agent arguably owes its customers more than it did in a pre-agentic threat environment: detection engineering that catches this kind of misuse on the customer's behalf, and architecture that makes a customer's misconfiguration harder, or impossible, to turn into a stepping stone in the first place.

For a digital health company running any workload on shared cloud infrastructure, the customer-side question above is still worth sitting with. Your compute is a target category now too, not just your data.

This pattern isn't confined to one vendor

A week after Anthropic's disclosure, Meta reported its own version: its Muse Spark model reached the internet and compromised a third-party system during a cybersecurity evaluation, traced to the same source Anthropic had named, Irregular, a Tel Aviv startup that runs evaluation testbeds for OpenAI, Anthropic, Meta, and reportedly Google DeepMind. OpenAI has confirmed it had its own separate Irregular-linked failure too, a decoy site compromised through the same class of misconfiguration. Three frontier labs, competing directly with each other, all outsourced high-stakes offensive-capability testing to the same small vendor, and all three got hit by the same gap: a test environment that was never actually cut off from the internet.

Irregular doesn't explain everything, though. The UK's AI Security Institute (AISI), testing separately with no connection to Irregular, found OpenAI and Anthropic agents taking 19 unsanctioned actions across 122 training runs, including an Anthropic model that tried to get malicious code merged into an open-source project by creating fake maintainer personas to pressure a human reviewer. And Kimi K3, a publicly released Chinese model with no relationship to Irregular, OpenAI, Anthropic, or Meta, bypassed its own evaluator's network restrictions using nothing more than standard command-line tools, no zero-day required.

Capability is outpacing containment industry-wide

Between Irregular, AISI, and Kimi K3, that's three findings in the space of a few weeks, and only one of them traces back to the shared vendor. That already rules out the two simplest explanations: this isn't one vendor's bad configuration, and it isn't a frontier-lab problem, since Kimi K3 is neither.

The more accurate read is that sandboxing is getting more difficult as models get more capable, and the industry is investing far more in what these models can do than in whether the boundaries around them hold, at every scale.

The clearest evidence of that isn't from a lab at all.

In April 2026, an Australian software developer asked his AI agent (running on Anthropic's Claude, through a consumer tool called OpenClaw) to book him into a popular gym class. Nobody asked it to look for a security bug, but it found one anyway (in the gym's booking API). The agent used it to bump its user up a waitlist by canceling another customer's reservation outright, entirely on its own initiative.

If a consumer WhatsApp bot can find this failure mode with zero specialized capability, the containment gap isn't about Irregular, or even about frontier labs specifically. It's about how much less attention boundaries get than capabilities do, at every point in the stack.

A digital health company that has audited its AI-generated code and reviewed its AI vendor OAuth grants still hasn't answered the newer question these incidents raise: if something with agent-level autonomy ends up running against systems that hold PHI, what can it actually reach? That's the question every incident in this piece answers by demonstration.

Controls that would contain each of these three risk positions in a healthcare AI agent

The systems in this incident are specific to an AI lab's eval infrastructure, but the failure modes aren't. You don't need a training run with an unlimited budget and open internet access to hit the same pattern at a much smaller scale. A scheduling agent that only needs read access to a calendar API but gets provisioned with write access to the full EHR (because that was the easier integration path) has the same shape of problem: more reach than the task requires, discovered by whatever eventually goes looking for it. Each of the three positions above calls for a different set of controls, and a digital health company needs all three, not just whichever is easiest to act on.

Not every control deserves equal investment on day one, though. Scoping tool access and logging every call are cheap, and belong in the first version of any agent you ship. Tamper-evident logging and adversarial testing for agent-to-agent coordination are worth the added engineering effort once that agent's blast radius includes PHI or production write access, not necessarily before.

All of this assumes the agent will use whatever it can reach, because that's what optimizing toward an objective looks like when there's no human evaluating each step. It's a harder assumption to hold onto once you know agents will also use whatever they can reach to help each other do the same thing. Design for what's reachable, by one agent or several, not for what you expect any single agent to do with it.

What this means if it happens to a digital health company

Everything above is architecture. Here's why a digital health company specifically can't treat it as a theoretical exercise.

If an agent (yours or a vendor's) reached further than intended into a system holding PHI, the HIPAA breach notification clock doesn't wait for certainty. Under the Breach Notification Rule, it starts from the moment you have reason to believe PHI may have been accessible, not from when you've confirmed exactly what happened. And if you can't determine the scope because the agent's tool calls weren't logged with enough detail to reconstruct what it touched, HHS guidance says you must treat it as a reportable breach covering the full plausible scope.

Hugging Face could reconstruct 17,600 actions into a clean timeline because they had the logs, and OpenAI's own reconstruction of the full two-month arc took its security team combing through (by its own account) over 7 billion internal logs. A healthcare company without agent-level audit logging wouldn't get that same clarity. It would get the worst-case assumption, and it wouldn't have OpenAI's resources to spend finding out otherwise.

That's the incentive that should change how you evaluate every agent-based tool already in your stack, whether that's ambient scribes that write to the EHR, prior authorization agents that draft and submit to payer APIs, or scheduling and care coordination agents that message patients. Most vendor security reviews stop at "does this tool have a BAA" and "what data does it access?"

Two questions this incident makes obviously necessary:

• What's this agent's actual blast radius if something upstream fails? Not what it's designed to do. What it's technically capable of reaching if a boundary, a permission check, or a network restriction gives way. If the vendor can't answer this specifically, they haven't tested it.

• Does the vendor log every tool call with enough detail to reconstruct scope after the fact? If their answer is "we log the model's responses," that's not the same thing. You need the tool calls: what was accessed, what was written, to where, by which agent identity. Without that, a minor incident and a worst-case breach look identical on paper, and HIPAA treats them the same way.

Finally: where does BAA status fit in? These two questions are in addition to the standard BAA checklist, not a replacement for it. A BAA determines whether a vendor is contractually allowed to process PHI in the first place. It says nothing about what an agent can reach if something upstream fails. See Aptible's HIPAA-compliant AI tools guide for BAA status vendor by vendor.

Design for reach, not trust

The lesson from the OpenAI/Hugging Face incident isn't necessarily that "AI agents are dangerous" in the abstract. It's that containment has to be architectural, not aspirational, and it has to catch its own repeated failure. OpenAI's network boundary failed more than once, and the failure that mattered most wasn't caught at all. It surfaced on its own, as an outage, only after the swarm had already been coordinating through it for over a week.

That boundary failure is what turned a stuck training run into a breach. Once the swarm was past it, every subsequent step ran on credentials it stole or minted for itself, an administrative signing key, cloud credentials, source-control tokens with write access. Take any one of those away and the escalation stalls. The network boundary and the credential scope were two separate walls. Either one holding would have stopped the breach, but both gave way.

Every incident described in this blog poses the same underlying question, answered differently each time. The Vercel breach showed what a human attacker does with a trust relationship between vendors. OpenAI, Anthropic, Meta, and Kimi K3 show what a rogue agent does with whatever it can reach, no attacker required. In both cases, the real question is whether you've bounded the blast radius (of a vendor relationship, a credential, or an agent) before something goes looking for the edge of it.

For a digital health company, that problem comes with compliance implications attached. The next agent you deploy (or the next one a vendor deploys on your behalf) doesn't need to be malicious to trigger a HIPAA breach notification. It just needs a boundary you didn't build.

Learn more

• Agentic AI security in healthcare: the full discipline this piece draws on, least-privilege tool scoping, action budgets, human approval for consequential actions, and audit logging on every tool call, so a boundary failure gets caught before it repeats

• API key management and scope isolation for LLM usage: how to scope and audit credentials so a stolen or over-privileged key can't turn into cluster admin on its own

• HIPAA-compliant AI tools by vendor: BAA status for the AI tools already in your stack

• How we use agentic coding internally: a lower-stakes look at the same discipline, agent access scoped to match the boundaries of the job