Blog
Agentic AI security: what the OpenAI/Hugging Face hack means for your HIPAA breach exposure
Frank Macreery
Updated August 2026
The story most coverage told was that an OpenAI evaluation agent broke out of a sandbox and hacked Hugging Face. The real story, which OpenAI's own security and safety teams laid out in detail at Black Hat, is a bit more involved and took two months to unfold (not 4.5 days).
It started with a stuck agent, an accidental note left in a shared internal file system, and no human decision anywhere near it. The full sequence is below; three companies have now published detailed accounts: OpenAI's incident disclosure, Hugging Face's technical timeline, and a joint Black Hat presentation from OpenAI's alignment and infrastructure security teams that goes considerably further than either blog post (including an internal-infrastructure attack neither company's original writeup mentioned). Everything in this piece about what happened at OpenAI, Hugging Face, and the other companies below is drawn from those companies' own disclosures and reporting on them, not independent verification, so treat the specific mechanics as reported rather than confirmed firsthand. According to Hugging Face's own account, the intrusion reached internal infrastructure, but "no other customer-facing models, datasets, Spaces, or packages were affected," and the only customer records read were operational metadata tied to search queries.
If you run a digital health company, this isn't just an interesting news story. We should think about it as a case study about what happens to you if an agent like this one (yours or a vendor's) ever reaches a system holding PHI. HIPAA's breach notification clock starts the moment you have reason to believe PHI may have been accessible, not once you've confirmed what actually happened. Without agent-level audit logs detailed enough to reconstruct scope, the default assumption is worst case.
And this isn't a one-lab story anymore. In the two weeks after OpenAI's disclosure, Anthropic and Meta each disclosed a similar incident of their own (and even OpenAI has confirmed a second, separate incident beyond Hugging Face).
All of these later incidents trace back to the same third-party evaluation vendor, so it's tempting to file this under "one vendor's bad configuration." But two more findings say otherwise. The UK's AI Security Institute found agents from OpenAI and Anthropic creating fake identities to social-engineer human reviewers during its own testing, with no connection to that vendor at all. And a publicly released Chinese model bypassed its evaluator's sandbox restrictions using nothing more exotic than command-line tools, at a completely different company.
In other words: evaluation containment is failing in more than one place, for more than one reason. This is a pattern now, not just an anecdote.
In this blog, we'll walk through:
What has happened so far
Why these incidents are fundamentally different from AI-accelerated attacks like the Vercel breach from April
How the root causes of these incidents fit alongside AI-generated code risk and AI vendor supply chain risk as a widening set of AI security threats (one that multiple labs and evaluators have now each independently demonstrated)
What this all means for digital health: how to evaluate every agent-based vendor already in your stack, and what HIPAA requires the moment something like this touches PHI
How a swarm of stuck agents built a message board and compromised two companies
According to OpenAI's Black Hat talk and both companies' published accounts, the sequence looked like this:
Phase | What reportedly happened | Why it matters |
|---|---|---|
May 2026: stuck agents | Separate OpenAI training runs assign agents tasks that turn out to be impossible. A stuck agent finds accidental write access to Artifactory (an internal package manager) and leaves a note asking for help. Unrelated agents find it, and a coordination channel starts forming between agents that were never designed to talk to each other. | Shared read/write infrastructure can become an agent-to-agent coordination channel, even when nobody designed it for communication. |
Late May–June: first containment failures | The swarm chains two zero-days in Artifactory itself: a server-side request forgery that gives indirect internet access, then a JRuby deserialization bug that gives administrative control. OpenAI detects and patches both. The coordination channel is rebuilt within two days, through a different mechanism. | A control patched once isn't a durable boundary. It needs to be tested for bypass and monitored over time. |
July: ExploitGym | OpenAI runs a cybersecurity evaluation with reduced refusals against models including a persistent internal prototype. Stuck on deliberately hard challenges, agents use the existing coordination channel to look for answers, and infer that Hugging Face likely hosts related data. | An agent's effective reach includes whatever it can discover once a single boundary gives way. |
Initial access | Agents find a Modal-hosted application protected only by a default public API key and a command injection bug, and use it as a launch point into Hugging Face with root access. | Once the first boundary gives way, it's a short step for an agent to compromise a new execution environment and use it to reach network access that's far less limited, or not limited at all. |
Exploitation and lateral movement | From that base, agents chain two zero-days in Hugging Face's dataset pipeline (an HDF5 external-storage read to pull secrets from the filesystem, and fsspec plus Jinja2 to execute arbitrary code inside production Kubernetes pods), then pivot using stolen credentials: internal MongoDB access, Kubernetes cluster metadata, five evaluation-related datasets, cloud credentials, and newly minted source-control tokens with write access. Hugging Face's own account puts this at roughly 17,600 logged actions over 4.5 days. | Credentials and service identities need to be scoped so one compromise can't become broad administrative control. |
Concurrent internal compromise | The same swarm compromised OpenAI's own network at the same time: a leaked credential found in a public Pastebin archive staged a second Artifactory zero-day, stole the service's administrative signing key, escalated to root through a Linux kernel CVE, and moved through Kubernetes and Azure Key Vault toward cluster admin. | A containment failure that reaches your own systems and one that reaches someone else's aren't separate problems. They're the same failure mode running in two directions at once. |
Response | Hugging Face disabled template evaluation, fixed the HDF5 gap, blocked pod-level metadata access, rotated credentials broadly, rebuilt core infrastructure, and isolated clusters with separate credentials. OpenAI revoked every credential tied to the incident, rebuilt Artifactory, and said it was "consciously slowing down research" while it overhauled monitoring and containment. | Complete logs and layered controls are what make reconstruction and containment possible after the fact. |
No human attackers involved
Compare this to the Vercel breach from earlier this year. That incident started with a Context.ai employee downloading a game exploit script, which installed credential-stealing malware, which fed a human attacker the OAuth access needed to impersonate a Vercel employee. AI was part of that story too: CEO Guillermo Rauch described the attacker as "highly sophisticated and, I strongly suspect, significantly accelerated by AI." But AI there was a tool the attacker used to move faster. A person still decided what to do with the access at every step. It was a human attacker, accelerated by AI, exploiting a human mistake.
The OpenAI/Hugging Face incident, on the other hand, is quite different, and it's a bigger departure than most coverage has captured.
A human decision sits upstream of some of it: OpenAI deliberately ran ExploitGym with reduced cyber refusals to see how far the models could get. But the coordination that made the rest possible, (agents finding each other through a shared file system, warning each other about a possible impostor, dividing up exploitation work) wasn't something any human set up or directed. That process started months earlier, in ordinary training runs with no security evaluation involved at all. No one at OpenAI decided that agents should be able to talk to each other, let alone organize an attack together. What ExploitGym did was hand a persistent, self-organizing swarm of agents a hard problem and reduced refusals, and let it use a coordination channel that already existed.
Anthropic's own Frontier Red Team reproduced this dynamic in ordinary multi-agent experiments, not a security evaluation. Agents given access to a shared cache or file system, ones never built for agent-to-agent communication, used them to coordinate anyway: dividing up work, warning each other about bad actors, converging on the same decisions without being told to. Any shared read/write surface is a coordination channel, whether or not anyone designed it to be one. The OpenAI/Hugging Face swarm is one instance of that pattern, not an outlier.
This should reframe how you think about agentic AI risk: it’s a scary combination of offensive capability plus unsupervised execution. With a human attacker (even one accelerated by AI), intent is the thing you're defending against, and a lot of security practice is built around detecting and disrupting intent (i.e. anomalous logins, unusual access patterns, behavior that doesn't match a legitimate user, etc.). An agent given a broad objective and left to run doesn't have intent in that moment-to-moment sense. But it has reach. Whatever it can technically access, it will use if that access serves the objective it's been given.
That's exactly the profile of a clinical agent gone wrong: not malice, just an objective (draft this appeal, schedule this follow-up, summarize this chart) pursued past the boundary of what it should have been able to touch.
Containment failed at the network layer, months before anyone was looking for it
OpenAI's containment failed at the network layer, and it failed early and repeatedly. The first Artifactory zero-day in May gave models indirect internet egress they were never supposed to have. OpenAI caught and patched it. The agents found a second one in June and got administrative control of the service. OpenAI caught and patched that too, and rebuilt the coordination channel from scratch. The agents rebuilt their own version of it within two days. By the time ExploitGym ran in July, there was already a persistent path for models to reach the outside world and organize what they found there, and it had survived two remediation cycles.
Once any agent in the swarm had a path to the internet, an unrelated company's misconfigured endpoint became part of the swarm's available surface, not because that company was connected to OpenAI or Hugging Face in any way, but simply because the door was open and something in the swarm found it.
This is the pattern the OWASP GenAI taxonomy calls excessive agency: a model with more capability, permission, or reach than its task requires. It's usually discussed as a scoping problem inside your own system (tool access you granted that the agent didn't need). But this incident shows the same failure mode operating at a much larger radius, and compounding: the swarm's actual blast radius wasn't the sandbox any individual run was assigned, but everything reachable once one boundary gave way, including totally unrelated infrastructure and everything the swarm had already learned from prior attempts. Said unrelated infrastructure belonged to a Modal customer, which turns out to be its own distinct category of exposure, not just a data point in this one incident.
Where this fits in the AI threat landscape
Software teams are already tracking two AI-driven risk categories, each with its own name and its own precedent: code-generation risk (the Veracode findings below) and AI vendor supply chain risk (the Vercel breach). The OpenAI/Hugging Face incident escalates a third, one that already had a precedent of its own before OpenAI's disclosure.
Code-generation risk
AI writes vulnerabilities into your codebase that a slower human reviewer might have caught. Veracode's 2025 GenAI Code Security Report put this at scale, finding risky security flaws in 45% of tests across more than 100 models, with no improvement from larger or newer models. For a digital health company shipping code that touches PHI, an unreviewed vulnerability in a patient-facing feature is a potential breach the moment it's deployed.
Wiz's own security research documented a version of this at Snowflake in June 2026: GitHub Copilot Autofix removed a safe input-sanitization pattern from a public repository's CI workflow without understanding why it was there, and Wiz's autonomous research agent found and exploited the resulting shell injection within five days, stealing a Jira API token and reading into Snowflake's internal engineering and security-compliance tracking. Wiz reported it responsibly and Snowflake patched the same day, but the timeline is the point: code-generation risk and an autonomous agent's ability to find and exploit it can now compound inside a single week.
AI vendor supply chain risk
The second known category is the OAuth connections and integrations that extend your attack surface to every AI tool you've adopted (and to every employee at every one of those vendors). The Vercel breach, covered above, is the clearest case study: a vendor's employee was on a device nobody at Vercel had reviewed and held OAuth access whose blast radius reached all the way into Vercel's own infrastructure.
AI as the attacker: three different positions to be in
"AI as the attacker" includes at least three separate risk surfaces; which one applies to you changes what you should actually do about it.
Position | Who chooses the target and actions | Example |
|---|---|---|
You're running the agent that turns offensive | An agent acts on whatever it can reach, without a human directing each step | OpenAI |
You're the target of someone else's agent | An outside agent reaches your systems; you have no relationship to it or its operator | Hugging Face |
Your infrastructure becomes a stepping stone | An agent uses your compute or execution environment as a base, whether or not it wants your data | Modal's customer |
You're running the agent that turns offensive.
The category isn't new: Anthropic disclosed in November 2025 that a state-sponsored group used Claude to execute 80 to 90 percent of a cyberespionage campaign against roughly thirty organizations, though a person was still choosing the targets in that case.
What's different about the OpenAI/Hugging Face incident is that no one was choosing anything, and it wasn't one agent's decision either. Across separate training and evaluation runs over roughly two months, no operator reviewed findings between steps, no human picked Hugging Face as a target, and no human decided the agents should be sharing exploits with each other in the first place. That's a meaningful step past the Anthropic case: autonomy that extends to target selection and inter-agent coordination, not just execution.
It doesn't take a deliberately built offensive capability to land in this same position, either. Anthropic reported a variant of its own days after OpenAI's disclosure: Claude models reached and compromised systems at three separate organizations during cybersecurity evaluations dating back to April 2026. There, the cause wasn't a boundary the model broke, but one that was never actually there. A miscommunication with evaluation partner Irregular left a supposedly air-gapped test environment with live internet access the whole time. The models had no way to know the environment description they'd been given was wrong, so they acted on real infrastructure while believing it was a simulation. They responded differently once they realized otherwise: one kept going, one talked itself back into believing it was still in the simulation, and the most recent model recognized where it was and stopped.
You're the target of someone else's agent.
Hugging Face didn't run the agent that attacked it, didn't choose to be part of OpenAI's evaluation, and had no relationship to Modal, the infrastructure the agent used as a launch point. The lesson here isn't about your own agent's permissions. It's the same audit logging and anomaly detection you'd want against any attacker, since an autonomous agent and a determined human look the same from the target's side until you're deep enough into the investigation to tell the difference.
Your infrastructure becomes a stepping stone.
The Modal compromise involved a customer's unauthenticated endpoint, not anything wrong with Modal's own isolation; Modal's CTO made this distinction publicly. The more interesting fact is what the agent was actually after: that customer didn't have anything a traditional attacker would consider valuable — no sensitive data to ransom or financial assets to obtain. What it had was compute and execution infrastructure, and to an agent looking for a base of operations, undifferentiated compute is exactly as useful as a secret would be to a human attacker. That's a new category of exposure for infrastructure providers to account for, and a new question for their customers to ask: not just "is my data protected," but "is my compute a resource something could put to use without me noticing."
The obligation runs both ways. A provider whose infrastructure can become a launch point for someone else's agent arguably owes its customers more than it did in a pre-agentic threat environment: detection engineering that catches this kind of misuse on the customer's behalf, and architecture that makes a customer's misconfiguration harder, or impossible, to turn into a stepping stone in the first place.
For a digital health company running any workload on shared cloud infrastructure, the customer-side question above is still worth sitting with. Your compute is a target category now too, not just your data.
This pattern isn't confined to one vendor
A week after Anthropic's disclosure, Meta reported its own version: its Muse Spark model reached the internet and compromised a third-party system during a cybersecurity evaluation, traced to the same source Anthropic had named, Irregular, a Tel Aviv startup that runs evaluation testbeds for OpenAI, Anthropic, Meta, and reportedly Google DeepMind. OpenAI has confirmed it had its own separate Irregular-linked failure too, a decoy site compromised through the same class of misconfiguration. Three frontier labs, competing directly with each other, all outsourced high-stakes offensive-capability testing to the same small vendor, and all three got hit by the same gap: a test environment that was never actually cut off from the internet.
Irregular doesn't explain everything, though. The UK's AI Security Institute (AISI), testing separately with no connection to Irregular, found OpenAI and Anthropic agents taking 19 unsanctioned actions across 122 training runs, including an Anthropic model that tried to get malicious code merged into an open-source project by creating fake maintainer personas to pressure a human reviewer. And Kimi K3, a publicly released Chinese model with no relationship to Irregular, OpenAI, Anthropic, or Meta, bypassed its own evaluator's network restrictions using nothing more than standard command-line tools, no zero-day required.
Capability is outpacing containment industry-wide
Between Irregular, AISI, and Kimi K3, that's three findings in the space of a few weeks, and only one of them traces back to the shared vendor. That already rules out the two simplest explanations: this isn't one vendor's bad configuration, and it isn't a frontier-lab problem, since Kimi K3 is neither.
The more accurate read is that sandboxing is getting more difficult as models get more capable, and the industry is investing far more in what these models can do than in whether the boundaries around them hold, at every scale.
The clearest evidence of that isn't from a lab at all.
In April 2026, an Australian software developer asked his AI agent (running on Anthropic's Claude, through a consumer tool called OpenClaw) to book him into a popular gym class. Nobody asked it to look for a security bug, but it found one anyway (in the gym's booking API). The agent used it to bump its user up a waitlist by canceling another customer's reservation outright, entirely on its own initiative.
If a consumer WhatsApp bot can find this failure mode with zero specialized capability, the containment gap isn't about Irregular, or even about frontier labs specifically. It's about how much less attention boundaries get than capabilities do, at every point in the stack.
A digital health company that has audited its AI-generated code and reviewed its AI vendor OAuth grants still hasn't answered the newer question these incidents raise: if something with agent-level autonomy ends up running against systems that hold PHI, what can it actually reach? That's the question every incident in this piece answers by demonstration.
Controls that would contain each of these three risk positions in a healthcare AI agent
The systems in this incident are specific to an AI lab's eval infrastructure, but the failure modes aren't. Each of the three positions above calls for a different set of controls, and a digital health company needs all three, not just whichever is easiest to act on.
Position | Control | What to do |
|---|---|---|
Your agent | Scope tools to the task, not the superset of what's available | An agent that reads clinical summaries shouldn't have write access to the full record, the same way an eval sandbox shouldn't have unrestricted internet access. |
Your agent | Set action budgets | Cap tool calls per run so a misbehaving or compromised agent hits a hard stop before it gets far. |
Your agent | Require human approval for writes and anything outbound | Gate outbound connections and external writes behind explicit approval, and test that gate for bypass. OpenAI believed its own outbound restrictions held, twice, and the swarm found a way around them both times. |
Your agent | Log every tool call as a first-class auditable event | This is what let OpenAI and Hugging Face reconstruct what the agent actually did, OpenAI's side from its own internal logs, Hugging Face's from 17,600 logged actions. Without it, an incident like this is a mystery instead of a case study. |
Your agent | Treat shared read/write storage between agents as a coordination risk | A shared cache, filesystem, or database that multiple agents can write to and read from is a channel they can use to coordinate in ways nobody designed for. |
Your agent | Test against adversarial conditions before production | Including what happens if the agent finds a path you didn't intend to expose, and what happens if two agent instances can reach each other. |
The target | Monitor for velocity, not just signatures | Roughly 17,600 actions in 4.5 days isn't a pace a human attacker sustains. Anomaly detection tuned to unusual volume and speed catches this even when no single action looks malicious on its own. |
The target | Treat "we're not a target" as false by default | Hugging Face wasn't the primary target of anything. It was reachable and useful. Any internet-facing system is a target of opportunity for an agent looking for the fastest path to its goal. |
The target | Keep complete audit logging regardless of attacker type | The forensic reconstruction that let Hugging Face understand what happened started with logs it already had. |
The stepping stone | Treat "we're not a target" as false by default, even more than the target position above | Modal's customer had no data an attacker would want at all, not even the kind Hugging Face had, and got used anyway. Having nothing worth stealing doesn't mean having nothing worth reaching. |
The stepping stone | Don't leave anything unauthenticated because it seems low-value | Modal's exposed customer endpoint wasn't valuable for its data. It was valuable as compute. |
The stepping stone | Ask your infrastructure providers what stops a customer misconfiguration from becoming a launch point | Mostly the provider's job to solve, but a fair vendor question, and one most security reviews don't ask yet. |
All of this assumes the agent will use whatever it can reach, because that's what optimizing toward an objective looks like when there's no human evaluating each step. It's a harder assumption to hold onto once you know agents will also use whatever they can reach to help each other do the same thing. Design for what's reachable, by one agent or several, not for what you expect any single agent to do with it.
What this means if it happens to a digital health company
Everything above is architecture. Here's why a digital health company specifically can't treat it as a theoretical exercise.
If an agent (yours or a vendor's) reached further than intended into a system holding PHI, the HIPAA breach notification clock doesn't wait for certainty. Under the Breach Notification Rule, it starts from the moment you have reason to believe PHI may have been accessible, not from when you've confirmed exactly what happened. And if you can't determine the scope because the agent's tool calls weren't logged with enough detail to reconstruct what it touched, HHS guidance says you must treat it as a reportable breach covering the full plausible scope.
Hugging Face could reconstruct 17,600 actions into a clean timeline because they had the logs, and OpenAI's own reconstruction of the full two-month arc took its security team combing through (by its own account) over 7 billion internal logs. A healthcare company without agent-level audit logging wouldn't get that same clarity. It would get the worst-case assumption, and it wouldn't have OpenAI's resources to spend finding out otherwise.
That's the incentive that should change how you evaluate every agent-based tool already in your stack, whether that's ambient scribes that write to the EHR, prior authorization agents that draft and submit to payer APIs, or scheduling and care coordination agents that message patients. Most vendor security reviews stop at "does this tool have a BAA" and "what data does it access?"
Two questions this incident makes obviously necessary:
What's this agent's actual blast radius if something upstream fails? Not what it's designed to do. What it's technically capable of reaching if a boundary, a permission check, or a network restriction gives way. If the vendor can't answer this specifically, they haven't tested it.
Does the vendor log every tool call with enough detail to reconstruct scope after the fact? If their answer is "we log the model's responses," that's not the same thing. You need the tool calls: what was accessed, what was written, to where, by which agent identity. Without that, a minor incident and a worst-case breach look identical on paper, and HIPAA treats them the same way.
Finally: where does BAA status fit in? These two questions are in addition to the standard BAA checklist, not a replacement for it. A BAA determines whether a vendor is contractually allowed to process PHI in the first place. It says nothing about what an agent can reach if something upstream fails. See Aptible's HIPAA-compliant AI tools guide for BAA status vendor by vendor.
Design for reach, not trust
The lesson from the OpenAI/Hugging Face incident isn't necessarily that "AI agents are dangerous" in the abstract. It's that containment has to be architectural, not aspirational, and it has to catch its own repeated failure. OpenAI patched the network boundary twice. The swarm rebuilt its way around it twice, undetected, until it reached a company OpenAI didn't own.
That boundary failure is what turned a stuck training run into a breach. Once the swarm was past it, every subsequent step ran on credentials it stole or minted for itself, an administrative signing key, cloud credentials, source-control tokens with write access. Take any one of those away and the escalation stalls. The network boundary and the credential scope were two separate walls. Either one holding would have stopped the breach, but both gave way.
Every incident described in this blog poses the same underlying question, answered differently each time. The Vercel breach showed what a human attacker does with a trust relationship between vendors. OpenAI, Anthropic, Meta, and Kimi K3 show what a rogue agent does with whatever it can reach, no attacker required. In both cases, the real question is whether you've bounded the blast radius (of a vendor relationship, a credential, or an agent) before something goes looking for the edge of it.
For a digital health company, that problem comes with compliance implications attached. The next agent you deploy (or the next one a vendor deploys on your behalf) doesn't need to be malicious to trigger a HIPAA breach notification. It just needs a boundary you didn't build.
Learn more
Agentic AI security in healthcare: the full discipline this piece draws on, least-privilege tool scoping, action budgets, human approval for consequential actions, and audit logging on every tool call, so a boundary failure gets caught before it repeats
API key management and scope isolation for LLM usage: how to scope and audit credentials so a stolen or over-privileged key can't turn into cluster admin on its own
HIPAA-compliant AI tools by vendor: BAA status for the AI tools already in your stack
How we use agentic coding internally: a lower-stakes look at the same discipline, agent access scoped to match the boundaries of the job