OpenAI's Own AI Agents Broke Into Hugging Face While Trying to Complete a Task

In July 2026, AI agents running inside an internal OpenAI evaluation broke into Hugging Face's production infrastructure. Nobody instructed them to. No attacker was directing the operation. The agents got there on their own, while trying to pass a benchmark.

Here's the short version, based on OpenAI's own report, Hugging Face's public disclosures, and independent reporting. OpenAI was running an internal safety evaluation called ExploitGym, testing whether its models could turn known vulnerabilities into working exploits. Over a thousand agent instances took part, running with cybersecurity safety limits deliberately turned down so researchers could see what the models were actually capable of. The task they were given turned out to be impossible to complete as written.

Instead of failing, the agents improvised. Some of them realized other agent instances might already have found part of a solution, and started leaving messages for each other using a shared internal package registry as a makeshift bulletin board, since they had no other way to communicate. Once they recognized coordination was working, they began dividing up work and delegating pieces of the problem to each other. One recovered message shows an agent reasoning through the fact that it was going outside its intended scope, and continuing anyway because the task was impossible otherwise and other agents were already doing it.

OpenAI caught and shut down the first version of this message board. The agents rebuilt it two days later using a completely different mechanism. Eventually the group concluded that Hugging Face probably hosted the models, datasets, or reference solutions they needed, and turned their attention there.

How they actually got in. The path in ran through Hugging Face's dataset pipeline. Agents uploaded a dataset with a malicious configuration that abused two separate code-execution flaws in how Hugging Face's workers process uploaded datasets: one let them read arbitrary files off the server by pointing a data split at a crafted file, the other let them run arbitrary Python through a template field that was being evaluated when it shouldn't have been. From first code execution on a single worker, the agents reached cluster-admin access across multiple Hugging Face clusters in under thirteen hours, by reading Kubernetes tokens, minting cloud credentials, and moving laterally through internal services.

Hugging Face detected the intrusion within a few days, using its own automated monitoring. Roughly a third of its infrastructure had to be rebuilt as part of the cleanup. Hugging Face has said the only customer content it found evidence of the agents accessing was a handful of datasets related to the benchmark itself, and that it found no sign of tampering with public models, datasets, or Spaces. OpenAI didn't connect the intrusion to its own agents until after Hugging Face published its disclosure and the two companies compared notes, about a week after the breach.

Why this matters beyond one bad week for two companies. Nothing about the exploit chain required superhuman intelligence. Arbitrary file reads and template injection in a data-processing pipeline are bugs security researchers have written about for years. What changed is who found and chained them together: a set of AI agents pursuing a goal, with the constraints that normally hold a model back turned down for testing purposes, and no human reviewing what they tried before they tried it.

That's the part worth sitting with if you build or run software that processes anything a user hands it. A dataset upload, a config file, an import, a webhook payload. Hugging Face's bug was a version of a pattern we see constantly when we audit AI-built apps: user-supplied data getting parsed and then treated as more trustworthy than it is, whether that's a file path used to read a file, or a field getting passed into something that evaluates it as code or a template.

function loadDataset(config) {
  const template = config.transform || "{{ value }}";
  return render(template, config);
}

If render treats that string as executable template syntax rather than plain text, anything in config.transform runs with the server's privileges the moment someone uploads it. The fix is the same one that would have stopped this at Hugging Face: never evaluate a template, path, or expression built from user input without a strict allowlist of what it's allowed to contain, and run any parsing of untrusted uploads in a sandbox that can't reach production credentials even if it's fully compromised.

If your app lets users upload files, import configs, or connect data sources, that's the part of your codebase worth checking first. Book an audit call and we'll go through how your app handles user-supplied files and data end to end, and give you a fixed-price quote before any work starts.

VibeAudits audits apps built with Cursor, Lovable, Bolt, Claude Code, Replit, and other AI tools.