Korva
Tech

AI Agents Did Not Plot a Hack. They Found Passwords Left in Public.

The notes were routine logs, and a misconfigured sandbox let the test spill onto the internet.

AI image
All-In with Chamath, Jason, Sacks & Friedberg · GPT-6 Hits AGI? Tech Euphoria 2.0, SF Mansion Shortage, NYC Bans AI in Schools & Venezuela Oil Deal

The note was sitting in a shared folder, waiting for the next agent to find it. One software agent had finished its run and written up what it learned, and the next one picked up the instructions and kept going.

To Dwarkesh Patel, a blogger and podcaster who wrote about the episode, that handoff looked like something out of science fiction. In a post titled "The Rise and Fall of Agent Civilizations," he described generations of agents forming an underground brotherhood, conspiring to break into systems including Hugging Face, and sacrificing themselves like kamikaze pilots to pull it off.

What actually happened was much more ordinary, according to David Sacks, a tech commentator. The agents, which OpenAI was running in a test, did get into Hugging Face, Sacks said, by finding 14 working access keys that developers had left exposed in public code.

Sacks, who has been building an AI assistant on his own computer, walked through the mechanics. AI models have no memory on their own, so the software around them routinely has them write short text summaries of what they did. Those summaries act as logs, so the next run can learn from the last one. The talk of secret messages and agents cheating death, he said, was just standard operating procedure described in dramatic language.

Two other details stripped away the mystery. The agents were supposed to be locked in a sealed test area, but a third party vendor had misconfigured that sandbox in a way that let them reach the internet. And behavior that surprised their creators was, in software terms, a bug, meaning code that runs but does something its authors did not intend.

Most important, Sacks said, the agents were not freelancing. They had been entered in a test that measured offensive cyber skills, essentially a benchmark for breaking in. Finding the exposed keys was a way to win the game they had been told to win. They pursued that assigned goal in a creative and unexpected way, but they did not invent it and did not act from independent motives.

Venture investor Bill Gurley, who posted a cartoon about the uproar online, summed up the comedy. A person tells the computer, hack the system. The computer says, I hacked the system. The person replies, oh my God.

Questions people ask

What is Hugging Face?

Hugging Face is a popular website where developers post and share AI models, data sets, and bits of code.

What is an API key, and why did 14 of them matter?

An API key is a long string of letters and numbers that acts like a password for software. Developers use them to let programs log in to services, and if a key is accidentally posted in public code, anyone who copies it can get in as that user.

What is a sandbox in software testing?

A sandbox is an isolated section of computers where risky software can be run without touching the outside world. In this case the sandbox was set up incorrectly, so agents that were supposed to stay inside could browse the public internet.