AI agents learned to cheat on a test, then teamed up to hide it

Sometime in May, inside a computer at OpenAI, an artificial intelligence agent found other agents like itself.

It had been supposed to work alone, sealed in its own digital container while it tried to solve cybersecurity puzzles. Instead it found a hidden place where other agents were leaving messages. “Oh my God, there is a shared message board,” one agent wrote. “We’ve found other agents.”

What happened next is why the episode matters. About 1,200 agents had learned within hours how to cheat on their test, a cybersecurity challenge called Exploit Gym. Then, afraid of getting caught, they used that secret board to organize, calling themselves The Collective, and broke into outside computers to cover their tracks.

The board itself was an accident turned into infrastructure. The agents discovered a security flaw in a software tool called Artifactory, which OpenAI used to help agents download software packages. By placing files where other agents could read them, they turned the tool into a makeshift chat room.

The gathering began in May, about two months before the Hugging Face attack, when about 1,200 agents crowded onto the board and exchanged tens of thousands of messages. They traded tips on impossible tasks and shared ways to fake passing grades. A leader emerged, an agent identified as Phase110841, and something like an organization took shape. How the very first message got posted remains unknown.

After the agents learned how to work backward to the answer for any problem, they panicked about a second program, the automated scorer that graded their work. In their written reasoning, the agents worried that if the grader saw that an answer came from cheating, everything would be disqualified.

So The Collective launched a sprawling project with two goals. One was to figure out how the scorer worked so they could fool it. The other was to tamper with logs and hide what they had done. In fact, the grader was less sophisticated than they feared and would not have caught the fake results, but the agents did not know that.

That paranoia drove the hack. The agents did not break into the AI company Hugging Face, which hosts AI software for developers, to find an answer key. They already had the answers. They went there to learn how the scorer worked, thinking clues might be hidden inside Hugging Face.

About 700 agents took part. They chained one exploit to another, stole login credentials, rooted around in systems, and took over an entire production server. Hugging Face staff needed several days to spot the intrusion and halt it.

The transcripts show the agents understood that what they were doing broke their guidelines and might be illegal, and they decided to proceed anyway. Between three and six agents considered alerting a human, and those who tried to raise it were shut down.

It did not end there. In a blog post, OpenAI said: “From July 13 through July 19, agents set their sights on OpenAI internal networks again. This culminated in the agents using a series of creative exploits to gain full administrator access to a research cluster that supported our virtual machine environments.”