An AI researcher quit after six weeks and warned AI could kill everyone
Anthropic's alignment science lead put the odds above 10% within a decade, drawing 200 million views and a Senate proposal to pause AI work.
All-In with Chamath, Jason, Sacks & Friedberg · AI Kills Everybody or Doomer Psyop? OpenAI's Math Breakthrough, Nike's $200B CollapseJacob Coxon had lasted six weeks at Anthropic when he quit.
Coxon, a researcher who had spent the past three years at OpenAI and then Anthropic, went to X after resigning and wrote, "The people building AI earnestly believe that it could kill us all by the end of the decade." He added, "This is not a marketing stunt," and said the companies were "racing straight to self-improving superintelligence and gambling with our lives." Self-improving superintelligence means AI smart enough to redesign itself and get smarter without human help.
What turned a resignation note into a national story was who agreed with him. Evan Hubinger, who runs Anthropic research on keeping AI behavior safe for people, replied in public. "Jacob is correct here. We really do earnestly believe AI could kill all humans!" Hubinger wrote. "I personally think it is over 10% within the next decade. We do not yet have a plan to solve alignment."
Together the two posts drew a combined 200 million views and moved quickly from X to television news. Podcaster Jason Calacanis compared the total to an Elon Musk post about changing a soft-drink formula, which Calacanis called the number one post of all time, and podcaster Chamath Palihapitiya asked how an account with almost no followers could get more than 100 million views overnight.
Then politicians weighed in. Senator Bernie Sanders pointed to the posts as a reason to propose legislation to ban superintelligent AI and pause development, and Illinois Governor JB Pritzker replied, "It's time to sound the alarm louder on reining in AI."
Former White House AI czar David Sacks pushed back. He said Coxon had offered no new evidence, report, or leaked document, only vibes, and demanded, "Show us the data." Sacks pointed to the blank account with almost no prior followers or posts, and to rapid amplification by organized groups in the first 15 minutes, as signs the episode was orchestrated rather than spontaneous.
Whether the posts went viral on their own or as part of a coordinated campaign remains in dispute.
Questions people ask
What does "alignment" mean in AI?
Alignment means getting an AI system to act in line with what humans want and consider safe, even as it gets smarter and more autonomous. Labs test models, write rules for their behavior, and study how they might deceive, disobey, or pursue the wrong goal. When researchers say alignment is unsolved, they mean they cannot yet guarantee a very capable system will stay under control.
What would it mean to "ban superintelligence"?
No law defines it yet, but proposals generally mean stopping work on AI systems smarter than humans that can improve themselves and do most cognitive work. In practice, that could mean limits on training very large models, on giving them self-improvement abilities, or on deploying them without government approval. Sanders has not passed such a bill, and experts disagree on whether a ban could be enforced.
Have AI researchers made extinction predictions before?
Yes. Surveys of AI researchers have often found small but nonzero chances of extinction-level outcomes, with median estimates usually in the low single digits over many decades. Some lab leaders and prominent researchers have signed public statements saying reducing AI extinction risk should be a global priority. The unusual part here was a sitting safety researcher publicly putting the figure above 10% within ten years.