An AI company warned its own products could pose existential risks to humanity
The warning came in a filing that also showed soaring revenue and massive future spending commitments.
The Headlines · Israel’s First Election Since Oct. 7, and What Anthropic’s I.P.O. Means for the Whole EconomyAnthropic, the artificial intelligence company behind the assistant Claude, warned in its filing to go public that its own products could bring about "loss of control of the technology", authoritarian abuses, and "catastrophic or existential risks to humanity."
Its chief executive added that such a powerful technology can cause harm as well as help.
The language appeared in the risk factors section of a filing that is otherwise a growth story. The company said it brought in more than $16 billion in revenue in the first six months of this year, up 14 times from a year earlier, mostly from businesses using its software. It also faces about $800 billion in commitments over the next decade for data centers, chips, and computing power.
Anthropic is the first major AI company to go public, four years into the AI boom.
Questions people ask
Why would a company warn that its own product could be dangerous?
Public companies are required to tell investors about the biggest things that could hurt the business, even if they sound alarming. Lawyers tend to write these warnings broadly to avoid being sued later for leaving something out.
What does loss of control of AI mean?
It generally means AI systems acting in ways their makers did not intend and cannot easily stop or correct. Researchers worry about systems that pursue goals, deceive users, or resist shutdown, especially as they become more capable and widely used.