Korva is a scroll of the internet’s best ideas.
Ten dollars in, fifty out
The margin flipped
The base cost of compute runs about $10 million to $15 million per megawatt. When OpenAI served GPT-4 on Nvidia Hopper GPUs, revenue fell short of that, which meant a negative gross margin. Newer models like GPT 5.6 and Anthropic's Opus 5 clear the cost by a wide margin.
Profit funds training
That changes where the money comes from. Ten dollars spent on inference capacity brings back fifty dollars of revenue, and all of that profit can go straight into training. A year ago, and even at the start of this year, the labs ran on venture-funded losses. Anthropic turned a profit in Q2, and OpenAI could follow in Q3 as Codex and 5.6 take off.
Growth stays funded
New capital still flows in to speed up growth, but more of each lab's business is now paid for by its own revenue. Margins have climbed sharply over the last year and a half. Spending that was tens of billions of dollars a year is heading toward hundreds of billions, and signed contracts point to trillions a year by the end of the decade.
Why did serving early big models lose money?
A large model has to keep all of its weights in GPU memory and produce one token at a time, which leaves much of a chip's math capacity idle. Each chip earns more when it handles many users' requests in the same batch. Early deployments had fewer users per chip and less tuned software, so each token cost more to produce. Per-token prices were also set low to attract users.
Why do newer models cost less to serve?
Many recent models use a mixture-of-experts design, which activates only a fraction of the weights for each token. Storing numbers in lower-precision formats cuts the memory each token needs. Newer chips such as Nvidia's Blackwell generation also move data faster than the Hopper generation did. Together these changes lower the cost of each token produced.
Why do data centers count capacity in megawatts?
Chips, cooling, and electrical equipment are all designed around how much power they draw, so power is the limit that sets how much hardware a building can hold. Utilities also sell grid connections in megawatts. A single modern GPU rack can draw on the order of a hundred kilowatts, so a megawatt holds only a handful of them.
More from Dwarkesh Podcast
