Korva is a scroll of the internet’s best ideas.
Labs will favor training over inference
Why training wins
Most people assume most compute will go to serving customers. Picture a lab earning $30 million to $40 million per megawatt with 40% of its compute on inference. At $60 million to $70 million, it could keep that 40% and pay out the profit as dividends and buybacks, or it could build AGI. Executives and boards at Anthropic and OpenAI both land on building AGI, so the training share ratchets up.
Best AI stays inside
The same logic applies to the best products. When OpenAI released ultra-fast mode, it allocated it to internal use as well as external, because the value of the fastest, best AI inside the lab beats what an outside buyer gets from it. Inference compute is as large as it is so the training fleet can grow.
The shift in the numbers
The shift has already started. Anthropic adds more compute each month than the month before, yet its revenue additions climbed and then plateaued, and it is no longer adding $25 billion of annualized revenue every month. A marginal megawatt therefore goes to research and development at a higher rate than to inference, a pattern visible over the last three months.
How does internal use of AI help a lab?
Labs use their own models to write code, run experiments, and analyze results, which speeds up each research cycle. Google said in 2024 that more than a quarter of its new code was generated by AI and then reviewed by engineers. A faster cycle means more experiments per month on the same staff.
How do companies choose between reinvesting and paying out?
A company reinvests when it expects a return above its cost of capital and returns cash to shareholders when it does not. Fast-growing companies typically pay no dividend because every dollar they earn can be put back to work. Dividends and buybacks are the way mature companies hand out profits they cannot use at a good return.
Why does a faster mode cost more to serve?
Speed comes from running smaller batches, so each chip serves fewer people at once. Smaller batches use less of a chip's capacity, which raises the cost of every token. Faster generation therefore ties up more hardware per user.
More from Dwarkesh Podcast
