Korva is a scroll of the internet’s best ideas.
Token maxing
People jokingly call it token maxing: consuming an inordinate amount of tokens. That cost is not levered or attached to revenue. The latest and greatest model can be ten to twenty to thirty times more expensive than a generic one, and the extra expense does not arrive as extra income.
The hedge fund squeeze
Picture a hedge fund. It feels it has to use the best model because a competitor might use it and out-compete the fund, which makes the choice a game of chicken. A fund that makes only so many dollars of profit each month can slide to break even or to a loss when the token bill climbs. A company that sells a fixed good at a fixed price faces the same squeeze: unable to pass the cost through, it absorbs it.
Pressure on margins
Lower margins are where this shows up. The companies buying the most expensive tokens will face pressure from their finance teams, their CFOs, and their shareholders to explain why they have not moved to cheaper models as the models cluster. They can step down to cheaper models from the same provider or host an open source model themselves.
Why do newer models cost so much more?
The cost of serving a model rises with its size, since each token requires work across all of the parameters that are active. Reasoning models add to this by generating long internal chains of thought before answering, and providers bill those tokens as output. A single hard question can therefore consume many times the tokens of a simple chat reply. Providers set the price per token to cover this compute along with the cost of training.
Why do output tokens cost more than input?
A model reads the input prompt in parallel, processing many tokens in one pass. It writes output one token at a time, and each new token requires another pass through the model. Generation is limited mainly by how fast the hardware can read the model's weights from memory. Major providers' price lists typically charge several times more for each output token than for each input token.
Why do rivals get stuck in games of chicken?
In the game of chicken, two drivers race toward each other, and each prefers that the other swerve first. If both hold course, the result is the worst for both. Game theorists model the structure as the hawk-dove game, where the best move depends entirely on what the rival does. Philosopher Bertrand Russell compared Cold War nuclear brinkmanship to the game in his 1959 book Common Sense and Nuclear Warfare.
More from All-In with Chamath, Jason, Sacks & Friedberg
