Korva is a scroll of the internet’s best ideas.
Uber blew its yearly AI budget
Engineers get faster
Engineers working with AI are getting much more efficient, and their throughput keeps rising. That carries a significant cost, so the company is slowing the growth of headcount at the same time. The stance for now is to drive adoption and encourage usage across the company.
Thin margins
Efficiency matters because Uber is not a high-margin business. It has over $10 billion in free cash flow, but that comes from well over 10 billion trips a year. Lower costs mean lower prices for riders and more earnings for the people who drive and deliver.
Matching models to stages
The way through is to use different models at different stages. The expensive frontier models are terrific for exploring a new interaction. Once an experience scales, the company looks to bring in models that are more efficient per token, or open source, to bring the cost down.
Why are smaller AI models so much cheaper?
Running a model means passing every token through its parameters, so a model with fewer parameters needs less memory, fewer chips, and less electricity per answer. Developers can also train a small model to imitate a larger one, a technique called distillation that Geoffrey Hinton and colleagues described in a 2015 paper. Some large models use a mixture-of-experts design, where only a fraction of the parameters work on each token. Open-weight models can be run by any company on its own hardware.
How is AI usage billed?
Most AI providers charge by the token, a chunk of text that averages roughly three-quarters of an English word. Input and output tokens are priced separately, and output usually costs more. Because the bill grows with every request rather than with the number of users, spending rises as a tool gets used more. Tokens also set how much text a model can read at once, a limit called the context window.
More from Invest Like the Best with Patrick O'Shaughnessy
