Korva is a scroll of the internet’s best ideas.
Inference providers are short on GPUs almost constantly, and the market is likely to stay supply constrained for a while. That raises the question of why Google, Amazon, or Azure don't run around, buy up all the GPUs, and take every inference provider out of business.
One of Nvidia's top priorities is avoiding customer concentration. It wants lots of customers, each with a separate allocation of GPUs, and it wants competition on the compute layer.
That variety is good for end users as well. It lets inference providers come up with new innovations in how to serve models better.
Why are AI chips in such short supply?
Leading AI chips depend on advanced packaging, with TSMC's CoWoS process a well-known bottleneck, and on high-bandwidth memory from only a few makers: SK hynix, Samsung, and Micron. Fabrication plants and packaging lines take years to build. Supply therefore cannot jump when demand does.
How do Google and Amazon avoid depending on Nvidia?
Google began using its own Tensor Processing Units internally in 2015 and later offered them through its cloud. Amazon built Trainium chips for training and Inferentia chips for inference. Both companies still buy Nvidia GPUs alongside their own chips.
Why would customer concentration worry a chipmaker?
If a few buyers account for most sales, one of them cutting orders can hit revenue hard, and those buyers gain leverage over pricing. Nvidia's annual reports to the SEC list customer concentration among its risk factors. Several of its largest customers are also designing their own chips.
More from 20VC with Harry Stebbings
