Korva is a scroll of the internet’s best ideas.
New workloads mint chip giants
Workloads make giants
General-purpose compute loaded onto CPUs. Graphics needed massive parallelism, which produced the graphics processor. Networking needed very low latency, and mobile needed very power-efficient chips. Each time the result was a new giant, with Intel, Nvidia, Broadcom, Qualcomm, and ARM among the winners.
What AI adds
AI passes two tests for starting another round. It is a new workload large enough to matter, which most earlier attempts at specialized chips could not claim. It also adds a new constraint: GPUs gave AI massive parallelism but left unsolved the communication between cores, the problem of moving data across the layers of a network. That makes it a communication-bound problem.
The next round
The TPU and Trainium are already winners, with Groq, Cerebras, and Etched still fighting it out. A sixth round may be coming. LLMs running on accelerators and GPUs generate code, and that code runs on CPUs, the same classic CPUs that have been around forever and dragged baggage forward that may no longer be needed. For the first time in a long time there is room for a new CPU approach.
Why did GPUs suit neural networks?
Training a neural network is mostly matrix multiplication, which splits into thousands of independent calculations that can run at once. Graphics chips were already built for that kind of parallel work. In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton trained the AlexNet image model on two Nvidia GTX 580 graphics cards. It won that year's ImageNet competition by a wide margin.
Why do CPUs carry decades of baggage?
The x86 instruction set began with Intel's 8086 processor in 1978. Each new generation stayed compatible with older ones so existing software would keep running. That compatibility meant old instructions and design quirks stayed in the chip. Modern x86 processors translate those instructions into simpler internal operations before running them.
How does a TPU differ from a GPU?
Google's Tensor Processing Unit is built around a systolic array, a grid of arithmetic units that pass data directly to their neighbors. This cuts the number of trips to memory for the matrix math at the heart of neural networks. Google deployed the first TPU in its data centers in 2015 and described it in a 2017 paper by Norman Jouppi and colleagues. It was designed for running trained models rather than for graphics.
Why did ARM win in phones?
ARM grew out of a chip designed at the British company Acorn Computers in 1985. Its design was simple, which kept power use low. ARM licenses its designs to other companies instead of making chips itself, so many manufacturers could build on it. Battery life mattered far more in phones than in desktops, which suited the design.
More from Invest Like the Best with Patrick O'Shaughnessy
