Korva is a scroll of the internet’s best ideas.
Five workloads
Multipurpose computing led to the CPU, and massive parallelism led to the graphics processor. Networking needed low-latency chips, and mobile needed power-efficient ones. Intel came out of CPUs, NVIDIA out of graphics, Broadcom out of networking, and Qualcomm and ARM out of mobile.
Two tests for AI
That history frames two questions for AI. Is it a big enough new workload? Plenty of specialized chips for other jobs ended up not mattering, so the workload has to be enormous. And does it bring a new constraint? AI benefited hugely from the parallelism of GPUs, but GPUs didn't solve the communication between cores, the layers of the network, which made AI a communication-bound problem. A giant new workload with a new constraint is what has produced a standalone winner each time.
A sixth generation
A sixth generation may be forming now. Large models running on accelerators and GPUs write code, and that code runs on CPUs of the classic kind. Those CPUs carry decades of baggage they may no longer need, which leaves room for a new approach for the first time in a long while.
Why did graphics chips suit deep learning?
A graphics processor is built to run the same simple math on millions of pixels at once. Training a neural network is mostly matrix multiplication, which splits across thousands of cores the same way. In 2012, Alex Krizhevsky trained the AlexNet image classifier on two NVIDIA GTX 580 graphics cards. It won that year's ImageNet competition by a wide margin.
Why does core-to-core communication limit AI?
Large models are divided across many chips, and every step of training or generation requires those chips to exchange results. Links between chips carry far less data per second than the wiring inside one chip. The time spent waiting on those links can exceed the time spent computing. NVIDIA introduced its NVLink interconnect in 2016 to speed up GPU-to-GPU traffic.
Why do phones need different chips than servers?
A phone runs on a small battery and has no fan, so a chip there is judged on work per watt. ARM designs emphasize that efficiency, and its processor designs power most smartphones. Qualcomm's Snapdragon chips are built on ARM designs. ARM began in 1990 as a joint venture of Acorn Computers, Apple, and VLSI Technology.
Why do CPUs carry decades of baggage?
Software compatibility keeps old instructions alive. Modern x86 chips still run programs written for decades-old processors, and supporting that history takes silicon for decoding legacy instructions and for compatibility modes. A chip free of that history could spend those resources elsewhere. The x86 instruction set began with Intel's 8086 processor in 1978.
More from Invest Like the Best with Patrick O'Shaughnessy
