Korva is a scroll of the internet’s best ideas.
Sole builder of Google's TPU servers
A fading industry
Nearly all of that industry moved offshore to China and turned into commodity work. Celestica hung on and kept the talent from its heritage in IBM supercomputing. Three years ago, it was the sole supplier of the Google TPU server, and its stock was trading at eight times earnings.
Why AI servers differ
An AI server is liquid cooled, runs much hotter, and is a machine worth $200,000 to $300,000, where an old server cost about $5,000 and got thrown away when it broke. When an AI server breaks, the whole thing goes down. That makes its builder critical infrastructure, like the supplier of a critical part on a plane, and a supplier like that never gets swapped out. Plenty of other companies tried liquid cooling and failed, and Celestica turned out to be quite good at it.
The switch business
Celestica also sold white-box Ethernet switches into the clouds, which is a code word for commodity switches. The upgrade cycle for those used to be seven years, from 100 gig to 400 to 800. Now the switches upgrade every year, which is really hard to do. Its engineers also helped write the open-source SONiC software layer and work closely with Broadcom, and the company holds roughly 50 to 60 percent of the cloud Ethernet switch market.
Why does liquid cooling suit AI chips?
Water carries far more heat per volume than air, so a modest flow of coolant can remove what would take a gale of fans. Cold plates sit directly on the processors, and the liquid circulates through them to heat exchangers at the edge of the rack. AI accelerators draw hundreds of watts each, and racks full of them reach tens of kilowatts or more. Many older air-cooled data center racks were designed for roughly 5 to 10 kilowatts.
How does a TPU differ from a GPU?
A TPU is a chip Google designed specifically for the matrix math inside neural networks. Its core is a systolic array, a grid of multiply units that pass data straight to their neighbors instead of going back to memory each time. GPUs began as graphics processors and grew into flexible parallel chips, while TPUs give up some flexibility for efficiency on this one job. Google's 2017 paper on its first TPU reported that the chip had been running in its data centers since 2015.
Why do AI clusters need faster switches?
The accelerators in a training cluster exchange data constantly, and a slow link leaves very expensive chips sitting idle. That ties the network's speed to each new generation of accelerators instead of to a long enterprise upgrade cycle. Ethernet port speeds have moved from 100 gigabits to 400 and 800 gigabits. Industry roadmaps already include 1.6 terabit Ethernet.
Where did SONiC come from?
SONiC stands for Software for Open Networking in the Cloud, and Microsoft started it to run the switches in its Azure data centers. It is a Linux-based network operating system that can run on switches built around chips from different vendors. Microsoft released it as open source in 2016. In 2022 it was moved under the Linux Foundation.
More from Invest Like the Best with Patrick O'Shaughnessy
