AI turned the $5,000 throwaway server into a $300,000 machine you cannot afford to break
Every part is being pushed to its physical limit, and forgotten suppliers have become hard to replace.
Invest Like the Best with Patrick O'Shaughnessy · Why the AI Boom Is Just Getting StartedAn old server cost about $5,000. If it broke, you threw it away. An AI server costs $200,000 to $300,000, runs so hot it must be cooled with liquid, and if it breaks the whole system goes down.
Alex, who started the investment firm Whale Rock, says the change ends 40 years of calm. The same basic Intel processor design dominated data centers since the 1990s, and computing work grew about 25 to 40 percent a year, slow enough for steady improvements to absorb it. With little pressure to innovate, he says, every part of the server became a commodity, from memory to circuit boards to networking. Moving from 1 gigabit to 10 gigabit networking took seven years, created one brief sales cycle, and then turned cheap again.
AI broke that cycle because workloads are growing about tenfold every year and pushing every part to its physical limit. Alex calls the result decommoditization, meaning hardware that had become interchangeable is now special again. Memory is stacked ten chips high to feed data fast enough. Data inputs and outputs handle ten times the traffic they used to, forcing suppliers to design three or four product generations ahead with AI chip leader Nvidia.
The shift revived companies left for dead. Celestica, a company that builds electronics designed by other companies, had been in what he calls a disaster industry since 1999, when production moved offshore and margins collapsed. It survived by keeping engineering talent from its roots in IBM supercomputing. About three years ago, he noticed it was the sole supplier of the server for Google's AI chip, and its stock traded at about eight times earnings.
What kept it there was skill that is hard to copy. He says many rivals tried liquid cooling and failed, while Celestica stayed good at it. Alex compares the position to "selling a critical part on a plane" where "you'll never get swapped out." That cooling skill is what keeps buyers from switching.
Questions people ask
Why does an AI server need liquid cooling?
A regular server can be cooled with fans blowing air over relatively low power chips. An AI server packs high end processors that together can draw roughly 10 kilowatts, about the average load of several homes, packed so tightly that air cannot carry the heat away. Liquid cooling circulates fluid directly to the hot components, which removes far more heat but requires pumps, seals, and plumbing that must never leak.
What is high bandwidth memory?
High bandwidth memory is a type of computer memory that stacks chips vertically and connects them with thousands of tiny wires, so data can move much faster over short distances. It sits right next to the AI chip that does the calculations, feeding it the huge volumes of numbers those models need. It is expensive and technically difficult to make, which is why only a few large makers dominate it.
What does a contract manufacturer do?
A contract manufacturer builds products designed by someone else, the way one factory might assemble devices designed by several different brands. The business usually competes on price and scale, so profits are thin and investors treat the stocks as cheap for a reason. In AI hardware, skill in cooling, high speed wiring, and testing matters more, so the best manufacturers can charge more and keep customers longer.