Korva is a scroll of the internet’s best ideas.
AI ended hardware's commodity era
Forty quiet years
Intel x86 became the data center chip in the 1990s and stayed that way through the cloud era. Compute workloads grew 25 to 40 percent a year, and Moore's law improved at about the same rate, so no tremendous innovation was required. Hardware had almost no growth for years, and the printed circuit board, memory, enclosures, and networking all commoditized. Moving from one gig to 10 gig networking took seven years.
AI pushes every limit
AI workloads grow 10x every year, and they push every aspect of the hardware to its physical limits. High bandwidth memory is stacked ten chips high, the input and output are 10x what they were, and it took Samsung years to do it. Memory makers have to work with Nvidia three or four generations in advance. A regular server needs a printed circuit board with 10 layers, and AI servers need 40, which very few suppliers can make.
Decommoditization
The industry calls this the decommoditization of the hardware industry. It brings tremendous unit growth and tremendous innovation together at every level of the server.
Why did Moore's law keep servers commodity?
Gordon Moore observed in 1965 that the number of transistors on a chip was doubling regularly, and in 1975 he settled on a doubling about every two years. That steady gain in chip speed matched the 25 to 40 percent yearly growth in everyday computing demand. Buyers could simply wait for faster processors instead of redesigning the rest of the machine. The competition left server makers differentiating mostly on price.
How does high bandwidth memory stack chips?
Memory dies are stacked vertically and connected by thin vertical channels called through-silicon vias that pass data between layers. The stack sits right beside the processor on a shared silicon base called an interposer, which gives a wide, short path for data. SK hynix produced the first HBM in 2013, and the JEDEC standards body adopted it that year. Heat is trapped in the stack, which makes the tall versions hard to manufacture.
Why do AI boards need so many layers?
A printed circuit board is built from alternating layers of copper traces and insulating laminate. Thousands of high-speed signals and heavy power feeds around an accelerator need separate layers to route without interfering. Each layer must line up with the others within tiny tolerances, and errors compound as the count rises. Standard computer motherboards typically have only around six to twelve layers.
How did x86 take over data centers?
In the 1990s, servers built on Intel's x86 chips began to undercut the proprietary Unix machines from Sun, IBM, and HP. They were cheaper, and software like Windows NT and later Linux ran on them. By the 2000s, the x86 design dominated server shipments. Its dominance meant motherboards, memory, and enclosures could be sourced from many interchangeable suppliers.
More from Invest Like the Best with Patrick O'Shaughnessy
