KorvaJoin the waitlist
Tech

AI May Replace Smart Packet Switches With Dumber Circuits

Internet switches were built to share bursty traffic, but AI clusters run hot and predictable.

AI image
The a16z Show · What the Internet Boom Teaches Us About the AI Buildout | Mike Volpi

The network of the future for artificial intelligence might look a lot like the telephone network of the past: a set of fixed, predictable connections.

To see why, start with how phones once worked. The old system used circuit switching, meaning it opened one dedicated line from caller A to caller B and held it open for the call.

That was wasteful when traffic became uneven. So the internet moved to packet switching, which chops messages into packets and mixes them together on shared lines.

The trick is called statistical multiplexing, sharing capacity on the bet that not everyone needs it at the same moment. It works because internet use is bursty, with peaks, pauses, and different paths at different times of day.

An AI data center breaks that bet, says Mike Volpi, a former Cisco executive who now invests in AI infrastructure. Graphics processors, the chips that train AI, want to run hot all the time, and their links stay heavily used with steady, repeating flows, especially in training.

Yet today's leading data center switches were built for the old assumption. They have enough bandwidth, Volpi says, but they still do the hard, expensive work of parking packets in memory, looking up addresses, and sorting unexpected arrivals.

If traffic is predictable and links are full, much of that cleverness is unnecessary. A simpler design could send data straight through on a schedule, using timed slots or paths made of light, called optical circuits, with far fewer buffers.

That is why Volpi expects a shift from random, shared packet plumbing to timed or optical circuits. It could be much cheaper, and a classic opening for a newcomer, because established switch makers have little reason to undercut their own complex products.

Questions people ask

What exactly is statistical multiplexing?

Statistical multiplexing means combining many uneven streams on one shared line, overselling capacity on the assumption they will not all peak together. It is why an office with 100 people does not need 100 phone lines, or an internet switch does not need a dedicated path for every possible conversation.

How would a timed circuit be different from a packet switch?

Time-division multiplexing gives each sender a repeating time slot, like assigned turns on a single track. Once the schedule is set, the switch does not need to inspect every packet deeply, it just forwards each slot to the right destination at the right moment.

Why is AI training traffic so predictable?

Training usually repeats the same calculation across many chips in lockstep, passing large chunks of data in a regular pattern. Inference, answering user prompts, is more variable, but grouping many requests together in batches can also make its traffic steadier and more scheduled.

More in Tech