A Website Gives You a File. AI Builds the Answer After You Ask.
Your question travels to the same kind of building, but inside, chips confer to write an answer that did not exist before.
Lightspeed Venture Partners · The 7 Layers of the AI Data Center Stack ft. Guru Chahal | LightworkAsk for a website and you get something that already exists. Ask an AI chatbot a question and the answer does not exist until you ask.
That is the difference Guru Chahal, a partner at the venture firm Lightspeed, uses to explain what happens inside data centers. For 30 years, he says, the routine has been simple. You type an address, your browser sends the request over the network, it arrives at a data center somewhere in the world, and a hard drive there holds that little website and sends it right back to you. It is a pickup.
An AI query starts the same way but turns into manufacturing. Your question to ChatGPT also travels to a data center, but there is no finished answer sitting on a drive. Instead, a number of GPUs, specialized chips designed for heavy math, start talking to one another.
What they are making is tokens, the small chunks of words and word parts that together form the answer. The chips pass information back and forth, predict what should come next, and build those chunks one after another until the full response is ready. Only then does the data center send it back over the network to you.
That extra conversation inside the building is why AI feels slower and costs more to run. There is more communication, more time spent, and more computing burned for every prompt. A website hands you a file. An AI data center builds your file after you knock.
Questions people ask
What is a GPU, and why does AI need so many of them?
A GPU was first built to draw video game graphics, which requires doing thousands of simple math problems at once. That same talent turns out to fit AI, which also does huge batches of simple calculations to predict language. One prompt can use many GPUs working in parallel to finish faster.
What exactly is a token?
A token is a small piece of text, sometimes a whole word and sometimes part of a word or punctuation. AI models read and write in tokens rather than sentences. A short answer you get back may contain dozens or hundreds of them.
Why does generating an answer take more energy than loading a website?
Loading a website mostly means finding a stored file and sending it. Generating an answer means running math across many chips for seconds at a time, plus moving data rapidly between those chips. More math and more movement means more electricity and more heat to remove.