The Two Harvard Dropouts Who raised $800M to take on NVIDIA

Faster per-token decode plus much higher concurrency compresses wall-clock time for long-horizon compute tasks because higher tokens/sec and parallel agents let year-long experiments finish in months or days and enable many more iterations.

Focusing on a narrow set of target models and using a kernel-first approach boosts performance and speed to market because you can build physics-level primitives tailored to those models instead of supporting every possible compiler or graph.

Building racks, software, thermal mock chips and production lines in parallel shortens bring-up because the real chips can be dropped into an already-working system, turning months of integration into weeks.

Replacing generic network stacks with a purpose-built interconnect above Ethernet cuts chip-to-chip hop latency, which makes remote memory practical and multiplies usable cluster bandwidth for large models.

Decode performance is mainly a memory problem because tokens/sec depend on how fast the cluster can load model weights and KV caches, so treating memory across chips as a single fast pool wins more than optimizing per-chip specs.

Running inference at much lower voltages lets you pack more usable flops into the same silicon because power falls roughly with the square of voltage, cutting heat and avoiding thermal throttling.

Adding raw flops often does not raise real performance because more switching increases power and heat, which forces chips to downclock, so peak flop counts matter less than the flops you can sustain under thermal limits.

Splitting inference into a prefill step that populates the model's KV cache and a separate decode step increases throughput because the prefill cluster can optimize memory setup while the decode cluster avoids redoing that work.