Korva is a scroll of the internet’s best ideas.
Cheap durable execution needs bare metal
State saved then deleted
Picture five LLM calls in a row: classify, get context, handle what the user typed in. Any step can fail, so the full state from each one is saved, letting the function restart from step three instead of the beginning. Then, when the function finishes, all of that state is deleted.
Cloud costs
That pattern moves a huge amount of data, even if it is encrypted. AWS, GCP, and the other large clouds charge a lot for both compute and bandwidth. A durable execution company on a public cloud either sells at a high price to cover those costs or runs its own servers.
Running its own hardware
Inngest runs its own: multiple data centers, its own racks, and everything from switches and firewalls to the machines. Only the power and the internet connectivity come from outside. The costs run about 20 times lower, and performance is better.
Many neoclouds resold AWS for a long time, which means their costs had to sit above AWS or they would lose money. Railway placed an early bet on bare metal, one of the few that did.
Why do clouds charge so much for bandwidth?
Public clouds typically bill data leaving their networks per gigabyte, while data coming in is usually free. AWS also bills traffic that moves between availability zones inside a single region. A system that writes and deletes state at every step creates a steady stream of such transfers. Some competitors, such as Cloudflare's R2 storage, advertise having no egress fees at all.
Why save state at every step?
Each LLM call costs money and takes time, so repeating steps one and two after a failure at step three wastes both. Saving each result lets the function restart from the failed step and reuse everything before it. Failures are common with model APIs, since rate limits and timeouts happen under load. HTTP status 429 is the standard code that signals a rate limit.
How do companies run their own data centers?
The usual route is colocation: a company rents space, power, cooling, and network connectivity inside a facility run by someone else, and fills its racks with its own machines. This matches a setup where only power and internet come from outside. The hardware is bought up front and used for several years, in contrast with the hourly rental of cloud servers.
More from The Peel with Turner Novak
