Korva is a scroll of the internet’s best ideas.
A stalled agent costs a day
Chat versus loop
In a chat, getting back on track is cheap. If one side did not hear the other, they ask and carry on. A loop is more like a pen pal: send the instructions, go watch a movie for three and a half hours, and come back to a pile of productive work. The gains are enormous, and so are the costs when it goes wrong.
Why models stop early
If the model goes off the rails and the problem only shows up a day or two later, that is a day or two of lost time. Models also get lazy. They stop at some point and ask for a response, which is understandable, since they began as conversational chatbots that eventually have to end a monologue and hand the turn back.
Catching a silent stop
Picture a task that takes a day, where the model gives up in hour three and nobody checks until hour 24. There is often no way to know it stopped. The labs are working on getting models to push through those early stopping points over very long trajectories. Temporal, a company working on durable execution, builds technology for running over long time horizons in a reliable way, and anyone can use it while crafting their own loops.
Why do chat models hand the turn back?
Chat models are fine-tuned on dialogue in which a user and an assistant take alternating turns. In OpenAI's 2022 InstructGPT work, the training data included human-written demonstrations and human rankings of replies. Each example was a bounded reply followed by the user's next message. That training method, reinforcement learning from human feedback, became standard across chat assistants.
How does durable execution survive a crash?
A durable execution system writes each completed step to a persistent event history. When a process dies, a fresh worker restarts the program and replays that history, skipping steps that already finished. The code then resumes from the point where it failed. Temporal's workflow engine descends from Cadence, a system built at Uber.
How do engineers catch a silently stalled process?
A common method is a heartbeat, a small signal the process sends at regular intervals. A monitor that misses several heartbeats in a row raises an alert. A clean exit looks the same as a finished job to most systems, so the heartbeat is the only sign that work is still going. Hardware watchdog timers work the same way, resetting a machine that stops sending its periodic signal.
More from Lightspeed Venture Partners
