Korva is a scroll of the internet’s best ideas.
A separate watchdog checks every action
Checking every action
Language models hallucinate, and an assistant with a credit card and an inbox can't afford that. So before a thinking trace becomes a tool call, a monitor decoupled from the agent's own incentive system validates it. It can approve the step, disapprove it, or intercept it before anything happens.
A made-up name
One example is a proper noun the model generated out of thin air because of a sampling error. The monitor is built to catch exactly that kind of mistake, which is easy to spot in hindsight.
Guarding the door
A second layer sits at the door. Every piece of text and media headed for the agent passes through firewalls that can reject malicious content trying to talk it into doing something. An early version of the product launched without the firewalls or the monitors, and the team answered by building a system to handle that whole class of problems.
Why do language models invent details?
A language model generates text by choosing each next word from a probability distribution over possible words. Sampling means an unlikely word is sometimes picked, and a proper noun chosen that way can read as perfectly fluent. The model has no built-in step that checks a name against reality. Lowering the sampling randomness reduces these errors but does not remove them.
How can text hijack an AI agent?
The attack is called prompt injection, a term the developer Simon Willison popularized in 2022. An agent reads text from emails, web pages, and files, and it can mistake instructions hidden in that text for commands from its owner. A 2023 paper by Kai Greshake and colleagues demonstrated this indirect form of prompt injection against assistants that read web content. Filtering incoming content before the model sees it is one of the defenses researchers have proposed.
Why must a monitor have different incentives?
Separation of duties is an old security principle: the person who requests an action should not be the one who approves it. Banks apply it when a large transfer needs two sign-offs. A monitor that shares the agent's incentives could repeat the agent's mistakes. Jerome Saltzer and Michael Schroeder listed separation of privilege among their design principles for protection mechanisms in a 1975 paper.
More from Invest Like the Best with Patrick O'Shaughnessy
