Korva is a scroll of the internet’s best ideas.
Prompt clutter handicaps smarter models
Less prompt, better results
Anthropic published an article showing that when stuff was removed from the system prompt, fewer contradictions showed up later against user prompts, and the model performed better. Many harnesses are deleting code right now in order to perform better with the latest frontier models.
Why harnesses stay
That doesn't make harnesses a dying idea. A harness is a way for developers who are not model labs to own a user relationship, which is valuable to the economy.
Choosing the model
Users also care which model they are talking to. In the early days of 2023 and 2024, it wasn't even clear which model an app was using, and the assumption was that people only wanted AI. People want to know who they are talking to, much as they want to know which employee they are dealing with, and Notion now lets users choose the model.
Why would extra prompt text hurt a model?
Researchers led by Nelson Liu published a 2023 paper called Lost in the Middle. They found that models use information at the start and end of a long input better than information buried in the middle. Extra text can also introduce instructions that conflict with each other, leaving the model to guess which one to follow.
How does a model decide between conflicting instructions?
OpenAI published a 2024 paper called The Instruction Hierarchy that trained models to give priority to the developer's system prompt over a user's message when the two clash. Without that training, models often follow whichever instruction appeared last or was phrased most forcefully. Researchers treat this ranking of instructions as one defense against prompt injection.
Why do people want to choose the model in an app?
Models differ in speed, price, and strengths, so the same question can get noticeably different answers from each. Users who try several models tend to settle on favorites for particular jobs. A model picker lets an app offer that choice instead of hiding it.
More from 20VC with Harry Stebbings
