Korva is a scroll of the internet’s best ideas.
Free credits would bias the leaderboard
Users get free access to the best models and can compare how they perform on their own tasks. That costs money, since Arena pays for access to all of them. If a provider gave Arena a lot of credits for free, Arena would probably use that provider's models more, because they cost nothing, and a bias would creep in.
The same neutrality covers customers. The labs Arena ranks are also potentially paying customers, and what Arena sells them is trust in its results. If Arena were not trusted, the signals it gives them would lose their value.
A provider submits a model checkpoint, and Arena puts it on the platform anonymously. Users vote, and after enough votes the provider receives its ranking in each category and overall, plus data on why it performs the way it does.
Why does a free resource create bias?
When something costs nothing, people use more of it, and that shifts which items get the most data. A ranking built from uneven samples can lean toward the items sampled most. Economists describe this as an incentive effect, since the funder's choices shape the evidence. Many research journals require authors to disclose funding sources for this reason.
Why do anonymous matchups matter for trust?
If voters knew which model wrote which answer, brand reputation would color their judgments. Hiding the identity means each answer is judged on its content. Medicine uses the same logic in double-blind trials, where neither patient nor doctor knows who received the real treatment. The practice dates back to controlled experiments in the mid-twentieth century.
Why do labs submit a model checkpoint?
A checkpoint is a saved snapshot of a model's weights at a particular point in training. Labs often train several versions at once and want to know which is strongest before choosing one to release. Testing a checkpoint before launch lets them compare candidates on real user prompts. Frequent checkpoints are saved during training so that a crash does not erase days of work.
More from Lightspeed Venture Partners
