Korva is a scroll of the internet’s best ideas.
A user asks a question and gets answers from two randomized models. They can vote for the first, the second, or call it a tie. Voting is optional, and many people vote anyway to contribute.
Each vote works like a game result in Elo rating, the system used in chess, with a similar rating in tennis. A win against a strong opponent lifts a rating more than a win against a weaker one, and a loss works the same way in reverse.
Arena aggregates all of those results into a leaderboard with a number rating for each model.
Because the leaderboard depends on trust, a team focused on data quality and integrity works to filter low-quality votes and bots, using many techniques.
How does Elo decide how many points move?
Elo compares the two ratings to compute each side's expected chance of winning. After the game, a player's rating moves in proportion to the gap between the actual result and that expectation. A heavy favorite who wins gains very little, while an underdog who wins gains a lot. The points one player gains are the points the other loses, so the total stays constant.
Who invented the Elo system?
Arpad Elo, a Hungarian-born American physics professor and chess master, designed it in the 1950s. The United States Chess Federation adopted it in 1960, and the international chess federation FIDE followed in 1970. Elo built it to replace older rating methods that were less consistent. The same math now rates players in many other games and sports.
Why do ties still count in a rating system?
A tie tells the system that two models are close in quality for that question, which is useful information. In Elo, a draw counts as half a win for each side. If the models had different ratings, a draw nudges the lower-rated one up and the higher-rated one down. Chess has used draws in its ratings since the system began.
More from Lightspeed Venture Partners
