Korva is a scroll of the internet’s best ideas.
Value a source by removing it
Removing a source
Run an agent with one piece of content taken out of the corpus and check whether the quality of its outputs declines, and by how much. Then try to claw that quality back with a little more compute or a better model. If a cent of compute gets the end customer the same quality, the missing source was worth close to a cent.
The Shapley value
The formal version of that intuition is the Shapley value, a game theory concept that also sits behind SHAP feature importances in machine learning. It answers how to divide a pie that collaboration made bigger than the sum of its parts, so everyone has a reason to collaborate. Computing it exactly means simulating every world where some subset collaborates and the rest don't, which is rarely possible. In web search it can be done by running the agent without a given URL or domain.
The catch is cost: computing exact Shapley values to pay a content owner one dollar could itself cost several dollars. So the values are estimated instead, with models trained on data from those simulated runs.
Who gets paid
Payment then follows two things at once: how unique the data is and how valuable the work done with it is. A banker doing an expensive job who reads a page pays more for that read than a retiree reading the same page.
Why is the exact Shapley value so costly?
The Shapley value averages a participant's contribution across every possible group that could form around it. The number of groups doubles with each added participant, so 30 participants already produce more than a billion groups. Lloyd Shapley introduced the idea in 1953 in the field of cooperative game theory. Because exact calculation grows this fast, practical systems rely on sampling and approximation.
How did a game theory idea reach machine learning?
In 2017, Scott Lundberg and Su-In Lee published a paper introducing SHAP, which treats each input feature of a model as a player in a game. The prediction is the payout, and the Shapley value says how much each feature contributed to it. This gave data scientists one consistent way to explain why a model produced a given output. The method has since become a standard tool for interpreting models.
Where else does Shapley's formula split payouts?
A classic example is the airport problem, which economists Littlechild and Owen analyzed in 1973. Airlines of different sizes share one runway, and the runway cost is divided by how much each airline needs. The Shapley value gives a split in which airlines needing longer runways pay more. Shapley shared the 2012 Nobel Memorial Prize in Economic Sciences with Alvin Roth.
More from Training Data
