Korva is a scroll of the internet’s best ideas.
An eval is the product spec
Why labs pay
A lab making an eval set is evaluating something its customers want to do, and doing it better would earn more revenue. Buying a training set then means hill-climbing that eval set. As long as the money spent on data is less than the revenue it brings in, labs are happy to crank the lever, and they want to crank it harder, since spend on data translates directly into more revenue. Procurement teams still negotiate for a better deal.
Enterprise models
The same logic reaches enterprises. If every enterprise needs specialized proprietary models, then even if capabilities start to saturate, the eval serves as the product requirements document for exactly what you want and as the optimization objective for better performance. As long as better models are valuable to the economy, there will be demand for eval sets and training sets.
Each specialized model would need enterprise-specific eval and training data to show it how to perform in its setting. The cases where the ROI justifies that are expected to increase over time.
Why can optimizing an eval backfire?
Economist Charles Goodhart noted in 1975 that a statistical measure tends to stop being reliable once it is used as a target, and anthropologist Marilyn Strathern later gave the idea its popular phrasing. Models can learn quirks of a test set, such as phrasing patterns, that raise the score without improving real ability. Researchers guard against this with held-out test sets that the model never trains on.
Why do specialized models need their own data?
A model trained on general text sees little of a field's jargon, formats, and unwritten conventions. Fine-tuning on domain examples shifts its behavior toward that field. Bloomberg trained BloombergGPT in 2023 on a large collection of financial documents alongside general text.
How do tests act as specifications in software?
In test-driven development, popularized by Kent Beck in his 2002 book Test-Driven Development: By Example, a programmer writes a test describing the desired behavior before writing any code. The test then defines what done means, and the code is revised until it passes. An eval plays the same role for a model, with the score standing in for a passing test.
More from 20VC with Harry Stebbings
