Korva is a scroll of the internet’s best ideas.
Frontier researchers feel less essential
A Recent Belief
The belief behind it is recent. Within the last twelve months, many researchers have come to think that recursive self-improvement, models that improve the models themselves, puts exponential intelligence a year or two away. Andrej Karpathy describes his own pattern with self-awareness: he thought it was two years away for about ten years, and he thinks it again now.
Shrinking Contribution
At the big labs, headcount and compute are so large that a researcher's contribution feels small. There was a time when moving the needle at OpenAI meant a team of 200 people, and the question of how to get there was really up to every single person. Now the demand is $750 billion of compute spend, with many thousands of people working on the problem, so people feel less ownership of the result.
Two Disempowering Beliefs
That leaves a large contingent holding one of two beliefs: what I do doesn't matter because the model is going to do it, or the only thing that matters is compute scale. Both are somewhat disempowering. Some scientists want to work on the problem whether or not it matters that they are the ones working on it.
How could models improve themselves?
The idea was laid out in 1965 by the mathematician I. J. Good, who described an ultraintelligent machine that could design even better machines. Each improvement would speed the next, producing what he called an intelligence explosion. Modern versions imagine AI systems writing code, running experiments, and designing training methods for their successors. Good was a British codebreaker who worked at Bletchley Park during World War II.
Why did compute scale come to dominate AI?
Computer scientist Richard Sutton argued in his 2019 essay The Bitter Lesson that general methods which scale with computation have beaten hand-built cleverness across 70 years of AI research. In 2020, OpenAI researchers led by Jared Kaplan reported that language model performance improved smoothly as a power law with more compute, data, and parameters. Those results gave labs a predictable reason to spend on larger clusters. Training budgets have since climbed into the hundreds of millions of dollars.
Why does a bigger team dilute ownership?
Psychologists have documented social loafing, the tendency of people to put in less effort in groups. Max Ringelmann, a French agricultural engineer, found in the 1880s that people pulling a rope together pulled less each than when pulling alone. Larger teams also divide work into narrower pieces, so each person sees a smaller part of the result. Both effects make individual contributions harder to see.
More from Invest Like the Best with Patrick O'Shaughnessy
