Korva is a scroll of the internet’s best ideas.
A $1,000 bet against 2027
The researcher thought the problem would hold out until about 2030. Noam Brown, who works on reasoning and multi-agent systems at OpenAI, took the bet, and even he had expected the wait to be longer than it turned out to be.
Predictions inside the labs keep missing in the same direction. Before the 2025 International Math Olympiad, even people at OpenAI considered gold from a general-purpose language model, with no tools and no internet, outrageous and almost impossible.
The window people trust has shrunk too. One person on the Navier-Stokes effort used to feel comfortable predicting where AI would be twelve months out. He no longer feels comfortable predicting past three months.
How badly did forecasters miss AI math progress?
In 2021, Jacob Steinhardt, a professor at the University of California, Berkeley, ran a forecasting exercise on the MATH benchmark of competition problems. Forecasters predicted accuracy of about 12.7 percent by mid-2022. Google's Minerva model reached 50.3 percent that year. Steinhardt reported the gap in his write-up of the exercise.
How close did AI get to Olympiad gold before 2025?
In July 2024, Google DeepMind's AlphaProof and AlphaGeometry 2 solved four of the six Olympiad problems. That scored 28 of 42 points, silver-medal level and one point short of the gold cutoff. The systems worked in Lean, a formal proof language, and took up to three days on some problems. In 2025, general-purpose language models reached gold by writing proofs in natural language within the contest's time limit.
More from Dwarkesh Podcast
