Korva is a scroll of the internet’s best ideas.
AI solves problems, struggles to pose them
Uneven strengths
Models are brilliant on some dimensions and weaker than human mathematicians on others, a jaggedness that makes the claim that AI is replacing mathematicians the wrong takeaway. They struggle to pose new problems and to understand which directions, or whole branches of mathematics, are worth exploring.
Why ML research differs
The same jaggedness could matter a lot for machine learning research. Math raises the question of which new branches are worth exploring. ML research has a very clear answer: certain metrics matter, and making a model do better on them is success. That makes the spikes useful for recursive self-improvement, where AI helps improve AI.
Narrowing weak spots
As models improve, their strengths get stronger and the weak spots narrow, so over time they could end up better across the board. How long that takes depends on how long the long tail of weaknesses is.
Why do AI abilities come out so uneven?
Roboticist Hans Moravec noted in the 1980s that tasks hard for people, such as chess, are easy for computers, while perception and movement that toddlers manage are very hard. AI systems have inherited this uneven profile. The phrase jagged frontier was applied to AI in a 2023 study by researchers at Harvard Business School and Boston Consulting Group, in which consultants used GPT-4 on a set of business tasks.
Why is judging which math matters so difficult?
G. H. Hardy wrote in A Mathematician's Apology in 1940 that number theory had no practical use, and he took pride in that. Number theory later became the foundation of public-key cryptography. Ron Rivest, Adi Shamir, and Leonard Adleman published the RSA system, which rests on the difficulty of factoring large numbers, in 1977.
Why do machine learning metrics make progress easy to judge?
The ImageNet challenge gave researchers a fixed set of labeled images and a single score, top-5 error. In 2012, AlexNet by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton cut that error to 15.3 percent. The runner-up scored 26.2 percent. Because everyone used the same test, the gap was undeniable, and the result set off the modern deep learning boom.