KorvaJoin the waitlist
Tech

Good Research Taste Comes From Being Wrong

AI can memorize the scientific literature, but it has not lived through failed predictions.

AI image
The a16z Show · Can AI Learn to Improve Itself? | Recursive Self-Improvement

The question was what AI still lacks as a researcher, and John Schulman, an AI researcher at Thinking Machines, pointed to something hard to teach: taste.

"Research taste is about knowing which ideas are important in the long run, which ideas really catch on and end up going beyond whatever paper they first appear in, but being general and being composable with other ideas."

Mark Chen, a research leader at OpenAI, drew the contrast. There is routine execution, he said, where the experiment is already planned and just needs to be carried out. Then there is taste, which in his words is "how you come up with the idea in the first place."

By that measure, models remain weak at that kind of judgment. They go deep on tuning, endlessly adjusting the small settings that control training, rather than judging which direction is worth pursuing.

Schulman said that judgment comes largely from personal experience doing research. It also comes from reading widely in a field and absorbing the lessons of earlier work.

AI can do part of that, Schulman said, noting it "can read all the literature and memorize it all." But he added that "memorization can be a blessing and a curse."

Schulman put it this way: "You don't have the experience of being wrong or of making predictions and then seeing how they fared."

Questions people ask

What is hyperparameter optimization?

They are the small settings that shape how an AI model learns, such as how fast it updates or how much data it sees at once. Tuning them can improve performance, but it does not decide whether the research question itself is important.

Could an AI model ever learn research taste?

Schulman suggested one path is training models on feedback from human researchers about which proposed ideas are good. The open question is whether rating ideas can substitute for the experience of betting on an idea, being wrong, and adjusting.

What is recursive self-improvement?

It is the idea that AI could speed up AI research, and eventually improve itself. Current speedups most likely come from human researchers working faster with AI help. Full recursive self-improvement would mean full automation of AI research.

More in Tech