Korva is a scroll of the internet’s best ideas.
Right answers for wrong reasons
The AlphaFold case
AlphaFold, Google's tool for predicting protein structures, runs on incredibly complex statistical associations that nobody can identify, and they seem to work. It is a black box even without a reasoning trace. If reasoning models can supercharge mathematics research the way AlphaFold did for computational biology, the argument goes, they can be embraced, idiosyncrasies and all, with the results verified afterward.
What builders want
The people building them want models that solve problems that matter and accelerate scientific research, and they find it more productive to discuss what the models can do than why they can only do it.
The case for right reasons
The other side wants the right answer for the right reason, so the answers can be trusted, and not only in domains where results can be verified. A system should apply an algorithm reliably, whether or not anyone calls that reasoning. Treating chains of thought too reverently, even when their results are verifiable, could also keep scientists from finding better ways of steering models toward accurate outputs.
How was AlphaFold checked without explanation?
Its predicted protein structures can be compared against structures measured in laboratories by methods such as X-ray crystallography. At the 2020 CASP14 competition, its predictions matched experimental structures far more closely than any other entry, according to the organizers. DeepMind and the European Bioinformatics Institute released a database in 2021, which by 2022 held predictions for more than 200 million proteins. The 2024 Nobel Prize in Chemistry went in part to Demis Hassabis and John Jumper for AlphaFold.
Why doesn't a verified answer settle trust?
A verified answer shows that one output is right, not that the process behind it will be right the next time. Software engineering has long carried this idea. Edsger Dijkstra wrote in 1970 that testing can show the presence of bugs but never their absence.