Korva is a scroll of the internet’s best ideas.
Checkable answers train reasoning models
Why code and math
AI researchers call coding and math verifiable domains. Code runs, or it doesn't; proofs are either correct or not. Those binary conditions, and the written steps tied to them, create convenient training signals for a reasoning model.
What the model learns
A model trained this way may not need to learn a general reasoning process, or to apply one reliably. It can absorb enough examples of what the steps look like to mimic them on its way to stitching together a plausible result, and the result can then be verified.
The inference horizon
The limit of a reasoning model's training and step-following ability is called the inference horizon. Apple researchers exposed it in 2025 with their paper Illusion of Thinking, which reported complete accuracy collapse under surprisingly simple conditions. Newer models appear to push the horizon further, though unevenly.
Most of the time they probably aren't learning the algorithm behind a reasoning process. They are more likely leveraging an ever-enlarging set of examples and clever reward signals.
Why does a checkable answer help training?
Reinforcement learning needs a reward after every attempt. A test suite that runs generated code, or a checker that compares a final math answer, can supply that automatically and at huge scale, without human graders. The approach is often called reinforcement learning with verifiable rewards. DeepSeek's 2025 R1 paper used rule-based rewards for math and code answers.
How did Apple's puzzles expose the horizon?
In Apple's 2025 Illusion of Thinking paper, researchers used puzzles such as the Tower of Hanoi, where difficulty rises with each added disk. That let them scale complexity one step at a time. They reported that accuracy fell to near zero past a certain complexity, and that the models' written reasoning effort shrank as problems got harder. A Tower of Hanoi with ten disks takes at least 1,023 moves.
How do computers check a math proof?
Proof assistants such as Lean take a proof written in a formal language and check every step against the rules of logic. A single failed step causes the whole proof to be rejected. Leonardo de Moura started Lean at Microsoft Research, with a first release in 2013. Mathematicians used it to formalize the Liquid Tensor Experiment, a project completed in 2022.
More from John Pavlus
