Korva is a scroll of the internet’s best ideas.
Thinking tokens work like mumbling
What a reasoning model is
A reasoning model is a language model with more specific training, with no extra magic. Writing out a step-by-step description of its reasoning before giving the solution would be a much harder task than guessing the solution, given how these models are trained. Instead, the model does what can be called approximate retrieval across its vast training corpus, somewhere in the middle between pattern matching and reasoning but closer to pattern matching.
How tokens steer it
The thinking tokens load up the model's context window so that it becomes more likely to predict reasoning-shaped strings of text. Everything the model absorbed in training, including countless call-and-response examples of written reasoning, is mulched into numerical embeddings that encode similarities and differences as geometric relationships in a high-dimensional space. Reaching an answer in that space may pass through tokens that map to coherent-looking thoughts in plain English, though it doesn't have to.
What the tokens can be
Some of those tokens could be bits of other languages, and some could be fake exclamations like aha. Under the right conditions, they could just be dots. Whether an embedding corresponds to a single word at all is beside the point.
How do embeddings turn words into geometry?
An embedding is a list of numbers the model learns for each token. Tokens that appear in similar contexts end up with similar lists, so distance and direction between the lists carry meaning. In Google's 2013 word2vec work, the vector for king minus man plus woman landed near the vector for queen. Modern language models use lists that run from hundreds to many thousands of numbers.
Why does context change what comes next?
A model predicts each new token from everything currently in its context window. Text already there shifts the probabilities of what follows, so a prompt that looks like a worked math solution makes math-like continuations more likely. The model's weights stay fixed while this happens, so the whole effect comes from the input. The attention mechanism, introduced in the 2017 Google paper Attention Is All You Need, lets each token weigh every earlier token.
Why can pattern matching solve hard problems?
Training text holds huge numbers of worked problems, so a new question often resembles an old one in structure even when the wording differs. Matching on structure can produce correct answers without any general procedure. Apple researchers reported in a 2024 paper called GSM-Symbolic that model accuracy on grade-school math dropped when only the numbers in a problem were changed. The same paper reported a further drop when an irrelevant sentence was added.
More from John Pavlus
