An AI writing one word at a time plans ahead to rhyme with rabbit
To guess the next word well, the model learned to think many words ahead.
Lenny's Podcast: Product | Career | Growth · Inside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic)When writing rhyming poems, an AI model that writes one word at a time can plan ahead to end a later line with the word rabbit, without saying the plan out loud.
Emmanuel Ameisen, who works on interpretability at Anthropic, said his team found this by looking inside the model at its intermediate results. After a line ends, the model is effectively noting this is an it rhyme and it should rhyme with rabbit. Every later word refers back to that plan.
The rhyming plan helps answer a puzzle Ameisen called one mystery of language models: how something predicting only the next word can write a coherent report or poem. His answer is that training in next word prediction rewards far ahead thinking, because, as Ameisen put it, "the best way to know what happens next is to think about" what happens further down the line.
Questions people ask
How can researchers see a plan the model never states?
They decode the lists of numbers the model produces between words. They pass a bunch of text through the model, record those intermediate numbers, and look for patterns tied to ideas like a rhyme. In the poetry case, they could see the it rhyme and rabbit plan present before the final word appeared.
Does the model always plan ahead like this?
No. The finding is that models sometimes form these hidden plans, especially when the task rewards it, like rhyming. Much routine writing can be handled with shorter range patterns. Planning shows up when getting the next word right depends on where the text is headed.
Does hidden planning mean the AI is conscious?
Not necessarily. In this context, planning means the model stores a representation of a future target and uses it to shape earlier words. That is a mechanical capability found inside the calculations. It is different from the larger question of whether a system has intentions or awareness.