No One at YouTube Can Explain Its Recommendations
Its recommendation system taught itself through a billion rounds of trial and error, and no human can look inside.
Tom Scott · Why The YouTube Algorithm Will Always Be A MysteryThe most frustrating thing about YouTube recommendations is not that the company will not explain them. It is that no one there can.
For years, the silence was strategy. Google, which owns YouTube, had learned that any hint about how to rank higher brought a flood of bad actors eager to exploit it, so the official guidance stayed simple: create something worthwhile and let Google sort out the rest. Now the silence is structural.
In a paper, YouTube engineers explained that recommendations come from Google's research into machine learning, the same family of programs behind realistic computer voices and behind the program that beat the top human player at the board game Go, which is played with black and white stones.
Machine learning, the way Google uses it, works like a sealed box. A neural network is software organized in layers that adjust themselves with experience. You feed it information about past viewing, and it returns guesses about what to suggest next.
Those guesses are tested, the score goes back into the box, and the cycle repeats a billion times or so. With enough repetitions, the system gets remarkably good.
The result is that even the engineers cannot inspect how the system reached its rules, which is why the workings stay hidden even from the people in charge.
Nobody drafted a list of instructions, because the behavior emerged from countless rounds of testing that left behind settings that work but do not translate into a plain explanation.
That kind of training works well for Go, where winning is clear from the score, but it is much harder when the task is recommending videos, because no one has settled what a good recommendation is.
Questions people ask
What exactly is a neural network?
A neural network is software organized in layers of simple mathematical filters, loosely inspired by how brain cells connect. Each layer transforms the input a little, and the system adjusts the strength of those connections based on whether its last guess was right.
Why does it take a billion repetitions to learn?
Each pass teaches the system only a tiny amount, and small improvements add up slowly. Modern systems also train on enormous amounts of data, so reaching useful accuracy can take hundreds of millions or billions of examples, which computers can run through very quickly.
Could YouTube just go back to human-written rules?
It could, but the old approach required people to guess which signals mattered, like titles, tags, or views, and creators quickly learned to fake those signals. Machine learning was adopted because it finds subtler patterns in actual behavior that are harder to game, even if they are also harder to explain.