Korva is a scroll of the internet’s best ideas.
The horse that read body language
Hans was born in the late 1890s and put on public display. Someone would ask him, "What's 3 + 3?" and start counting aloud: 1, 2, 3, 4, 5. Hans would stomp his foot six times. He handled other kinds of problems too, not just sums.
A German psychologist came to believe Hans was not doing mathematics. He was watching the questioner. As the correct number approached, the person gave off an involuntary signal, and Hans had learned to read it and stop. It worked a little like a Pavlovian response, but with more kinds of triggers.
Whether Hans could do any arithmetic is still disputed, and it will stay that way: he died in the First World War and was probably eaten. The same pattern turns up in AI. A model can produce the right answer by latching onto a cue that happens to go with it, and an AI model might do exactly what Hans did.
How did anyone prove Hans was reading cues?
In 1907 the psychologist Oskar Pfungst ran a series of controlled tests on Hans. When the questioner did not know the answer, or when Hans could not see the questioner, his accuracy dropped sharply. Pfungst also found that Hans watched for tiny movements of the head and body as the right number approached. Pfungst published his findings in 1907, and the case became a standard example in psychology textbooks.
Why do researchers now run double-blind experiments?
The Hans case showed that an experimenter can steer a subject without meaning to. Researchers later named this the observer-expectancy effect. In a double-blind design, neither the person running the test nor the subject knows which condition is which, so there is no unintended signal to pick up. Double-blind trials are now standard in medicine and psychology for this reason.
How can a model use a cue instead of the real thing?
A model trained on examples learns whatever pattern most reliably predicts the label in its training data, whether or not that pattern is the thing humans care about. In a well-known 2016 study by Ribeiro, Singh, and Guestrin, an image classifier appeared to tell huskies from wolves but was partly keying on snow in the background. Researchers call this shortcut learning. It tends to stay hidden until the model meets data where the cue and the answer come apart.
More from Computerphile
