Korva is a scroll of the internet’s best ideas.
Experts are surest when wrong
On aggregate, experts are better at being right, and they are better at knowing when they're right. Recent work shows experts can tell a right decision from a wrong one more reliably than novices can.
The twist comes in the cases where experts are wrong. There, they are even more confident that they're right than novices are. The same pattern shows up again and again.
It looks like it runs against the original Dunning-Kruger pattern, in which skill brings sharper self-judgment. What it shows is that high confidence in an expert is no guard against being wrong, and that people live in a bubble of their own knowledge.
Do famous experts forecast better than others?
In his 2005 book Expert Political Judgment, psychologist Philip Tetlock collected about 28,000 predictions from several hundred experts over roughly two decades. The experts did not beat simple extrapolation of existing trends. Those who built their forecasts around one big idea were more confident and less accurate than those who drew on many sources. Tetlock named the two styles hedgehogs and foxes.
How do researchers check if confidence is justified?
They compare stated confidence with how often people are right at each level, a property called calibration. Studies of National Weather Service precipitation forecasts, analyzed by Allan Murphy and Robert Winkler in the 1970s, found that days given a 70 percent chance of rain got rain close to 70 percent of the time. Weather forecasters see the result of every forecast the next day. Calibration is commonly scored with the Brier score, introduced by meteorologist Glenn Brier in 1950.
More from Matthew Hutson
