You are more overconfident than you think
A 20-question bracketing game that should yield 19 right answers usually yields about 12, and that gap can make you a better decider.
Peter Attia MD · #60 – Annie Duke: How to improve decision making, learning, & apply ‘backcasting’ for future successPeter Attia still winces about the test. Attia, a physician who once worked in McKinsey's risk practice, took it at a company off-site meant to teach what a 95% confidence interval feels like. He thought he would breeze through. He got 12 out of 20 right.
The exercise, a favorite of forecasting researcher Phil Tetlock, who wrote Superforecasting and Expert Political Judgment, is brutally simple. You get 20 questions with knowable numerical answers. For each one, you write down a range you are 95% sure contains the truth.
For example, if asked for the average height of an American man, you might answer between 4 feet 6 inches and 6 feet 1 inch. After all 20 answers, you should miss only one. That is what 95% means: 19 hits and one miss.
The 95% part matters. If the goal were 100% certainty, you could cheat, as Attia put it, by answering "from 1 inch tall to a million miles tall." At 95%, you are supposed to make a reasonable best guess, wide enough to be careful but narrow enough to mean something. The questions are deliberately hard and outside most people's knowledge, like how many votes Abraham Lincoln received or how far Jupiter sits from its nearest neighbor.
Almost everyone fails the way Attia did. He expected to miss one. He missed eight. His brackets held the truth only about 60% of the time, even though every answer was knowable. That gap is the lesson.
Annie Duke, a former professional poker player and cognitive psychologist, says the point is not to become perfectly probabilistic. The world is probabilistic, so the goal is to shift a little toward probability wherever you start, and to catch your gut certainty faster.
To try it, gather 20 trivia questions with numerical answers and write your 19-out-of-20 ranges before you look anything up.
Questions people ask
Where do I get 20 good questions?
You can make your own from any almanac, quiz book, or trivia site, as long as each question has a single numerical answer you do not already know. Pick a mix of history, science, sports, and geography, so you cannot lean on one specialty. The key is to commit to your ranges before checking.
Should I just make my ranges extra wide to pass?
No, that defeats the purpose. Absurdly wide ranges would get you to 19 out of 20, but they teach you nothing about your real uncertainty. The rule is to give the narrowest range you honestly believe has a 95% chance of containing the truth.
Will playing once fix my overconfidence?
One round will humble you, but it will not rewire you. Calibration researchers find that repeated rounds with prompt scoring help people learn how wide 95% really needs to be. Try it every few months, track your hits, and watch whether you move closer to 19.