Amazon's Chart Said 60 Seconds, So Jeff Bezos Called to Check
The report looked good, so he checked the phone line himself.
Farnam Street · The Map Is Not the TerritoryWhen Amazon was still a young company, Jeff Bezos sat with his team for the weekly business review, going over documents that summed up the business.
The head of customer service had good news. The wait-time figure to reach help was under 60 seconds, in line with what managers expected.
Bezos was puzzled because he had heard many customers complain about long waits for help. The report said one thing. The complaints said another.
The clash shows why a shortcut number can mislead. One summary figure can hide what callers went through. That figure can hide how frustrated shoppers felt and how hard it was to get through. The figure is the summary. What a caller actually goes through is the real thing.
So Bezos tried the real thing himself. He stopped the meeting, picked up a telephone, and called Amazon's own help line.
He stayed on the line for over ten minutes, more than ten times the figure in the report. As he later put it in an interview, "When the data and the anecdotes disagree, the anecdotes are usually right." His long hold had shown something was wrong with how the company measured waits.
Questions people ask
How could the official number be so wrong?
Usually it means the metric was defined badly, measured in the wrong place, or averaged in a misleading way. Common culprits are starting the clock after an automated menu, dropping abandoned calls from the average, or sampling only good hours. That is why checking the real customer experience can reveal a problem the dashboard hides.
What should a manager do when data and complaints disagree?
Treat numbers as a starting point and talk to customers and frontline workers. Shop your own store, call your own helpline, and read raw complaints before they are summarized. If many anecdotes point one way and the metric points the other, assume the metric needs investigating.