KorvaJoin the waitlist
Startups

Test Your Gut When You Trust It Most

Harvard professor Stefan Thomke calls high confidence plus pressure to act the danger zone, when testing assumptions matters most.

AI image
productmasterynow.com · 617: Test your gut, don’t trust it –the scientific mindset for product teams – with Stefan Thomke, PhD – Product Mastery Now

Harvard Business School professor Stefan Thomke came to experimentation as an engineer, learning to design tests to fix a manufacturing issue at a chip factory. After more than three decades studying innovation, he points product teams to a familiar moment, when certainty and urgency meet.

Thomke calls that moment the danger zone. “The greatest danger zone is when you are most confident in your assumptions and under pressure to act.”

That combination can feel like a reason to skip testing, but for Thomke it is a reason to insist on it.

That caution fits a large review of nearly 22,000 online trials, most comparing two versions of a page, across over 1,300 firms, where only around a fifth of tweaks helped in a measurable way, while half hurt performance.

In Thomke's account, the problem starts with how people think. The brain is a belief engine that spots connections between unrelated things and mistakes correlation for cause. As physicist Richard Feynman put it: “The first principle is that you must not fool yourself, and you’re the easiest person to fool.”

In his view, experience and data help, but they can also reinforce an untested story about why something works. Experiments earn their keep because they can prove a team wrong.

His prescription is tests that are reversible and inexpensive before they lead to larger commitments, with rigor matched to the uncertainty and stakes.

For Thomke, surprising information carries special weight. “Effective leaders don't trust their gut. They test their gut.”

Questions people ask

Why is surprising information more valuable than confirming what you already think?

A surprise means your mental model was wrong, so correcting it changes many future choices, not just one. Confirming what you already believed mostly leaves decisions unchanged. In business, avoiding one confident but costly mistake can pay for many small tests.

How can a busy product team test an assumption without slowing everything down?

Start with the riskiest assumption, phrase it so it can be measured and proven wrong, and watch what customers do in a small, reversible test. A subset of stores, a limited web page version, or a small pilot can be enough. Match the rigor to the stakes, with heavier methods saved for large, hard to reverse commitments.

Does this mean experience and data do not matter?

Thomke says experience and data are valuable, but limited. Experience comes from past conditions that may not apply, and data can show correlation without proving cause. He argues for using experience and data plus experiments, because only a test can show that a favored explanation is wrong.

More in Startups