Test the one question that decides if your product works
OpenAI product lead Tara Seshan says fast, pointed experiments beat careful theory when the market will not sit still.
Lenny's Podcast: Product | Growth | Career · AI’s third era: the rise of persistent AI coworkers | Tara Seshan (OpenAI’s product lead)Tara Seshan, a product lead at OpenAI, learned to work a different way. Where capabilities shift with each research advance, she now tries to get a rough version in front of users quickly, a switch she said felt jarring at first.
Earlier in her career in payments, an established market, she could reason rigorously from first principles about what competitors would do next. In that world, she said, the winners thought more rigorously than everyone else, and skipping that work looked careless.
In AI, she said, that kind of rigorous planning does not fit as well, because the market is emergent, fast changing, and tied to research. Trying lots of real tests with users counts for more than careful reasoning from theory. Instead of writing what she called almost a PhD thesis for the next stretch of time, her job became asking, as Tara Seshan put it, "How do I get to something I can try out and test with users as fast as possible?"
Her method is to get pointed about the core hypothesis, what she calls the eigen question, meaning the single most important thing to test, the one that decides whether the rest of the plan matters. To use it, define, as Tara Seshan put it, the "thing that will determine whether your product works or doesn't work." Pull together what you know about users, the market, and the technology to make that hypothesis as sharp as you can, build the fastest effective test, look at the results, and feed what you learn into refining the hypothesis and running it again. At her company, she said, engineers, engineering managers, data scientists, and designers now all work this way.
Questions people ask
What exactly is an eigen question?
It means the single variable that decides the outcome, the question where the answer tells you whether the rest of the plan even matters. Seshan borrows the math term eigen, which suggests something essential or characteristic, to describe the most important thing to test first.
What counts as a fast test?
It depends on the product, but the test is whether real users can try a working version rather than read your reasoning about it. The point is to learn from behavior and results, then refine the hypothesis and test again, instead of debating a document.