Stop Averaging Old and New Users When You Measure Retention
Group users by when they joined, track whether they come back, and spend on growth only when they do.

Village Global Podcast · Growth Strategies For Early-Stage Companies with Fareed Mosavat and Jeff ChangJeff Chang, who leads growth engineering at Pinterest and invests as an angel, uses one test before a startup spends more to find customers. The product has to be good enough to deserve it, and he judges that by whether users come back.
He prefers retention over customer surveys like NPS, or asking how disappointed users would be without the product. Retention measures what everyone did, not just the people who answered, so it avoids survey bias and shows whether users actually loved the product enough to reuse it.
A mistake he sees often is mixing old and new users into one retention number. Low-intent users tend to drop out early, leaving a core of high-intent older users, so the blended average mostly reflects the mix of new versus old users rather than real stickiness.
His fix is to measure in cohorts. Take people who signed up in the same period, and who got roughly the same experience when they joined, and track what share returns. Then compare that cohort with the next one, rather than averaging them together.
Fareed Mosavat, an entrepreneur in residence at Reforge and a director of product at Slack, agrees "that retention is the most important metric." Chang says early teams should prize such quality metrics over quantity, since whether they have 50 percent long-term retention or 10 percent matters far more at the start than whether they have 10,000 monthly users or 1,000.
The question is whether people value the product enough to return at its natural frequency, and eventually to pay and to pay more over time. A daily project management tool should be used daily, for example, so its ratio of daily to monthly users should be strong, while a product meant for occasional use should be judged on its own rhythm.
Questions people ask
What exactly is a cohort?
A cohort is a set of users who started around the same time, such as everyone who signed up in the same week. Grouping them this way lets you compare people who saw a similar version of the product.
What is DAU/MAU?
DAU/MAU divides the number of people who use a product on an average day by the number who use it in a month. A high ratio means many monthly users return daily, which matters for products meant for daily use but less for products used occasionally.
What counts as good retention?
There is no single good number, because expectations differ by type of product, frequency of use, and business model. Growth teams usually compare a startup with similar products, and look for cohorts that stay flat over time rather than falling toward zero.


