Korva is a scroll of the internet’s best ideas.
Snap's phrase as metrics
At Snap the phrase was the fastest way to share a moment with the people you care about. Fast broke down into time to load, share a moment into opening on the camera and sending quickly, and people you care about into best friends. A roadmap question then becomes a test: does this help Snap be the fastest way to share a moment with best friends?
Discord's version
Discord's version was the best way to talk and hang out with your friends before, during, and after playing games, and it splits the same way into talk, hang out, games, before, during, after. Metric definitions came with each phrase, and they flowed into the growth books and experiments, asking whether a change moved core product value. The metrics had to be causally demonstrated with retention, which is tough.
Onboarding with a phrase
The phrase and the metrics onboard people fast. At Snap, a presentation on core product value went to every new hire for seven years straight, every week and later every other week, and new data scientists got on the program quickly. It was part of the cult: this is why people choose Snapchat every single day in the face of tons of alternatives.
How do teams prove a metric causes retention?
A metric that moves with retention may only be a marker for users who were already committed. Chamath Palihapitiya, who led Facebook's growth team, has described the finding that new users who reached seven friends within ten days tended to stay. Randomized experiments are used to tell a cause from a marker. In them, one group of users is pushed toward the behavior and a matched group is not, and retention is compared.
How do A/B tests check a product change?
An A/B test randomly assigns users to the current version or the changed version and compares an outcome across the two groups. Random assignment makes the groups alike in every other respect. Ron Kohavi, who led experimentation at Microsoft, wrote in the Harvard Business Review in 2017 about a Bing change to how ad headlines were displayed. It raised revenue by about 12 percent, worth more than 100 million dollars a year in the United States.
More from Lenny's Podcast: Product | Growth | Career
