❝Story11:00
The Chess.com Positivity Flip That Grew Reviews 25%
Chess.com found that 80% of players review a game after a win, not a loss, which contradicted the feature's original design assumption. So they flipped the post-loss experience to surface brilliant moves and encouraging coach messages instead of blunders. That single change grew game reviews 25%, subscriptions 20%, and lifted retention significantly.
- 80% of people who review their games do so after a win, not a loss
- The team originally assumed players wanted to study their mistakes
- After a loss, they now show best moves and encouraging coach copy
- Result: +25% game reviews, +20% subscriptions, large retention lift
- The insight was then spread to adjacent teams (puzzles, etc.) to compound
“what he observes is that 80% of people that review their games actually do so after a win”
“That change alone was pretty dramatic for us. It grew game reviews by 25%, subscriptions by 20%, user retention by a lot as well.”
#growth#retention#product#chess
❝Story23:00
The Grammarly Move That Nearly Doubled Upgrades
Most free Grammarly users experienced it as a spelling-and-grammar fixer because those were the only free suggestions shown. The team interspersed a limited taste of paid suggestions (tone, clarity, rewrites) directly into free users' writing. Despite fears of giving too much away, upgrade rates nearly doubled because users suddenly saw Grammarly as far more powerful.
- Free users only saw correctness suggestions, so they undervalued the product
- They sampled paid suggestions intermingled into free users' writing in real time
- Concern that giving features away would kill conversion proved false
- Upgrade rates nearly doubled from the change
- Lesson: make your free product a reflection of everything the product can do
“all of a sudden people were seeing Gramly as a much more powerful tool than they were before and our upgrade rates like nearly doubled…”
“It's basically like a reverse free trial but in real time like while you're writing as opposed to a time based one.”
#monetization#freemium#grammarly#conversion
❝Story57:00
The 1,000-Experiments Goal He Openly Made Up
Chess.com went from practically zero experiments before 2023 to ~50, then ~250, with a target of 1,000 next year. Albert cheerfully admits he invented the number. The point isn't hitting it; a stretch goal forces the conversation about what would have to be true to get there, which surfaces the need to enable experimentation across lifecycle marketing, app store assets, and no-code screens.
- Chess.com barely experimented before 2023; ~50 last year, ~250 this year, 1,000 target
- Albert openly admits he made the 1,000 number up
- A goal's value is forcing the 'what would need to be true' conversation
- Hitting the number matters less than the capabilities it unlocks
- Enables experiments in lifecycle marketing, app store assets, and no-code screens
“Did I make it up? Yes, absolutely. I made it up.”
“The whole point of setting a goal is that you can have conversations about what would need to be true to actually hit that goal.”
#experimentation#goals#growth#chess
❝Story1:01:30
The Experiment Tool That Had Retention Backwards
Albert warns that the experimentation system matters as much as any single experiment, and instrumentation is where it breaks. At one company, an in-house tool had user retention configured backwards for three months, so every positive result was actually negative. It's a cautionary tale for why you must instrument your product thoroughly before trusting results.
- The system and instrumentation matter as much as any single experiment
- One in-house tool had retention configured backwards for ~3 months
- All 'positive' results were actually negative during that window
- Bad instrumentation produces wonky, misleading experiment results
“we realized that user retention was actually configured backwards. So all positive results were negative results.”
#experimentation#instrumentation#data-quality#growth
❝Story1:02:30
How Duolingo Found Its Viral Moments With Screenshot Tracking
Rather than forcing virality, Duolingo added temporary screenshot tracking to discover where users were already organically sharing. Hotspots included streak milestones and funny challenges (not, say, advancing in the leaderboard top three). They then staffed those exact moments with illustrators and animators to make them far more delightful, driving 5-10x more sharing.
- Virality is hard to manufacture; find where users already share
- Temporary screenshot tracking revealed organic sharing hotspots
- Streak milestones and funny challenges were highly shared; leaderboard climbs were not
- They staffed those moments with illustrators/animators to amplify them
- Leaning into existing behavior drove roughly 5-10x more sharing
“we invested actually in some time to um essentially add screenshot tracking for like a brief period of time in the app”
“grab the moments where users are already organically screenshotting and make those much much much better”
#virality#duolingo#growth#sharing
❝Story1:13:30
The Chariot Failure: A Solution Searching for a Problem
At the commuter-shuttle startup Chariot, Albert's team built 'Chariot Direct' to add Uber/Lyft-style dynamic routes onto fixed shuttle lines. It failed, teaching three lessons: it was a solution searching for a problem rather than a real user need; in a marketplace they over-focused on the rider app and neglected drivers and operations; and they ran PR before validating demand, which created sunk-cost pressure to see a doomed idea through.
- 'Chariot Direct' added dynamic Uber/Lyft-style routing to fixed shuttle lines and failed
- Lesson 1: don't chase 'wouldn't it be nice' ideas; start from a real user and problem
- Lesson 2: in marketplaces, there's more than one end user; they neglected drivers and ops
- Lesson 3: heavy PR before validation created costly sunk-cost momentum
- The opposite failure mode of validating everything before telling anyone
“this was kind of a solution searching for a problem.”
“doing it before you have validation that customers definitely want the thing is quite risky.”
#failure#product#marketplace#validation