Simplest-V1-Then-Amplify Experimentation
Ship the barest encapsulation of a hypothesis, confirm it has legs, then beat the heck out of it.
- Difficulty
- Moderate
- Time to result
- ~ongoing to results
- Steps
- 4
- Confidence
- 93%
Duolingo's operating engine for running 600+ streak experiments in four years: resist the big V1. Strip an idea to its core hypothesis, ship the simplest possible version, verify the win is real, then iterate relentlessly on that vein. This avoids winning 'because of all the whistles you added' rather than the core insight, ships faster, and compounds learnings iteration over iteration.
Origin
Codified by Jackson Shuttleworth and the Duolingo retention team as their default testing philosophy; the streak-goal feature is the canonical worked example.
Core principles
- 01Resist the big V1 — strip to the core hypothesis first
- 02A win on a loaded V1 doesn't tell you which lever worked
- 03Simple features are faster to design, approve, ship, and learn from
- 04When a vein wins, go deep and 'beat the heck out of it'
- 05Shut down neutral additions — they add cognitive load and complexity for no gain
How to run it
- 1
Reduce the idea to one testable hypothesis
Strip away everything but the single mechanism you believe will move the metric, so a win is attributable to that mechanism alone.
Pro tip 'I can add enough bells and whistles to make something win' — which is exactly why you must isolate the core lever.
Watch out Loading up a V1 lets you win for the wrong reason and mislearn.
- 2
Ship the simplest encapsulation and check for legs
Launch the minimal version and confirm it produces a real, attributable win before investing further.
Pro tip Duolingo's first streak-goal test was one message: 'you're seven times more likely to finish the course if you have a 30-day streak.'
- 3
Amplify the winning vein with isolated arms
Once a vein wins, run follow-on experiments — often with deliberately many arms — to isolate each additional hypothesis (goal length, opt-out button, pre-selection).
Pro tip Test 'way too many arms' so you can isolate each hypothesis cleanly rather than bundling them.
Watch out Pre-selecting the 'harder/better' option for users lost significantly — the act of choosing was where the engagement came from.
- 4
Kill neutral additions; ship neutral only with conviction
If an added feature is DAU-neutral, shut it down to avoid cognitive load and UI complexity. Ship a neutral result only when it's a deliberate platform for future wins — and if so, build that platform into V1.
Pro tip Shipping a neutral experiment is justified only when you have real conviction it unlocks a roadmap of future positive experiments.
Watch out Every neutral feature you keep is complexity you must design around forever.
In the wild
Started with a single 'you're 7x more likely to finish' message borrowed from the monetization team. It won, so they tested different goal lengths (14/50 days), then added an opt-out button as its own isolated arm, then a goal-picker.
→ Each iteration compounded: the opt-out button alone was 'almost just as big a win', and the intentional act of choosing 30 days drove most of the engagement.
After building a goal picker, they tried pre-selecting a harder goal to speed users through, assuming a harder goal meant better retention.
→ 'Lost pretty significantly' — the act of selecting, not the goal itself, was the engagement engine.
Common mistakes
Designing the big complex V1
A feature-rich first version wins for ambiguous reasons and is slower to design, approve, and ship; you lose the ability to attribute the win to a specific lever.
Shipping neutral additions to avoid 'wasting' the work
A DAU-neutral feature adds permanent cognitive load and UI complexity you must build around; shut it down unless it's a conviction-backed platform for future wins.
Is it for you?
Best for
Product teams with enough traffic to A/B test continuously who want to compound learnings and ship fast.
Not ideal for
Low-traffic products that can't reach experiment significance, or one-shot launches with no iteration runway.
From the transcript
“rather than design the big complex feature for V1 just do like the simplest encapsulation of what that feature can be see if it has…”
“we really resist the urge to do the big V1”
“I'm a huge fan of testing like way too many arms for an experiment just to like be able to isolate your hypotheses”
“in general shutting down a neutral experiment so you don't introduce more complexity to the app is is is the way we tend to go”
“it's easy to add things to features that make them win”
From the episode
Behind the product: Duolingo streaks
Jackson Shuttleworth (Group PM, Retention Team)