Conviction-Led Product Management
You can't split-test your way to love — build conviction on what matters, then make one big change
- Difficulty
- Advanced
- Time to result
- ~months to results
- Steps
- 4
- Confidence
- 88%
Rather than testing every variable and letting the data pick, Wise builds conviction about the few things that matter (price, speed, ease) through customer research and qualitative insight, then makes large, decisive changes. Experiments are used to validate and to catch breakage, not to discover strategy. The approach saves scarce engineering effort and forces teams to deeply understand the real problem.
Origin
Nilan Peiris at Wise, framed by the internal maxim 'you can't split test your way to love'.
Core principles
- 01Engineering is expensive; scattershot experimentation wastes it
- 02Small incremental tests take forever to reveal what a strong prior already knows
- 03Conviction must be backed by a data 'net' so it doesn't curdle into hubris
- 04Some strategic bets (lowest cost + fast + high quality wins volume) are believed and pursued, not A/B-proven
- 05Split tests are for detecting breakage and material results, not for finding love
How to run it
- 1
Send teams to build a prior, not a test plan
When a PM wants to test the landing page, subject line and program all at once, redirect them: spend the time talking to customers and gathering qualitative insight to form a gut feel about what actually matters.
Pro tip Give a constraint like 'in three weeks pick ONE thing to change' — it forces clarity on the real problem instead of a sprawling test matrix.
- 2
Make one big, conviction-backed change
Launch a substantial change based on the built conviction, then test it to confirm it worked — the experiment validates the bet rather than generating it.
- 3
Keep a data net to avoid hubris
Maintain analysis (pre/post, holdout groups) so conviction stays honest. A holdout to detect negative impact can be smaller than a test powered for significance.
Pro tip If you already know you won't reverse a change, don't slow the release with a full significance test — ship it and reduce the need for validation.
Watch out Not testing everything means you may hurt some metrics without knowing. Use small holdouts/betas to catch breakage even when you skip the full experiment.
- 4
Track the lever, not the per-change ROI
For a strategic pillar like price, set goals as 'cut price by X per quarter' and track the cut itself, because you already believe cheaper drives growth.
In the wild
A new PM on the viral/invite team planned to test landing page, subject line and the whole program before deciding what to build. Peiris instead gave him three weeks to talk to people, build a gut feel, and pick one thing to change and launch.
→ The team learned to build conviction on what matters, enabling much bigger changes than pure experimentation would allow.
Rather than A/B-testing every price drop for revenue impact, Wise held the strategic belief that the lowest-cost, fastest, highest-quality platform would win the world's volume, and simply drove price down step by step.
→ Goals became 'make it this much cheaper each quarter/year' — a conviction-led roadmap instead of an experiment-led one.
Common mistakes
Throwing many ideas at the wall to see what sticks
Experiment-led PM burns expensive engineering and rarely produces step-changes; you can learn what matters to customers through cheaper qualitative means.
Letting conviction become one person's untested opinion
Without a data net, 'I think it's X' becomes hubris where everyone plays Steve Jobs; conviction must be grounded in data plus qualitative insight, not authority.
Is it for you?
Best for
Product leaders at word-of-mouth-driven products where the core levers are few and known, and engineering capacity is the bottleneck
Not ideal for
Optimization-stage products with cheap experiments, ambiguous levers, or a culture prone to unaccountable opinion-driven decisions — there, indexing toward experiments is safer
From the transcript
“you can't split test your way to love”
“you're gonna go talk to people and get quantitative insights and build your own gut feel around what matters”
“you need a data net to make sure it doesn't become a hubris”
“if you know you're not going to run it back then okay you should just run it out and try to reduce the need for…”
From the episode
How to drive word of mouth
Nilan Peiris (CPO of Wise)