The Sean Ellis Scale Gate
Never scale a product until a calibrated share of users would be 'very disappointed' without it.
- Difficulty
- Moderate
- Time to result
- ~months to results
- Steps
- 6
- Confidence
- 95%
Nubank treats the Sean Ellis product-market-fit survey not as a vanity metric but as a hard gate on scaling. Nothing gets marketing spend, distribution, or headcount until the 'very disappointed' percentage clears a threshold that is deliberately calibrated for the local culture. Below the threshold, the product stays in the lab and iterates. The logic: scaling a small problem produces a big mess, and a large existing user base can make any bad product look alive for years.
Origin
The underlying survey is Sean Ellis's product-market-fit methodology (the 40% 'very disappointed' benchmark), popularized over a decade ago. Duggal describes Nubank as 'amongst the most fanatical followers of his methodology anywhere in the world' and adds two of their own layers: cultural threshold calibration and bullseye-cohort mining.
Core principles
- 01Product-market fit is measurable, not a vibe — bring science to the judgment call.
- 02Adjust the benchmark for cultural response bias rather than accepting a flattering number.
- 03Do not scale a small problem; you will end up with a big mess.
- 04A big app can keep a dead product looking alive for months or years — 'bouncing a dead cat'.
- 05Good enough isn't good enough; the bar is great enough.
- 06Retention (churn) must be checked alongside the score, or the viral loop leaks.
How to run it
- 1
Launch small and survey the first cohort
Ship an early version to a limited user base (Reid Hoffman style — if you aren't embarrassed by V1 you shipped too late) and ask users: how disappointed would you be if this product went away? Use the three-point scale: not disappointed / somewhat disappointed / very disappointed.
Pro tip Run the survey at every stage, starting with a tiny sample and growing it — do not wait until you have a statistically beautiful n.
- 2
Calibrate the threshold to your market's response bias
Sean Ellis's benchmark is 40% 'very disappointed'. Nubank raised theirs to 50% for Brazil because Brazilians are culturally more polite and optimistic than the global average, which inflates scores. Set your own threshold based on how your population answers surveys, and hold it.
Pro tip The same correction applies to NPS: Nubank compensates for cultural bias before celebrating a 90+ score.
Watch out Adjusting the threshold down to hit it is self-deception. The adjustment must make the bar harder where the population is generous.
- 3
Make the score the standing question in every post-launch review
Do not codify it as a written rule — make it the question everyone knows is coming. In every post-launch product review, the first question is 'what's the Sean Ellis score, and how do you know customers love it, not just like it?' That cultural expectation does the enforcement.
Watch out A hard black-and-white rule invites gaming. A universally expected question invites honest preparation.
- 4
Mine the bullseye cohort when the overall score is mediocre
If the aggregate score is borderline, segment it. Find the cohorts where the score is far above threshold and ask what is structurally different about them. Nubank's bill-payments product scored a borderline 40 overall, but the sliver of customers with 4+ bills registered across at least 2 of the 4 payment rails scored 70.
Pro tip Ask the team two questions: what are the three to five things we must build to get decisively past the threshold, and which segments already clear it?
- 5
Rebuild the product so the bullseye behaviour becomes the default
Redesign so that the condition producing the high score is easy for everyone to reach. Nubank rebuilt bill payments to consolidate across rails and onboard multiple bills in one flow, turning a tiny bullseye cohort into a large share of users, then kept iterating on small quality improvements.
- 6
Only then scale — and check the bucket isn't leaking
Once the score clears the bar, scale. Watch churn in parallel to confirm you are not filling a leaky bucket; word-of-mouth growth only compounds if the users you acquire are retained.
Pro tip If the product is genuinely great enough, a large part of scaling takes care of itself because customers tell their friends.
Watch out Resist the executive pressure to scale early. The whole management chain will be excited and will forget they pushed you when the mess appears.
In the wild
Paying bills in Brazil spans four different payment rails, some unreliable, with a credit-bureau penalty for a missed bill. Nubank's V1 bill assistant scored a borderline 40 on the Sean Ellis survey. Segmenting revealed a small cohort with 4+ commitments registered, and within that an even smaller cohort with commitments on at least two of the four rails — that group scored 70. The team concluded the value was cross-rail consolidation plus easy multi-bill onboarding, and made dozens of iterations to make that the default experience.
→ The product went from struggling in the hundreds of thousands of monthly actives to over 10 million monthly actives in roughly 15 months, with a further year of quality iterations still planned.
Nubank launched its high-income rewards card on 4 July 2021. Many segments tried it, but only one loved it: customers who spent enough each month to earn the fee waiver, for whom the other benefits then landed. Customers paying the monthly fee did not find the benefits compelling. Rather than scaling, the team stayed 'in the lab' for two to two and a half years, iterating on where the fit actually was.
→ Scaling only began aggressively two-plus years post-launch, once the fit was understood — avoiding a large, expensive mess in a credit product.
Common mistakes
Scaling a product that works at small scale 'well enough'
Duggal's rule: 'we are not going to take a small problem and scale it because if we do that we end up with a big mess.' Problems that are cheap to fix at 100k users are ruinous to untangle at 10M.
Trusting adoption numbers from a large existing user base
If you already have a big app, you can make any product look great for months or years by cross-promoting it. That's 'bouncing a dead cat' — motion without life. The survey score, not the usage curve, is the honest signal.
Accepting an inflated score from a polite population
Some markets over-report enthusiasm. Taking 40% at face value in a culture that answers generously means shipping a product nobody actually loves.
Reading only the aggregate score
A mediocre average can hide a cohort scoring 70. Discarding the product on the aggregate throws away the signal that tells you exactly what to build.
Is it for you?
Best for
Product leaders and PMs at companies with an existing distribution channel who must decide whether a newly launched product deserves investment and scale.
Not ideal for
Pre-launch concept validation with no users to survey, or infrastructure/compliance features that are mandatory regardless of how users score them.
From the transcript
“we rarely scale a project a product we've launched until we know the Sha Ellis score and we know that it's hit a threshold that…”
“for us our threshold isn't 40% we've generally moved it up to 50%”
“we are not going to take a small problem and scale it because if we do that we end up with a big mess”
“the key is we need to consolidate across these rails not have them one by one”
“over 10 million monthly actives on that product when we were struggling in the hundreds of thousands about 15 months ago”
“good enough isn't good enough is it great enough”
From the episode
Be fundamentally different, not incrementally better
Jag Duggal (Nubank, Facebook, Google, Quantcast)