Kill NPS, Ask CSAT
NPS is the marketing industry marketing itself — a satisfaction question has better data properties.
- Difficulty
- Easy
- Time to result
- ~weeks to results
- Steps
- 4
- Confidence
- 93%
Antin's survey-science case against Net Promoter Score and his replacement. The likelihood-to-recommend question fails on multiple measurement grounds: an 11-point scale (precision degrades past ~5-7 points), typically unlabeled mid-points, below-the-fold options on mobile, and a fundamentally flawed premise (most people do not go around recommending products). Its benchmarking defence also fails because NPS moves idiosyncratically and is asked inconsistently across companies. The replacement is a simple overall-satisfaction question, which has better data properties, more precision, and stronger correlation with business outcomes.
Origin
Judd Antin, based on the consensus in the survey-science community and on replication work he commissioned at Airbnb with Mike Moramarco, who led survey science there. NPS itself originates with Fred Reichheld/Bain (2003).
Core principles
- 01NPS is the best example of the marketing industry marketing itself.
- 02Garbage in, garbage out — a badly-designed item cannot be rescued by a clever score formula.
- 03Precision degrades after about five (maybe seven) scale points; 11 is worse, not richer.
- 04On mobile, options below the fold are options that don't exist.
- 05The cross-company benchmark defence collapses because NPS is idiosyncratic and inconsistently administered.
How to run it
- 1
Audit your NPS instrument against survey-science basics
Check the scale length (0-10 = 11 items), the labelling (usually only the poles are labelled — not the gold standard), and the mobile rendering (what fraction of response options sit below the fold?).
Watch out Every one of these failures compounds; the resulting number is noisy before you even compute promoters minus detractors.
- 2
Interrogate the premise, not just the scale
Ask whether your users are people who recommend products at all. 'How likely are you to recommend Windows 11 to friends and family' is unanswerable for someone who never recommends operating systems. Loyalty is being inferred from a behaviour most respondents never perform.
- 3
Swap in customer satisfaction
Ask 'overall, how satisfied are you with your experience with [product]?' — or a scoped version, e.g. satisfaction with customer service when you had a problem. Better data properties, more precise, more correlated with business outcomes.
Pro tip Scope the CSAT question to the moment you care about rather than the whole brand — it becomes actionable rather than a vanity number.
- 4
Replicate the comparison on your own data before switching
Antin didn't just cite the literature — he had Airbnb's survey science lead redo the analysis on Airbnb data to confirm CSAT outperformed NPS internally. Do the same to survive the political fight.
Pro tip Expect resistance: an entire industry of consultants and software vendors has a livelihood staked on NPS being useful.
Watch out Do not lead with 'NPS is garbage' in the exec room; lead with the correlation-to-business-outcome comparison on your own data.
In the wild
Working with Mike Moramarco, who led survey science at Airbnb, Antin re-ran the survey-science comparison of NPS versus CSAT on Airbnb's own data to test whether the academic consensus held for their product.
→ It held. Conclusion: don't ask NPS, ask customer satisfaction.
The standard objection is that everyone uses NPS, so at least it benchmarks against the industry. Antin's counter: the research shows NPS is idiosyncratic — it moves up and down for reasons nobody understands, and it is asked inconsistently across companies.
→ The comparison isn't apples-to-apples, so the benchmark you're defending doesn't exist.
Common mistakes
Keeping NPS for the benchmark
Inconsistent administration and idiosyncratic movement mean your NPS cannot be meaningfully compared to anyone else's, which removes the only remaining reason to keep it.
Assuming the vendor consensus is a scientific consensus
The consultants and software providers who sell NPS have a commercial interest in your believing it is accurate. The survey-science community's consensus runs the other way.
Is it for you?
Best for
Product, research, and growth leaders who own a customer-metric program and can influence what gets reported to the board
Not ideal for
Organisations where NPS is contractually or board-mandated and cannot be replaced — there, add CSAT alongside rather than fighting the swap
From the transcript
“the consensus in the survey science Community is that NPS makes all the mistakes”
“customer satisfaction a simple seat metric is better it has better data properties it is more precise it is more correlated to business outcomes”
“it's simple don't ask NPS ask customer satisfaction”
“the research shows that NPS is idiosyncratic”
From the episode
The UX research reckoning is here
Judd Antin (Airbnb, Meta)