The Nielsen Number: Right-Size Your Research
Interview 7-14 people — fewer teaches too little, more teaches nothing new
- Difficulty
- Easy
- Time to result
- ~weeks to results
- Steps
- 3
- Confidence
- 90%
Before applying any LLM or structured reasoning to research, Clowes insists on a rigorous foundation: interviewing the right number of people. The Nielsen number holds that you learn little below seven interviews and stop learning anything new above fourteen. Getting the sample size wrong — plus leading questions — dooms the analysis before it starts.
Origin
Clowes attributes this to 'the Nielsen number' (Jakob Nielsen's usability research on sample sizes), framing 7-14 interviews as the learning band.
Core principles
- 01Between 7 and 14 interviews is the productive learning band
- 02Below 7 you don't have enough data; above ~22 you have too much
- 03Right-sizing effort comes before any tooling or LLM synthesis
- 04Leading questions destroy results regardless of sample size
- 05No amount of LLM assistance rescues badly-collected research
How to run it
- 1
Right-size the interview count
Plan for roughly 7 to 14 interviews for a given question. Two people is too few to trust; twenty-two is more than you need. Match effort to the learning curve.
Pro tip Treat 7 as a floor and ~14 as the point of diminishing returns for a single research question.
Watch out If you interview two people you probably don't have enough data; if you interviewed 22 you probably had too much.
- 2
Ask non-leading questions
Design questions to learn, not to confirm. Avoid phrasing engineered to get the customer to say what you already want to be true.
Pro tip If you can predict the answer you want, rewrite the question.
Watch out Leading questions 'blow up all of the results before they've even heard anything.'
- 3
Only then apply synthesis tooling
With a right-sized, cleanly-collected sample, apply LLMs or structured reasoning to find patterns. Tooling amplifies good research; it cannot fix bad research.
Pro tip Sequence matters: foundation first, tools second.
Watch out If the input is you reading back what you want to hear, the LLM just returns a summarized version of what you want to hear.
In the wild
Clowes warns that if research is not right-sized and set up to learn, 'no amount of applying LLMs or any type of structured reasoning is going to help' — you end up 'reading back what you want to hear or some weird summarized version of what you want to hear.'
→ Wasted tooling effort on top of pre-broken research.
Common mistakes
Not right-sizing efforts at all
Teams interview far too few or far too many people, so they either lack data or drown in redundant input — both undermine the conclusions.
Contaminating with leading questions
Questions designed to elicit the desired answer destroy the data before analysis begins, so even correct sample sizes yield useless results.
Is it for you?
Best for
PMs and researchers scoping qualitative discovery interviews who want a defensible sample size
Not ideal for
Quantitative studies or A/B tests where statistical power, not interview saturation, governs sample size
From the transcript
“once you interview between seven and 14 people you stop learning new things less than seven you don't learn enough more than 14 you stop…”
“if you interviewed 22 you probably had too much”
“they go into these conversations asking leading questions which really are designed to get the customer to say what they already want to be true”
From the episode
Why great AI products are all about the data
Shaun Clowes (CPO Confluent, ex-Salesforce, Atlassian)