Seek the Counterfactual, Not Confirmation
The competitive edge lives in the data you're trying to prove yourself wrong with, not the data you hoped to see
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 3
- Confidence
- 88%
Clowes distinguishes genuinely rigorous customer research from the common illusion of it. Most people talk to the same familiar sources, ask leading questions, and confirm what they already believe. The edge comes from deliberately hunting for disconfirming evidence — the counterfactual, the competitor signal, and how the product is actually used versus how people say it is used.
Origin
Clowes' synthesis of research discipline, drawing on availability and confirmation bias and the practice of seeking the counterfactual.
Core principles
- 01Avoid availability and confirmation bias — don't just talk to the people you always talk to
- 02Seek the proof that you're wrong, not the proof that you're right
- 03Analyze what competitors do and ask what it must tell you about the market
- 04Compare how the product is actually used versus how people say it is used
- 05The advantage is extracted from what other people don't see
How to run it
- 1
Hear from the places you don't normally hear from
Deliberately go beyond your usual contacts and escalations. Sample sources you would not naturally reach, so the input isn't pre-filtered to what you expect.
Pro tip Treat unfamiliar or uncomfortable sources as the highest-signal ones.
Watch out Familiar sources mostly tell you what you already believe — you 'learn nothing particularly new.'
- 2
Actively seek the counterfactual
For any belief, go looking for the evidence that would disprove it. Ask where a customer probes at the edges of what you're trying to do and where your view is wrong.
Pro tip Frame the exercise as 'try to prove to yourself that you're wrong' — it is the fastest way to find the real edge.
- 3
Synthesize competitor and usage signals into insight
Read competitor behavior for what it implies about the market, and reconcile stated behavior against actual product-usage data. Turn raw inputs into a claim about where a well-placed bet yields outsized returns.
Pro tip The prize is 'figuring out what other people don't see' and where you are wrong.
Watch out All data no analysis is not useful — bringing back random inputs without synthesis gains you almost nothing.
In the wild
Clowes contrasts two PMs: one brings back 'an omnibus edition of random stuff I heard on a Tuesday' with no synthesis; the other extracts the competitive advantage by figuring out what others don't see, where the team is wrong, and where a well-placed bet could have outlandish returns.
→ Only the second PM converts research effort into a real market edge.
Common mistakes
Asking leading questions
Going into conversations 'asking leading questions which really are designed to get the customer to say what they already want to be true' blows up the results before any real learning can happen.
No structured way of doing research
People believe they do a lot of research but have no structured method, so it is really occasional customer calls conveniently bucketed as rigor — yielding little insight.
Is it for you?
Best for
Product managers and researchers forming strategy who need to pressure-test beliefs before betting on them
Not ideal for
Situations demanding fast execution on an already-validated direction where re-litigating settled evidence wastes time
From the transcript
“they don't seek out the counterfactual they don't seek out the proof that they're wrong”
“most of the time people go talk to the people they always talk to and they learn nothing particularly new”
“the competitive Advantage is is extracted out of figuring out what other people don't see”
“provoke the answers you don't want to hear provoke the problems”
From the episode
Why great AI products are all about the data
Shaun Clowes (CPO Confluent, ex-Salesforce, Atlassian)