Diagnose With Data, Treat With Design
Data tells you where the problem is; only a creative process tells you how to solve it.
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 3
- Confidence
- 95%
Zhuo's heuristic for when to trust data versus intuition: data (broadly defined to include TikToks, tweets, and interviews, not just A/B tests) is a diagnostic that reflects reality and surfaces problems and opportunities — but it never tells you what to build. The solution requires a creative, intuitive design process. This dissolves the false either/or of data-driven vs. instinct-driven decision-making.
Origin
A phrase Julie Zhuo developed with her Sundial co-founder and shares with the companies they work with.
Core principles
- 01Data captures reality; our own stories about ourselves are biased and flattering
- 02Data (qualitative and quantitative alike) shows what is happening and where problems/opportunities are
- 03No formula or science tells you how to make a hit — solving requires creativity
- 04Numbers create false precision: which metrics you look at and how you interpret them are art, not science
- 05You cannot A/B test your way into a great product, but you should not throw data out either
How to run it
- 1
Use data to diagnose reality
Look at what people actually do, like, and engage with — quantitative metrics plus TikToks, tweets, and interviews — to see the true picture rather than your flattering story.
Pro tip Count qualitative signals as data too; AI now helps synthesize them.
Watch out Do not expect data to tell you the solution — 'data is not a tool that's going to tell you what you should build.'
- 2
Locate the problem or opportunity
Let the data point you to where retention leaks, where engagement drops, or where an opportunity sits.
Watch out Interpreting whether a 5% move is good or bad is itself an interpretation, not an objective fact.
- 3
Treat with a creative design process
Go back and invent, dream, and design the actual solution — the part no metric can hand you.
Pro tip Experiments let you try more things and understand short-term effects more rigorously, but tests never tell you the long run.
Watch out Beware false precision: believing 'if we did ABC everything will be great' ignores real ambiguity.
In the wild
Zhuo notes data can tell you that you have a retention problem and where it might be, 'but you still need to go back and undergo a very creative process to figure out what's the best way to solve that.'
→ A clear division of labor: data scopes the problem, design produces the fix.
Both Zhuo and the host observe that every great designer they've met is obsessed with understanding what users really think and do — leaning into data — rather than relying on 'a sense of what's right.'
→ Reframes data as a designer's ally for understanding reality, not a threat to craft.
Common mistakes
Expecting data to prescribe the solution
Teams treat metrics as an oracle for what to build; data only reflects reality and locates problems, so they stall waiting for an answer it cannot give.
Trusting the false precision of numbers
Assuming a rising number is unambiguously good ignores that choosing which numbers to watch and interpreting their movement are art, leading to overconfident decisions.
Is it for you?
Best for
Designers, PMs, and founders deciding how much weight to give data versus intuition
Not ideal for
Teams that need a prescriptive formula and won't invest in a creative solution step
From the transcript
“what you really want is you want to diagnose with data and treat with design”
“data is not a tool that's going to tell you what you should build or what the solution is”
“it can tell you if you have a problem and where that problem or opportunity might be. But you still need to go back and…”
“the fact that you still have to choose which things you look at is an art, not a science”
From the episode
From managing people to managing AI: The leadership skills everyone needs now
Julie Zhuo (Facebook VP, Sundial CEO, The Making of a Manage