Experiment at 30: Trend Over Precision
You can run meaningful experiments with as few as 30 users — more data buys precision, not different trends.
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 3
- Confidence
- 90%
Founders avoid experimenting early because they think they lack users, but the underlying trend a test reveals stays roughly the same from 30 data points to 100 — additional data only sharpens precision. Since ideas are dramatically cheaper to test at small scale, early-stage teams should run experiments now and look for a large fraction of a small group taking the target action.
Origin
Widjaja's applied statistical framing, grounded in sampling theory (precision improves with n, central tendency stabilizes early).
Core principles
- 01Zero data points is the only truly uninformative sample — 30 beats 0 decisively
- 02More data improves the precision of a trend, not the trend itself
- 03Small-scale experiments are far cheaper and faster, so run more of them
- 04Look for a large proportion of the sample doing something, not a huge absolute count
How to run it
- 1
Accept 30 as a workable sample
Stop waiting for statistical scale. Recognize that with ~30 users you can already detect directional trends; the numbers you get back generally won't change in kind at 100, only in confidence.
Pro tip Every idea is cheaper to test at 30 than at 100, so early stage is an advantage for experiment velocity, not a handicap.
Watch out Precision is genuinely lower — treat a 30-person result as directional, and re-confirm before betting the company on the exact magnitude.
- 2
Look for a large proportion, not an absolute number
Judge the result by whether a big share of the small group did the target action (e.g. ~20 of 30), which signals a real behavioral pattern rather than noise.
- 3
Trace one step before the decision
Instead of chasing the end conversion, identify the succeeding events — what a user must do just before converting — and measure whether the experiment moved that precursor step.
Pro tip Users make decisions based on preceding events; fixing the setup step often unlocks the aha step.
In the wild
Widjaja argues that startups too early to have 'enough' users should still experiment, because a sample of 30 returns the same trend as 100 with lower precision — and 30 is infinitely more informative than the zero data most teams settle for.
→ Reframes 'we're too small to test' into a reason to test more, since small-scale experiments are cheap and fast.
Common mistakes
Waiting for 'enough' data before running any experiment
Teams postpone learning until they have scale, sitting on zero data. The trend is available at 30 users, so waiting only delays actionable insight while the cheap-to-test window is open.
Fixating on the final conversion metric
Chasing 'get users from step 0 to 100' ignores that users decide based on the step just before conversion; not measuring that precursor step hides the real lever.
Is it for you?
Best for
Pre-product-market-fit founders who believe they have too few users to learn anything from experiments.
Not ideal for
High-stakes decisions requiring precise effect sizes, or detecting small effects where 30 users genuinely lack statistical power.
From the transcript
“even if you have a sample size of the data you get back generally does not change but its precision will so mathematically speaking you're…”
“what's better than having zero is definitely 30”
“you're looking for like 20 of them to do something like a large percentage of that group does something”
“users make decisions based on succeeding events so what's one step before the user makes that decision”
From the episode
How to scrappily hire for, measure, and unlock growth
Crystal Widjaja, Gojek and Kumu