The 1,000-Experiments 'What Would Need To Be True' Goal
Set an audacious experiment-volume goal — not to hit it, but to force the conversations that unlock a whole experimentation culture.
- Difficulty
- Advanced
- Time to result
- ~ongoing to results
- Steps
- 4
- Confidence
- 90%
A goal-setting technique for scaling an experimentation culture. Set a deliberately ambitious experiment-count target so the team must ask 'what would need to be true' to hit it — which surfaces the cross-functional enablement (life-cycle marketing, ASO, no-code screens, observability) that actually builds the muscle. The number is a forcing function, not the point.
Origin
Cheng set Chess.com a target of 1,000 experiments a year (from ~50 in 2023 to ~250 in 2025). He openly admits he made the number up and doesn't much care if they hit it — the value is the conversations it forces.
Core principles
- 01The purpose of a goal is to enable conversations about what would need to be true to reach it.
- 02Volume alone is worthless — 1,000 experiments with no learning or impact is wasted time.
- 03Hitting the number is less important than accomplishing the enabling changes it surfaces.
- 04The system matters as much as, or more than, any single experiment.
How to run it
- 1
Set the audacious count
Pick an experiment-volume target well beyond current pace to stretch the organization's thinking.
Pro tip It's fine to invent the number — Chess.com's 1,000 was made up.
- 2
Ask what would need to be true
Work backward: reaching the number forces the realization that PM and engineering alone can't run that many experiments.
- 3
Unlock cross-functional experimentation
Enable other functions to experiment: life-cycle marketing (push/email copy), app-store screenshots and keywords, content marketing, and no-code configurable screens (home, pricing).
Pro tip Have engineering enable no-code on high-traffic screens so tests ship without engineering support.
- 4
Instrument observability
Track progress and build observability so you can confirm experiments are actually learning and impacting, not just counting.
Watch out Don't let the vanity of the count replace real learning and impact.
In the wild
Chess.com went from practically zero experiments before 2023, to ~50 in 2024, to ~250 in 2025, with a stated 1,000 target for the next year — driving experimentation into marketing, ASO, and no-code screens.
→ Culture shifted from lumpy, intuition-led growth toward systematic experimentation; leadership bought in.
Common mistakes
Chasing the number for its own sake
1,000 experiments with no learning or impact is a waste of time; the goal exists to force enabling conversations.
Setting goals in a vacuum
People must see wins working in practice to buy in — you can't just decree a target from the top.
Is it for you?
Best for
Growth or product leaders trying to shift an intuition-led company toward an experimentation culture.
Not ideal for
Teams without instrumentation or scale to run and read many experiments, where volume goals would just produce noise.
From the transcript
“The whole point of setting a goal is that you can have conversations about what would need to be true to actually hit that goal”
“don't tell the team, but I don't actually care that much if we actually hit a thousand”
“The system matters just as much as any given experiment. Probably even more.”
From the episode
How to find hidden growth opportunities in your product
Albert Cheng (Duolingo, Grammarly, Chess.com)