Intentionality Before Experimentation
Set the product destination first, then use experiments to find the route.
- Difficulty
- Advanced
- Time to result
- ~ongoing to results
- Steps
- 6
- Confidence
- 97%
Intentionality Before Experimentation separates product direction from product optimization. First decide the customer problem and future experience you intend to create. That direction can draw on immersion, judgment, and intuition; Saxena describes intuition as data that is not yet statistically significant. Then build a real expression of the vision and use A/B tests, customer feedback, and iteration to find the fastest route toward it. The GPS analogy is central: a navigation system can optimize the route only after a person chooses the destination. Teams must persist long enough for a novel idea to mature, celebrate failed experiments as learning, and recover rapidly when evidence disproves a premise. Data informs the route and can eventually force a pivot, but it does not replace the responsibility to choose where the product should go.
Origin
Saxena learned product intentionality from Kevin Systrom at Instagram and Brian Chesky at Airbnb. Instagram's all-in commitment to Stories is his main example of choosing a destination before optimizing it.
Core principles
- 01Experiments can optimize a route but cannot choose a meaningful destination.
- 02Gut is accumulated evidence that may not yet be statistically significant.
- 03Product vision requires selecting which customer needs to prioritize.
- 04Strong ideas need impatience to start and patience to mature.
- 05Failure should improve the next move through rapid recovery.
How to run it
- 1
Choose the destination
State the future customer experience and the problem it should solve. Make the direction specific enough that the team can distinguish progress from unrelated metric gains.
Pro tip Use the GPS test: if the statement only describes a route or metric, the destination is still missing.
Watch out An experiment backlog is not a product vision.
- 2
Surface the intuition
Explain what customer immersion, market observation, or operator experience makes the direction plausible. Treat intuition as evidence with uncertain statistical strength.
Pro tip Name the observations that produced the instinct so others can challenge the premise.
Watch out Gut without immersion is merely preference.
- 3
Build the intended experience
Create a real version that expresses the direction instead of testing disconnected assumptions forever. Establish a new baseline from which the product can improve.
Watch out Overprotecting the existing business can prevent the new behavior from receiving a fair test.
- 4
Test the route
Use A/B tests, community feedback, and product data to determine whether execution is moving toward the chosen experience. Change the route when a better one appears.
Pro tip Design tests around progress toward the end state, not around whatever metric is easiest to move.
- 5
Persist through early noise
Give a bold idea enough time and iteration to become legible. Be impatient about starting but patient about seeing the direction through.
Pro tip Set a learning horizon before launch so a weak first result does not trigger an automatic retreat.
Watch out Persistence is not denial; revisit the premise when evidence changes materially.
- 6
Recover or pivot
When evidence invalidates the route or destination, capture the learning and move quickly. Strong opinions and fast changes are compatible when the evidence is genuinely new.
Watch out Do not use sunk cost or reputation to preserve a disproven direction.
In the wild
Kevin Systrom intended Stories to become a real-time view of what was happening in the world. Even though phone data and user research could support sharing older photos, the original product restricted selection to recent content. Instagram committed to the format, accepted effects on the feed, and iterated from the new baseline rather than asking tests to decide what product to become.
→ Stories developed around a coherent real-time behavior instead of becoming a collection of individually requested sharing options.
Common mistakes
Asking tests to choose the destination
A/B tests compare routes and implementations. Without an intended end state, they optimize local metrics without defining the product.
Calling unsupported preference intuition
Saxena's intuition comes from immersion with customers, builders, and the market. Personal taste without that input lacks the same evidentiary basis.
Killing the idea at the first weak result
Novel behaviors often need sustained iteration. Judging the destination from an immature implementation can destroy a strong idea prematurely.
Is it for you?
Best for
Founders and product leaders creating a new behavior, format, or experience where historical data cannot specify the answer.
Not ideal for
Mature, low-risk optimizations where the desired outcome is already clear and controlled testing can answer the remaining question.
From the episode
Sanchan Saxena (VP of Product at Coinbase) on the inside story of how Airbnb made it through Covid; what he’s learned from Brian Chesky, Brian Armstrong, and Kevin Systrom; much more