The Four-Box Framework
Write your hypothesis in words before you touch data, so you can debug the idea and the measurement separately.
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 5
- Confidence
- 90%
A napkin-simple tool for turning a hypothesis into a measurable test. You draw four boxes: two on top for the words (concept A leads to concept B) and two below for the data (how each concept is measured). You always start with words, validate the sentence with stakeholders, then attach data proxies. When results fall apart, you can point to whether the problem is the idea (words) or the measurement (data) — depersonalizing the debate.
Origin
Developed by Nicole Forsgren when she was a professor; she still draws it on napkins at bars and conferences, and former students report it's the most useful thing they learned from her.
Core principles
- 01Always start with words, never with data.
- 02The top row is a hypothesis you can state as a sentence and get stakeholders to agree with.
- 03The bottom row is the data proxy for each concept — name what stands in for what.
- 04Separating words from data lets you locate a failure as either a bad idea or a bad measurement, not a personal attack.
- 05Advanced mode reverses it: start from available data, then go back up to words and check what sentence the correlations actually represent.
- 06Beware spurious correlations — interrelated data will happily produce relationships that mean nothing.
How to run it
- 1
Draw the four boxes
Two boxes on top, two on the bottom, aligned. Label the top row 'words' and the bottom row 'data.' Draw an arrow between the two top boxes and between the two bottom boxes.
- 2
Fill the top boxes with words first
Put your hypothesized cause in the first top box and the hypothesized effect in the second (e.g. 'customer satisfaction' → 'return customers'). It should read as a sentence.
Watch out Do not start with data. Always start with words — starting from data is how people run every correlation they can think of without ever stating what they're actually testing.
- 3
Validate the sentence with stakeholders
Take the worded hypothesis to a few people — managers, stakeholders — and ask whether they agree this is actually what you're doing before you invest in measuring it.
- 4
Attach data proxies in the bottom boxes
For each top-box concept, decide how you'll measure it (e.g. customer satisfaction → CSAT or NPS; return customers → repeat purchases via the website, a follow-up survey, or a referral link). Get explicit that each is a proxy standing in for the concept.
Pro tip Don't use your effect as a proxy for your cause — if you think satisfaction leads to return customers, you can't measure satisfaction by return customers.
- 5
Run correlations and diagnose failures by box
Hand the data boxes to a data scientist and run correlations over time. If the result falls apart, point to the specific box: bad data quality, missing data, a bad proxy — or go back up to the words and conclude the hypothesis itself doesn't hold.
Pro tip This depersonalizes disagreement — 'this proxy was problematic' instead of 'Lenny's idea was stupid.'
Watch out In advanced (data-first) mode, always translate the correlations back into a sentence and interview to confirm before trusting them — spurious correlations are everywhere.
In the wild
Forsgren walks through: top boxes read 'customer satisfaction → return customers'; the bottom boxes ask how to measure each — CSAT/NPS (and, as a stretch, spend) for satisfaction, and website repeat visits, a follow-up survey, or a referral link for return customers. She's careful not to let 'return customers' proxy for satisfaction since that's the very relationship being tested.
→ A vague belief becomes a crisply testable, communicable hypothesis with named proxies, ready to hand to a data scientist.
Lenny applies the framework to engineering: top boxes 'improving build time → improves speed and stability,' bottom boxes 'the data points I have available to measure build time.'
→ Shows the four-box framework generalizes from business hypotheses to engineering-metric hypotheses.
Common mistakes
Starting with data instead of words
People run every correlation they can think of without ever writing down what relationship they expect or what it means — so they can't communicate the finding and can't tell a real signal from a spurious one.
Using the effect as a proxy for the cause
If your hypothesis is that satisfaction leads to return customers, measuring satisfaction via return customers collapses the test — the proxy must be independent of the outcome you're trying to predict.
Is it for you?
Best for
Product managers, analysts, and leaders who have a hypothesis they want to test and need to get crisp about the idea and its measurement before running any analysis.
Not ideal for
Situations with no hypothesis at all, or pure exploratory data mining where you're not yet trying to test a stated relationship.
From the transcript
“draw four boxes on piece of paper two on the top two on the bottom”
“always start with words do not start with data you always start with words”
“if something here falls apart now you can point to the data boxes and we can get mad about the things in the data boxes…”
“the challenge is I will see people run every correlation they could think of but they haven't turned it into a word or a sentence…”
From the episode
How to measure and improve developer productivity
Nicole Forsgren (Microsoft Research, GitHub, Google)