Measurements vs. Insights Instrumentation
A measurement is an observed fact; an insight answers 'why' and changes what you do — instrument for the latter.
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 4
- Confidence
- 95%
Most analytics efforts fail because teams track data to monitor KPIs rather than to generate insights. A measurement is a raw observation ('power users book 4x more'); an insight adds the segmented context and the 'why' that lets you act ('power users convert on free-shipping only on high-GMV baskets'). The fix is instrumenting rich properties onto each event so you can segment behavior, form hypotheses, and find causal reasons — not just log that an event happened.
Origin
From Crystal Widjaja's widely-read Reforge blog post 'Why most analytics efforts fail'; she uses the 'made to stick' orange/construction-site mental model (Chip & Dan Heath) to argue for concrete, contextual metrics.
Core principles
- 01Real news is information that changes what you do; if it doesn't change your behavior, it's just entertainment
- 02A measurement is an observation; an insight is an observation plus context plus a 'why' you can act on
- 03The value of data comes when the 'why' is answered and you act differently
- 04Instrument properties onto events so you can segment who does what, then hypothesize and test
- 05'Retention' as a goal is too vague — constrain the target to a concrete, actionable behavior
How to run it
- 1
Stop treating metrics as entertainment
Audit whether the numbers you gather actually change any decision. If a dashboard is 'interesting' but never alters what you build or spend, you're consuming entertainment, not insight.
Pro tip Define, for each core metric, the decision it would change — if there isn't one, stop tracking it as a goal.
- 2
Separate observations from insights
Recognize that a transactional fact ('power users book 4x more') is only an observation. Push each one toward an insight by asking what context and 'why' would make it actionable.
Pro tip Insight test: 'She's with a man I don't know' is an observation; 'it's her cousin, I'm just paranoid and need to change my behavior' is the insight — the why that changes what you do.
- 3
Instrument rich properties onto every event
For each event, attach the contextual properties that describe the user's experience — for a map-load event: number of drivers visible, pickup city, lat/long, surge status, minimum fare, voucher presence.
Pro tip The symptom of bad instrumentation is many event rows each carrying one property or none; good specs attach many contextual properties per event.
Watch out Without properties you literally cannot know context (e.g. that only two drivers were on screen), so you can never explain why a user didn't convert.
- 4
Segment, hypothesize, and test the 'why'
Use the properties to segment behavior (users seeing 2 drivers convert worse than those seeing 5), form a causal hypothesis, then test it and confirm causation before acting.
Pro tip Do the 'second layer' — after spotting a correlation, ask in which cities/coordinates it occurs and why, instead of stopping at 'that's weird.'
In the wild
'GoFood users use vouchers more' is a bare observation. The insight — 'power users are more likely to use a free-shipping discount on a high-GMV basket than non-power users' — tells you exactly where to spend: give power users free-shipping on high-GMV baskets, and don't waste it on non-power users who won't convert better.
→ Converted a vague observation into a targeted, spend-changing marketing rule.
By instrumenting the number of visible drivers onto the map-load event, the team could see that users seeing only two drivers converted far less than those seeing five, then drill into which cities and coordinates that occurred in to ask why.
→ Turned a raw event into a segmentable, explainable driver of conversion instead of an unexplained anomaly.
Common mistakes
Tracking events with one property or none
Logging that an event fired without contextual properties strips away the user's experience, so you can't segment or explain non-conversion — the single most common instrumentation failure.
Using 'retention' as the goal
Retention is too unspecific to action; like being told to picture 'everything orange' vs 'everything orange on a construction site,' teams need a concrete constrained behavior to grasp and act on it.
Is it for you?
Best for
Product and growth teams whose dashboards track KPIs but rarely produce decisions they can act on.
Not ideal for
Teams that first need basic data plumbing (a single warehouse) before richer event instrumentation is worth the effort.
From the transcript
“real news is information that changes what you do in the real world and if you don't change what you're doing what you are doing…”
“measurements do not equate to insights a measurement would be an observation it's a data point in your database”
“the insight will provide value when you have this why answered”
“the symptom of a bad data tracking approach is you have a ton of rows with a ton of events but every event has like…”
“being able to do the second layer approach of the why and not just stop”
From the episode
How to scrappily hire for, measure, and unlock growth
Crystal Widjaja, Gojek and Kumu