Unquantifiable-First Decision-Making
Demote metrics to a support function and let taste, fun, and delight drive — because Goodhart's law means any metric-as-goal degrades.
- Difficulty
- Advanced
- Time to result
- ~ongoing to results
- Steps
- 4
- Confidence
- 88%
Because any metric that becomes a goal stops being a good metric (Goodhart's law = business overfitting), Lütke refuses to run Shopify on KPIs/OKRs as the objective. He estimates only ~20% of the most valuable product work is fully quantifiable, leaving 80% addressable only by taste, quality, passion, fun, and delight. Data is heavily instrumented but treated as a 'cockpit' — the pilot still makes the decision, and strong emotions (especially fun) are treated as leading indicators that metrics will follow.
Origin
Tobi Lütke, explicitly grounding it in Goodhart's law (which he equates to ML overfitting on a loss function); Shopify's growth head Archie Abrams described the product team operating on taste over specific goals.
Core principles
- 01Any metric that becomes a goal ceases to be a good metric (Goodhart / overfitting)
- 02No single metric is a complete heuristic for a complex, multi-tension business
- 03Only ~20% of the most valuable product work is fully quantifiable
- 04Unquantifiables — taste, quality, passion, fun, delight — must be made stable, legitimate things people defer to
- 05Fun is a leading indicator: if metrics dip but fun rises, metrics follow with a delay
How to run it
- 1
Instrument everything, but as a cockpit
Invest heavily in data systems that put every cohort and experiment at your fingertips — then treat them as instruments for a pilot, not as the decision-maker.
Pro tip Keep sophisticated holdouts and rollout systems so you get told immediately if something goes the wrong way.
- 2
Refuse metric-as-goal
Don't set KPIs/OKRs as the objective. A metric elevated to a goal will be overfit and gamed, just like a model cheating its benchmark (or kids cheating for marks).
Watch out Environments where the only way to get promoted is driving a metric up train people to overfit — avoid recreating that.
- 3
Legitimize unquantifiables
Make taste, quality, craft satisfaction, and 'this is just really great, we're shipping it' into stable things the organization genuinely defers to.
Pro tip Ask product questions metrics can't answer: are you representing your brand? Are you proud of this? Does it feel like your own?
- 4
Read fun as a leading indicator
Treat fun and delight as upstream signals. If all metrics point down but everyone reports having far more fun, expect the metrics to turn up after a delay.
Watch out If the delayed metric improvement doesn't materialize, then adjust course — fun is a signal, not a permanent excuse.
In the wild
Shopify deliberately has no KPIs or OKRs in the classic sense, yet is extremely data-informed, with systems that let founders dig into any constituent of a cohort formed 15 minutes ago. Metrics take a support role; decisions often defer to less quantifiable, sometimes emotional judgments.
→ The product team can ship things justified by taste and craft, addressing the ~80% of value space that purely quantitative teams leave untouched.
Lütke contrasts an easily-quantified signal ('do the unit tests pass') with harder proxies like the deep satisfaction a craftsperson feels doing a job well — arguing the latter is often the better proxy for the outcome you actually want.
→ By allowing craft satisfaction to count, Shopify optimizes for the outcomes metrics only approximate, while safety-net systems catch anything that truly regresses.
Common mistakes
Optimizing only the quantifiable
Fully-quantifiable work is where everyone competes because it's gratifying and game-like, but it's only ~20% of the value — chasing it alone leaves 80% of value unaddressed.
Elevating a metric to a goal
The moment a metric becomes the goal it gets overfit and gamed (Goodhart's law), so it stops measuring what you actually care about.
Is it for you?
Best for
Founder-led product companies with strong instrumentation and high-taste leaders who want to escape metric-gaming and address the unquantifiable majority of product value.
Not ideal for
Organizations without trusted taste-makers or robust safety-net analytics, where dropping KPIs would just remove accountability with nothing rigorous to replace it.
From the transcript
“any metric that um becomes a goal ceases to be a a good metric”
“the overlap of most valuable things you can do with a product and the things that happen to be fully quantifiable is like maybe 20%”
“we think about it as a cockpit for a uh pilot and um um uh the decisions are still made by pilots”
“if all the metrics are pointing down but everyone says my God I'm having so much um more fun I think that the very next…”
From the episode
Tobi Lütke’s leadership playbook: Playing infinite games, operating from first principles, and maximizing human potential (founder and CEO of Shopify)