LLenny's Podcast
← All episodes
Annie Duke (author of “Thinking in Bets” and “Quit,” former 02 May 2024

This will make you a better decision-maker

7Frameworks
14Insights

Episode overview

Lenny Rachitsky talks with Annie Duke about turning implicit judgments into explicit rubrics so teams can test intuition and improve decisions. Duke argues that discovery and final decisions should happen independently, while meetings should focus on discussing genuine disagreements. They also cover shortening apparently long feedback loops through intermediate forecasts, pairing premortems with kill criteria, and recognizing that people usually quit too late because of uncertainty, sunk costs, and identity.

Key ideas

  • Make implicit judgments explicit through structured rubrics, shared definitions, and measurable forecasts.
  • Collect opinions independently and asynchronously before discussion to reduce conformity and outsized influence from senior, loud, or confident participants.
  • Use meetings for discussion, especially disagreement; gather inputs and make decisions outside the meeting.
  • Shorten long feedback loops by identifying necessary intermediate milestones and outcomes correlated with the eventual goal.
  • Pair a premortem with explicit kill criteria and precommitted actions for when warning signals appear.
  • If quitting has entered consideration, biases such as sunk cost and endowment often mean the best stopping point has already passed.
  • Mental time travel helps decision-makers escape the emotional intensity of the present and evaluate choices from a future perspective.

Transcript available · source text is retained privately and is not published

Frameworks in this episode

Leadership6 steps

Discover–Discuss–Decide (Nominal Group Meeting Protocol)

Only discussion belongs in the meeting; discovery and decision happen independently and asynchronously

Read →
Strategy4 steps

Kill Criteria (Premortem Plus Pre-Commitment)

A premortem only works if each predicted failure signal is bound to a pre-committed action

Read →
Strategy5 steps

Make the Implicit Explicit: Decision Rubrics and Mediating Judgments

Excavate your gut into a rubric, then score the rubric against reality to learn where your gut is wrong

Read →
Mindset3 steps

Mental Time Travel

Ask how you'll feel about this in ten years to dissolve the focusing effect of right now

Read →
Self-Mastery5 steps

Quit Before You're Certain (Would I Start This Today?)

If you're thinking about quitting, you're probably already late — waste is prospective, not retrospective

Read →
Communication4 steps

The Nevertheless Close

Let people be fully heard, then decide anyway — the word that separates listening from agreeing

Read →
Strategy4 steps

There Is No Such Thing As a Long Feedback Loop

Feedback loops are as long as you choose — shorten them with necessary-but-not-sufficient milestones

Read →

People & resources mentioned

Attributed to the moment in the episode. Timestamps are approximate.

People · 17

  • Annie DukeMentionsAnnie is the author of the bestselling book thinking in bets and also her more recent book quit

    “today my guest is Annie Duke Annie is the author of the bestselling book thinking in bets”

    ▶ Watch at ~0:30approx.
  • Daniel KahnemanMentionshis influence was so huge you know and his intellect was so huge and his Insight was was so change making

    “I want to start with a question about Daniel Conan I know you two are very close”

    ▶ Watch at ~4:00approx.
  • Angus DeatonMentionsmentioned

    “Danny did work on happiness as it relates to wealth conoman uh and a collaborator a a Angus Deon”

    ▶ Watch at ~6:30approx.
  • Matthew KillingsworthMentionsmentioned

    “but then a decade later Matthew K Killingsworth said that's not true”

    ▶ Watch at ~7:00approx.
  • Barbara MellersMentionsmentioned

    “they joined forces along with bar mellers um to basically try to you know to try to sort of resolve that issue”

    ▶ Watch at ~7:30approx.
  • Daniel KahnemanMentionshis influence was so huge you know and his intellect was so huge and his Insight was was so change making

    “I'll actually quote Danny Conan on this one nothing is as important as it seems when you're thinking about it”

    ▶ Watch at ~15:00approx.
  • Becky KennedyMentionsmentioned

    “it connects to something uh Dr Becky teaches if you know her”

    ▶ Watch at ~19:30approx.
  • Todd JacksonMentionsDescribed as a former Gmail product leader and venture capitalist.

    “we actually just had Todd Jackson on the podcast talking about Todd love Todd talking about product Market fit”

    ▶ Watch at ~44:30approx.
  • Brett BersonMentionsit's first rounds podcast by Brett Baron he interviews a lot of operators

    “also so ask Brett Bon your colleague at first round what to ask you”

    ▶ Watch at ~45:00approx.
  • Renata QuintiniMentionsmentioned

    “Renegade partners that was ran uh wik and Rada quintini they're they're incredible”

    ▶ Watch at ~45:30approx.
  • Roseanne WincekMentionsmentioned

    “Renegade partners that was ran uh wik and Rada quintini they're they're incredible”

    ▶ Watch at ~45:30approx.
  • Josh KopelmanMentionsJosh Kopelman is the partner there who was just a massive supporter of mine

    “when I was talking to Josh coppelman about it he said well what do you mean we don't get an exit for 10 years”

    ▶ Watch at ~48:00approx.
  • Lenny RachitskyMentionstoday we've got another very special compilation episode something I've been pulling on more and more with the podcast and the newsletter

    “here's my question for you Lenny is 16 months shorter than 10 years”

    ▶ Watch at ~51:30approx.
  • Maurice SchweitzerMentionsmentioned

    “I actually did this work with Marie schwitzer and Len aandi who are both a pen”

    ▶ Watch at ~1:05:30approx.
  • Jeff BezosCoinedthe Jeff Bezos thing that like the success of today was set in motion three years ago

    “it's why you know basos has the 70% rle to try to roll people back and be willing to accept that uncertainty”

    ▶ Watch at ~1:12:00approx.
  • Richard ThalerMentionsmentioned

    “as Richard Thor put it most people won't quit until actually isn't a decision”

    ▶ Watch at ~1:12:30approx.
  • Stewart ButterfieldCoinedStewart founded Flickr and then Slack, which he sold to Salesforce in one of the biggest acquisitions in tech history at the time.

    “so seart Butterfield creating a product called glitch and glitch is a a massive multiplayer online World building Cooperative game”

    ▶ Watch at ~1:15:00approx.

Resources · 30

  • Thinking in BetsCoinedbook · Annie Duke

    “Annie is the author of the bestselling book thinking in bets”

    ▶ Watch at ~0:30approx.
  • Quit: The Power of Knowing When to Walk AwayCoinedbook · Annie Duke

    “and also her more recent book quit the power of knowing when to walk away”

    ▶ Watch at ~0:30approx.
  • First Round CapitalMentionscompany

    “she's also a special partner at first round Capital which we spent some time on”

    ▶ Watch at ~1:00approx.
  • World Series of PokerMentionsevent

    “including winning a World Series of Poker bracelet”

    ▶ Watch at ~1:00approx.
  • World Series of Poker Tournament of ChampionsMentionsevent

    “she's the only woman who's won the World Series of Poker Tournament of Champions”

    ▶ Watch at ~1:00approx.
  • National Heads-Up Poker ChampionshipMentionsevent

    “and the national poker headsup Championship”

    ▶ Watch at ~1:00approx.
  • Vanta AIMentionssoftware · Vanta

    “you can save hours by completing security questionnaires with Vana AI”

    ▶ Watch at ~2:30approx.
  • VantaRecommendssoftware · Christina Cacioppo

    “get $1,000 off vanta when you go to vana.com Lenny”

    ▶ Watch at ~3:00approx.
  • UserTestingRecommendssoftware · UserTesting

    “get started today at usertesting.com Lenny”

    ▶ Watch at ~3:30approx.
  • Thinking, Fast and SlowMentionsbook · Daniel Kahneman

    “I got asked uh in a class that I was teaching about um thinking fast and slow”

    ▶ Watch at ~5:00approx.
  • AirtableRecommendssoftware · Airtable

    “you can actually create a rubric that lives on like air table or Koda or Google Sheets”

    ▶ Watch at ~30:00approx.
  • Google FormsRecommendssoftware · Google

    “with Google you can use Google forms in order to do it and then it dumps into a spreadsheet that only you can see”

    ▶ Watch at ~30:00approx.
  • Google SheetsRecommendssoftware · Google

    “you can actually create a rubric that lives on like air table or Koda or Google Sheets”

    ▶ Watch at ~30:00approx.
  • CodaRecommendssoftware · Coda

    “you can actually create a rubric that lives on like air table or Koda or Google Sheets”

    ▶ Watch at ~30:00approx.
  • CensusMentionscompany

    “two of my portfolio companies webflow and census are LinkedIn success stories”

    ▶ Watch at ~34:30approx.
  • WebflowMentionssoftware · Webflow

    “two of my portfolio companies webflow and census are LinkedIn success stories”

    ▶ Watch at ~34:30approx.
  • LinkedIn AdsRecommendsproduct · LinkedIn

    “make B2B marketing everything it can be and get $100 credit on your next campaign”

    ▶ Watch at ~35:00approx.
  • Renegade PartnersMentionscompany · Roseanne Wincek and Renata Quintini

    “I have another client too who they're amazing I Renegade partners”

    ▶ Watch at ~45:30approx.
  • Mount EverestMentionsplace

    “that's why there's many people who climb Everest in the middle of a blizzard even though they shouldn't be doing that”

    ▶ Watch at ~1:09:30approx.
  • GlitchCoinedsoftware · Stewart Butterfield and Tiny Speck

    “so seart Butterfield creating a product called glitch and glitch is a a massive multiplayer online World building Cooperative game”

    ▶ Watch at ~1:15:00approx.
  • Andreessen HorowitzMentionscompany · Marc Andreessen and Ben Horowitz

    “they have incredible investors in andrees and horites and Excel they have $6 million in the bank”

    ▶ Watch at ~1:15:30approx.
  • AccelMentionscompany

    “they have incredible investors in andrees and horites and Excel they have $6 million in the bank”

    ▶ Watch at ~1:15:30approx.
  • SlackCoinedsoftware · Slack Technologies

    “that thing which had no name at the time now gets a name which is searchable log of all company knowledge which is slack”

    ▶ Watch at ~1:18:30approx.
  • MavenRecommendswebsite · Maven

    “if you're interested in that you can go to Maven and check it out”

    ▶ Watch at ~1:19:30approx.
  • Thinking in BetsCoinedbook · Annie Duke

    “I have a substock thinking and be uh please go check that out”

    ▶ Watch at ~1:19:30approx.
  • Effective Decision-MakingCoinedcourse · Annie Duke

    “I teach a class on maven.com twice a year on effective decision- making”

    ▶ Watch at ~1:19:30approx.
  • Alliance for Decision EducationCoinedcompany · Annie Duke and co-founders

    “I co-founded an organization called The Alliance for decision education”

    ▶ Watch at ~1:20:00approx.
  • Apple PodcastsRecommendssoftware · Apple Inc.

    “you can subscribe to the show on Apple podcast Spotify or your favorite podcast app”

    ▶ Watch at ~1:20:30approx.
  • SpotifyRecommendssoftware · Spotify

    “you can subscribe to the show on Apple podcast Spotify or your favorite podcast app”

    ▶ Watch at ~1:20:30approx.
  • Lenny's PodcastMentionspodcast · Lenny Rachitsky

    “you can find all past episodes or learn more about the show at Lenny podcast.com”

    ▶ Watch at ~1:20:30approx.

Spot an error or want something removed? Request a correction or removal.

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Myth Buster· 1

Myth Buster47:30

Poker Is Much Noisier Than You Think

VCs told Duke her decision methods didn't apply because in poker you get instant feedback (win or lose the hand) while their feedback loops are a decade long. She pushes back: poker is far noisier than outsiders assume — winning a single hand tells you almost nothing, and you rarely see your opponent's cards, so you must play many hands before you know if you actually have an edge.

  • Winning a hand doesn't tell you why you won — a single hand is nearly useless signal
  • You almost never see your opponent's cards, leaving deep uncertainty
  • You need many, many hands before you know whether you truly have an edge
  • Instant win/loss is not the same as fast learning about decision quality

“oh poker is much noisier than you think because when I win a hand I have no idea why”

Annie Duke · 47:30
#poker#feedback-loops#uncertainty#decision-making

Hot Take· 3

Hot Take20:30

Even Kahneman Couldn't Use His Own Bias Research Day-to-Day

Lenny raises that people asked Kahneman whether knowing all the biases made his life optimal, and Kahneman famously said no — the knowledge didn't actually change his day-to-day decisions. Duke uses this to frame the real question: how much better can someone actually get at making decisions?

  • Kahneman said he couldn't use the biases in practice; the knowledge didn't affect his daily decisions
  • Knowing about biases and errors doesn't automatically improve real decisions
  • The gap between knowing and doing is the central problem in decision-making

“I can't use these in practice this is just stuff that I have learned but it doesn't actually impact my day-to-day”

Lenny · 20:30
#decision-making#kahneman#biases
Hot Take31:30

"The Word Alignment Is Stupid"

Duke argues bluntly that the goal of 'alignment' in group decisions is a myth that should be dropped. Ten different people won't leave a room agreeing, and expecting them to isn't reality. Worse, making agreement the goal turns meetings coercive — you end up trying to convince people you're right instead of conveying why you believe what you do.

  • Alignment doesn't exist — different people won't come out of a room agreeing
  • Expecting agreement isn't reality and papers over real disagreement
  • If the goal is alignment/agreement, the meeting becomes coercive
  • You should convey why you believe something, not work to convince others you're right

“the word alignment is stupid and it shouldn't be used”

Annie Duke · 31:30

“if the goal is alignment if the goal is agreement then the meeting becomes coercive and you never want that”

Annie Duke · 32:30
#decision-making#meetings#alignment#leadership
Hot Take50:00

There Is No Such Thing as a Long Feedback Loop

Duke's bold claim: a feedback loop is only as long as you choose to make it. VCs say they can't measure decision quality because exits take a decade — but she points out all the correlated signals along the way (does it fund at Series A, product-market fit, traction, churn). Track those and a '10-year' loop collapses to roughly 16 months.

  • You choose whether to live in a long feedback loop; you can shorten it
  • Shorten it by tracking things necessary-but-not-sufficient or correlated with the desired outcome
  • No billion-dollar company failed to fund at Series A — so Series A funding is a fast, correlated signal
  • Companies raise Series A in ~16 months, far shorter than a 10-year exit horizon

“there is no such thing as a long feedback loop”

Annie Duke · 50:00

“the feedback loop is as long as you choose it to be”

Annie Duke · 51:30
#feedback-loops#venture-capital#decision-quality#signals

Explainer· 4

Explainer06:30

The $75,000 Money-and-Happiness Debate, Resolved Together

Duke uses Kahneman's happiness research as a case study in adversarial collaboration. Kahneman and Angus Deaton published in 2010 that money stops buying happiness beyond $75,000. A decade later Matthew Killingsworth found the opposite — happiness keeps rising with income — and rather than feuding, they joined forces to resolve it.

  • Kahneman and Deaton's 2010 result: money doesn't buy happiness beyond $75,000 once basic needs are met
  • That threshold would be higher today after adjusting for inflation
  • A decade later Matthew Killingsworth found money and happiness are correlated across all income levels
  • Instead of tearing each other down, they collaborated (with Barbara Mellers) to reconcile the findings

“the idea was money doesn't buy happiness Beyond $75,000”

Annie Duke · 06:30
#happiness#kahneman#research#money
Explainer21:00

Structured Hiring Lifts Your Hit Rate From 50% to 65%

Duke explains Kahneman's early work on hiring: taking an unstructured 'I know a great PM when I see one' process and making the implicit criteria explicit, turning them into a decision rubric and structured interview. Applying intuition after that process (not before) raised the hiring hit rate from about 50% to 65% — a huge gain. The catch: almost nobody actually does it.

  • Excavate the implicit model behind 'I know a great candidate when I see one' and make it an explicit rubric
  • Use intuition after the structured process, not before it
  • Hit rate on hiring rose from about 50% to 65%
  • The edge is enormous because so few people are willing to actually do it

“it was from about a 50% hit rate to 65 that's pretty huge”

Annie Duke · 22:00

“problem is Nobody Does it”

Annie Duke · 22:30
#hiring#decision-making#structured-process#intuition
Explainer54:00

Why Long Feedback Loops Feel Psychologically Safe

Duke explains why people prefer long feedback loops: if you got early into a winner like Uber, staying uncertain lets you keep the reputation without risking finding out your bet was luck, not skill. At the core, humans hate feeling wrong in the moment even when it helps them long-term — the same reason we eat cupcakes we know are bad for us.

  • A tight feedback loop risks quickly revealing you were wrong
  • Being early into a hit like Uber could be insight — or just a buddy's startup; you can't tell without measuring quality
  • Keeping the loop long protects a positive self-narrative and reputation
  • Humans trade the long run for feeling good now — like eating chocolate and cupcakes we know are bad for us

“it's very very difficult for human beings to deal with feeling wrong in the moment even if it helps them in the long run”

Annie Duke · 54:00

“that's why we're all eating chocolate and cupcakes and stuff that we know is bad for us because it feels good”

Annie Duke · 54:30
#decision-making#psychology#feedback-loops#self-narrative
Explainer1:11:00

By the Time You Consider Quitting, You've Already Waited Too Long

Duke explains why people quit far too late. We start things under uncertainty, accept luck and hidden information, and then refuse to walk away until we know for sure — but that certainty only comes after the thing has already blown up. Sunk cost, endowment (over-valuing what we've built), and identity all bias us against stopping. Crucially, waste is a prospective problem, not a retrospective one.

  • The same uncertainty that makes starting hard makes stopping hard — we won't quit until it 'isn't a decision'
  • Sunk cost, endowment (over-valuing what we own), and identity all bias us against quitting
  • Waste is prospective: if you wouldn't start it today, everything you put in going forward is the real waste
  • People quit jobs/relationships/products only when they feel they have no other choice

“the data is pretty strong that by the time you quit it's probably long after should”

Annie Duke · 1:11:30

“waste is a a prospective problem not a retrospective one”

Annie Duke · 1:13:30
#quitting#sunk-cost#decision-making#endowment

Story· 4

Story04:30

What Annie Duke Learned From Daniel Kahneman's Humility

Annie Duke reflects on her late friend Daniel Kahneman, describing how his defining trait was open-mindedness rather than his towering intellect. He readily admitted when his own published work (like priming) failed to replicate, and he pioneered 'adversarial collaboration' — writing papers with people who disagreed with him to resolve the dispute together.

  • Kahneman was strikingly humble despite his huge influence, always eager to hear what others thought
  • He openly wished failed-to-replicate work like priming wasn't in his book, instead of defending it
  • He pioneered 'adversarial collaboration' — co-writing papers with people who disagreed to find the study that would resolve the issue
  • Curiosity about being wrong correlates with finding genuinely interesting insights

“I wish priming wasn't in the book doesn't replicate”

Annie Duke · 05:30

“find someone who really disagrees with you and then write a paper with them”

Annie Duke · 06:00
#decision-making#kahneman#humility#open-mindedness
Story10:00

Two Pieces of Parenting Advice: Love Them, and You'll Drop the Baby

Asked what frameworks help in parenting, Duke gives two pieces of advice she offers expecting parents. First, the only thing that truly matters is that your kids deeply know you love them — everything else (breastfeeding, sleep training, schooling) pales in comparison. Second, at some point you will drop your baby on its head, and it will be fine — don't beat yourself up over mistakes.

  • The one thing that matters is kids deeply knowing you'd lay down in front of a bus for them
  • Debates over breastfeeding, co-sleeping, and schooling are dwarfed by that
  • Every parent will eventually drop the baby (often when it first learns to roll over) — kids are resilient
  • A mistake isn't a lasting scar; the real lesson is to take precautions once you learn a new risk

“the only thing that matters really is that your kids know you love them”

Annie Duke · 10:00

“at some point you will drop your baby on its head”

Annie Duke · 11:00
#parenting#advice#resilience
Story56:00

Inside First Round's Decision Rubric and Forecast Scorecards

Duke describes how she reshaped decision-making at First Round Capital over five years. Partners now rate market, team, founder, and product on a 1-7 scale with shared definitions ('mediating judgments') and explicitly forecast the probability a company funds at Series A. With hundreds of tracked outcomes, they can now score each partner's forecasting accuracy and even show which factors each partner is genuinely good at judging.

  • Ratings moved from good/bad to a 1-7 scale with shared definitions for precision and spread
  • 'Mediating judgments' define components (like competitive landscape) before judging the whole
  • Partners explicitly forecast the probability a company funds at Series A
  • Tracked outcomes let them feed back each partner's accuracy and refine the rubric from data, not just intuition

“let's make it explicit because you're doing it implicitly anyway”

Annie Duke · 58:30
#venture-capital#first-round#decision-rubric#forecasting
Story1:15:00

How Quitting Glitch Gave Stuart Butterfield Slack

Duke tells the story of Stuart Butterfield's game Glitch — critically loved, $6M in the bank, 5,000 diehard users, but a brutal customer-acquisition problem. After a six-week paid-marketing push, Butterfield woke up 'with the dead certainty' it was over, did back-of-envelope math showing it would never be venture-scale, and shut it down. Days later, the team's internal communication tool became Slack. The point: continuing Glitch would have cost him Slack.

  • Glitch had great PR and 5,000 20-hours-a-week users but ~95-99 bounced users for every diehard
  • After a strong six-week marketing test, Butterfield concluded it was not a venture-scale business
  • He shut it down for his equity-holding employees even though nobody else saw it coming
  • The hidden cost of continuing is the attention you can't give to other opportunities — quitting Glitch is what surfaced Slack

“I woke up this morning with the dead certainty the glitch was over”

Annie Duke · 1:16:30

“as smart as Stuart Butterfield is he couldn't see slack until he quit glitch”

Annie Duke · 1:19:00
#quitting#slack#startups#opportunity-cost

Takeaway· 2

Takeaway13:00

You Have Almost No Influence on Your Kid's Personality

Duke argues parents have essentially no influence over a child's personality — what you can do is make sure they say please and thank you. She illustrates 'resulting' (judging a decision by its outcome): parents of one well-behaved child feel smug about their parenting, but with four kids you learn temperament, not parenting, drove the result.

  • Parents have basically no influence over their child's personality
  • What you can control: teaching them to say please and thank you
  • Feeling like a great parent because your one child is well-behaved is 'resulting' — crediting the outcome to your decisions
  • Her second child's different high-school path revealed the first child's behavior was a temperament thing

“you have no influence whatsoever basically on what your child's personality is”

Annie Duke · 13:00
#parenting#resulting#decision-making
Takeaway1:02:30

The Thing Experts Pound the Table About Often Isn't Predictive

A surprising finding from the First Round data: the factors partners most vehemently insist matter are sometimes highly predictive across everyone — and sometimes not predictive even for the person pounding the table. Intuition is sometimes right and sometimes wrong; if you never make it explicit, you never get to find out which.

  • Experts express equal confidence in factors that turn out predictive and factors that don't
  • A table-pounded factor is sometimes predictive for everyone, sometimes for no one — including its champion
  • We wrongly assume brilliant people must have good intuition about what matters
  • Making intuition explicit is the only way to discover when it's wrong

“if you don't make it explicit then you don't get to find out when it's wrong”

Annie Duke · 1:03:30

“it's not that intuition is crap”

Annie Duke · 1:03:30
#intuition#decision-making#expertise#prediction