LLenny's Podcast
← All episodes
Shaun Clowes (CPO Confluent, ex-Salesforce, Atlassian)29 December 2024

Why great AI products are all about the data

10Frameworks
16Insights

Frameworks in this episode

Innovation3 steps

Context Over Models: The Data-Management Bet

AI products win on getting good, timely, well-structured data to the model — not on the model itself

Read →
Productivity3 steps

Data Is a Compass, Not a GPS

Data disproves the ridiculous — it rarely hands you the answer, so validate findings before you trust them

Read →
Sales3 steps

Dual-Motion Growth: PLG and Sales That Feed Each Other

Win by having both many customers and lots of revenue — with PLG and sales feeding each other

Read →
Innovation3 steps

LLM as Disconfirmation Engine

Point the model at where your strategy does NOT fit — and reverse-engineer competitors from their public docs

Read →
Strategy3 steps

Meet the Bar and Be Different

Incumbents overshoot average utility — win by meeting the bar while being materially different

Read →
Productivity3 steps

Outside-the-Building Product Management

Spend 80% of your time thinking outside the building and argue every case from the market's point of view

Read →
Strategy3 steps

Seek the Counterfactual, Not Confirmation

The competitive edge lives in the data you're trying to prove yourself wrong with, not the data you hoped to see

Read →
Strategy3 steps

The Business-Rules Moat: Why 'Forms on Databases' Are Unclonable

B2B SaaS lock-in isn't the UI or data model — it's years of accreted, configured business rules

Read →
Self-Mastery3 steps

The Career Bingo Card

Deliberately take adjacent-but-different roles to fill in squares and become 'scribble-shaped'

Read →
Productivity3 steps

The Nielsen Number: Right-Size Your Research

Interview 7-14 people — fewer teaches too little, more teaches nothing new

Read →

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Hot Take· 4

Hot Take05:30

Why Product Management Is Still Such a Random Discipline

Shaun Clowes argues that 15-20 years into the profession, product management is still strangely undeveloped — outcomes, behaviors, and individual performance all look random. He sees making PM reproducibly great as part of his life's work.

  • Product management is 15-20 years old but outcomes remain random
  • Individual PM performance is wildly distributed, unlike more mature disciplines
  • We are not reliably producing '10x' product managers
  • Something about the current state isn't getting at the truly value-added work

“the outcomes are random the behaviors are random individual performance is random”

Shaun Clowes · 06:00

“we could all say that we're not reliably producing you know 10 times product managers every day every day of the week”

Shaun Clowes · 07:30
#product-management#careers#hot-take
Hot Take20:00

LLMs Are Very Smart and Also Insanely Dumb

Clowes' core AI thesis: models are brilliant but only know what they were trained on or what you feed them in the moment — then they forget it. Combined with the decay rate of information, this means the real AI challenge for PMs is data management, not the model.

  • Models only know their training data or what you hand them in the moment
  • Information has a decay rate — new data loses value very quickly
  • An LLM is a 'synthesis machine' that is useless without fresh data to work on
  • Therefore AI's biggest impact on PM is data management, not the models

“these models are very very very smart but they're also like insanely dumb”

Shaun Clowes · 20:00

“you've got this synthesis machine which is this llm thing that's going to help you help you do synthesis but if it hasn't got all…”

Shaun Clowes · 20:30
#ai#llms#data#hot-take
Hot Take28:30

Why AI Won't Clone Salesforce Out of Business

Even if AI makes it trivial to build a forms-on-databases app, Clowes argues that drives the value of each new clone toward zero and strengthens incumbents. Even a headless, agent-only future still needs the business rules — and those rules are exactly what the incumbents own.

  • Easy cloning drives incremental value of each new app toward zero
  • No one gets fired for buying Salesforce — buyers default to the premier vendor
  • Even agent-only workflows must operate against the system's business rules
  • Most likely outcome: currently dominant companies get even more dominant

“imagine Salesforce had no UI it would still have those business rules that I was talking about and those business rules are what Define what…”

Shaun Clowes · 29:00

“the most likely outcome is that the is that the currently dominant companies are going to get more dominant”

Shaun Clowes · 30:30
#ai#b2b-saas#disruption#hot-take
Hot Take37:00

Data Is a Compass, Not a GPS

Having run data teams across many companies, Clowes now believes being 'data-driven' is easy to overdo. Data rarely gives you the answer — it tells you whether what you just said is ridiculous. It's for disproving hypotheses, not generating direction.

  • Treating data as the answer-giver makes you 'wrong or slow, sometimes both'
  • Data mostly disproves what you already think, rather than deciding for you
  • Your intuition already encodes a ton of data you've absorbed
  • Right-size data; don't use it as a weapon to force people your way

“data is more like a compass than a GPS right like if you look at data as a way of like giving you the answer…”

Shaun Clowes · 37:00
#data#decision-making#product-management#hot-take

Explainer· 5

Explainer07:00

Why a 10x PM Actually Delivers 100x Return

Because product management is fundamentally about leverage — helping other people have dramatically more impact — a great PM compounds. A 10x PM 10x's the return on 10x resources, producing 100x or more.

  • PM is about leverage: organizing goals so others have more impact
  • A 10x PM multiplies the return on already-amplified resources
  • That compounding is why the outcomes and benefits are so wildly distributed

“a 10 times product manager has 100 times return or more because because they're 10 timesing the return on 10 times resources”

Shaun Clowes · 07:00
#product-management#leverage#careers
Explainer15:30

Use LLMs to Prove Your Strategy Wrong, Not Right

Clowes' favorite AI move for PMs: paste customer interviews into ChatGPT and ask where your strategy does NOT fit what customers said. You can also feed in competitors' public documents to reverse-engineer their likely strategy, which he finds 'at times creepy' good.

  • Ask the LLM where your strategy does not fit customer input, not where it does
  • People over-index on finding what they hope to see
  • Feed competitor public docs into the LLM to summarize their probable strategy
  • Only works if you push at the edges and try to prove yourself wrong

“hey chat GPT this is my strategy tell me where my strategy does not fit what these customers talked about”

Shaun Clowes · 15:30

“people spend far too much time looking for what they're hoping to see not for what they're not looking to see”

Shaun Clowes · 15:30
#ai#llms#strategy#customer-research
Explainer17:30

Great PMs Swim in a 'Feedback River'

Borrowing a concept from ex-LinkedIn leader Sachin Rey, Clowes describes the best PMs as constantly swimming in a feedback river — deliberately surrounding themselves with user interviews, direct feedback, NPS, and competitor data. LLMs make maintaining that river far easier.

  • Concept credited to Sachin Rey (ex-LinkedIn): the 'feedback river'
  • Great PMs deliberately wash themselves in continuous information
  • At Confluent, LLMs cluster thousands of inbound requests semantically
  • Semantic grouping reveals which ideas are rising or falling in popularity

“they really smart product managers are constantly swimming in a feedback River”

Shaun Clowes · 17:30
#ai#customer-feedback#product-management#llms
Explainer21:30

Great AI Products Win on Context, Not the Model

If you're building an AI feature, the differentiator won't be the model (mostly replaceable) or even prompts — it's the context: getting good, timely, well-structured data to the LLM. Using a hypothetical Workday HR bot, Clowes shows how much linked data real 'smarts' require.

  • Models are mostly replaceable; prompts help but aren't the edge
  • The real answer is context — all the data you feed the model
  • A smart HR bot needs employee, benefits, legal, and policy data all linked
  • Getting good, timely, well-structured data to the LLM is where 90% of the work goes

“it's obvious that the real answer is the context like all the context you're going to give it all the data you're going to copy…”

Shaun Clowes · 21:30

“that's where 90% of the calories go”

Shaun Clowes · 23:30
#ai#product-development#data#context
Explainer25:30

B2B SaaS Is 'Forms on Databases' — and Business Rules Are the Moat

Uncharitably, Jira, Workday, and Salesforce are all just forms on databases — which sounds easy to clone but is unbelievably hard. The real lock-in isn't the UI or data model; it's the years of accumulated business rules and configuration that make the software uniquely yours.

  • All vertical/business SaaS is ultimately 'forms on databases'
  • Lock-in isn't the UI or data model — it's the business rules
  • Configuration makes the app native to your company and a black box to everyone else
  • Salesforce couldn't describe its own sales process without reading its Salesforce config

“the jira is a form on a database you know workday is a form on a database so sales force they all forms on databases”

Shaun Clowes · 25:30

“the real thing just staring everybody in the face is it's all about the business rules like that is what drives the lock in”

Shaun Clowes · 26:30
#b2b-saas#product-strategy#moats

Story· 2

Story56:30

Treat Your Career Like a Bingo Card

Clowes describes his deliberately varied career (Atlassian, Metromile, Salesforce, Confluent) as filling in a bingo card — chasing adjacent-but-different roles to become more versatile. The best people, he says, aren't 'T-shaped' but 'scribble-shaped': deep in many things at once.

  • He chose roles to fill in boxes he hadn't yet filled, building versatility
  • Each new domain becomes a pattern-match superpower for future problems
  • Pick adjacencies that stretch you — don't jump out of a plane never having parachuted
  • The best people are 'scribble-shaped,' deep in more than one thing, not just T-shaped

“my career's been a little bit like a bingo card like I've always been looking for to fill in boxes I didn't have filled”

Shaun Clowes · 57:00

“it's more like people are scribbl shaped”

Shaun Clowes · 1:04:00
#careers#professional-growth#story
Story1:08:00

Failure Corner: Killing the Product the Day a Customer Wanted to Pay

Clowes shares a product that should have obviously failed — an environmental-impact tool his company had no right to win in. After two years, the forcing function to kill it was, ironically, a customer finally wanting to pay millions. The lesson: call a spade a spade far earlier.

  • Right idea, but wrong company, wrong players, wrong time, wrong distribution
  • The product limped along in market for two years as a 'zombie'
  • They killed it the moment a customer was ready to pay millions — signing would trap them
  • Regret: it was obviously dead at six months; he should have said so and forced a decision

“wrong company wrong players wrong time wrong distribution like we had literally no right to win no right to play”

Shaun Clowes · 1:09:30

“the final straw was actually when a when a customer finally wanted to pay for it”

Shaun Clowes · 1:10:00
#failure#product-management#lessons#story

Takeaway· 5

Takeaway08:00

A PM's Job Is to Say No to 90% of Things

Clowes explains why PMs are the easiest role to criticize: the job is to say no to 90% of requests so you can say yes to the right 10%. That puts you behind the eightball from the start, so you have to quickly prove you have the right insights and data.

  • Saying no to 90% makes the PM the 'bad person' by default
  • You say no to 90% so you can say yes to the vital 10%
  • You have to get 'runs on the board' fast or you don't get another swing
  • This is why PMs are the easiest team member to single out and criticize

“your job is to say no to 90% of things that that get brought your away”

Shaun Clowes · 08:00

“so you're saying no to 90% so you can say yes to 10%”

Shaun Clowes · 08:00
#product-management#prioritization#careers
Takeaway09:30

Spend 80% of Your Time Thinking Outside the Building

The single biggest lever for PMs, per Clowes, is orienting outward. Most PMs get dragged into internal politics, scrum, and delivery — and you can never earn an A that way because the job is finding reliable, differentiated value in the market, not execution.

  • Echoing Steve Blank: spend 80% of your time thinking outside the building
  • Most PMs get pulled into internal politics, scrum, and delivery
  • You can't win on execution — the job is finding unique market value
  • Always argue from the customer, market, and competitor perspective, then back it with data

“you should spend 80% of your time thinking outside the building”

Shaun Clowes · 09:30
#product-management#customer-research#strategy
Takeaway32:00

In the AI Era, Distribution Gets Harder, Not Easier

Distribution has always been the hardest problem, and AI is making it worse — cold email and LinkedIn outreach are drowning in LLM-generated spam that desensitizes everyone. Being a cheaper clone won't cut through; you have to be materially different, e.g. by bringing data into the workflow.

  • Distribution channels are getting more crowded and expensive
  • Half of LinkedIn outreach now reads as LLM-generated spam, worsening signal-to-noise
  • 'Cheaper Salesforce' isn't a strategy — you must be materially better on some angle
  • Next-gen apps win by bringing data in as a first-class citizen (e.g. Ashby's ATS)

“in the world of AI it seems like distribution is more likely to get hard than easy”

Shaun Clowes · 32:00

“meet the bar and be different meet the bar and be different is is the way to cut through”

Shaun Clowes · 34:00
#distribution#go-to-market#ai#differentiation
Takeaway39:30

When Data Defies Intuition, Trust Your Intuition First

If a result contradicts your intuition strongly, the most likely explanation (Occam's razor) is that the analysis is wrong. Believe your intuition and go prove yourself right — and rigorously check whether the data is representative before presenting it, or you lose credibility.

  • A wildly counterintuitive result is usually just a broken analysis
  • Believe your intuition first, then try to prove yourself right
  • Occasionally the anomaly is real gold — but only diligence reveals which
  • Presenting a hole-filled analysis with authority costs you 'brownie points'

“first believe your intuition and go and prove yourself right”

Shaun Clowes · 39:30

“it would be better not to show up with an analysis that isn't clear then it would be to show up with an analysis that's…”

Shaun Clowes · 40:30
#data#analysis#decision-making#product-management
Takeaway1:17:00

People Don't Care What You Know Until They Know You Care

Clowes' guiding motto, learned after starting out as an 'engineer's engineer' obsessed with technical correctness: influence isn't about being right. It's built on trust, relationships, and caring about others' outcomes and incentives — everything good sits on that foundation.

  • He began his career prizing technical correctness and 'the one right answer'
  • Real influence isn't about being right — it's about trust and relationships
  • Care about others' outcomes and incentives first
  • Strong partnerships and progress are built on that foundation

“people don't care what you know until they know that you care”

Shaun Clowes · 1:17:30
#leadership#influence#careers#takeaway