LLenny's Podcast
← All episodes
Aparna Chennapragada18 May 2025

Microsoft CPO: If you aren’t prototyping with AI, you’re doing it wrong

6Frameworks
16Insights

Frameworks in this episode

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Myth Buster· 1

Myth Buster39:30

The Danger of Metrics in Zero-to-One Products

Aparna warns against grabbing grown-up metrics too early on a new product — it's false precision. CTR with a thousand users means nothing, and retention may not either. In the solve stage you should look for qualitative signals, like the sound of a click or whether a voice assistant nails one or two things (setting a timer, playing music) before claiming it can do anything.

  • Deciding on a metric too early is false precision
  • CTR at a thousand users, and even retention, can be meaningless early on
  • Look for qualitative signals in the solve stage instead of grown-up metrics
  • Voice assistants succeeded by nailing a few narrow things (timer, music) before promising anything

if you decide on a metric too prematurely that's false precision

Aparna Chennapragada · 40:00

CTR when you have like thousand people doesn't mean anything

Aparna Chennapragada · 40:00
#metrics#zero-to-one#product-strategy#measurement

Hot Take· 5

Hot Take18:00

NLX Is the New UX: Conversations Have Grammars Too

Aparna's contrarian design argument: 'the model eats the product' is wrong — natural language interfaces are still designed, just elastic rather than rigid like GUIs. Conversations have grammars, structures, and invisible UI elements, and new constructs are emerging: the prompt itself, editable plans, and 'showing the work' progress.

  • GUIs are rigid and explicitly designed; natural language is elastic but still designed
  • The prompt is a new UI construct, just like a dropdown or menu once was
  • Editable plans from an agent are a new construct for high-level goals
  • 'Showing the work' must be calibrated — too verbose feels like a cron job, too terse loses confidence

NLX is the new UX

Aparna Chennapragada · 18:00

prompt itself is a is a new construct

Aparna Chennapragada · 19:00
#nlx#ux#design#conversational-interfaces
Hot Take22:30

Prompt Sets Are the New PRDs — Demos Before Memos

Aparna insists that if you're not prototyping to see what you want to build, you're doing it wrong. Prototypes and prompt sets replace written PRDs as the fastest, highest-bandwidth way to communicate an idea. The catch: time-to-first-demo shrinks while time-to-full-deployment grows, so the bar for breaking out above the noise rises.

  • Prompt sets and prototypes are the new PRDs — the fastest path to experiencing what's in your mind
  • Prototyping acts as a loop accelerator for product building
  • Inner-loop prototyping shortens, but the bar for scaled deployment gets much higher
  • A massive increase in the supply of ideas means you must avoid chasing every idea

if you're not prototyping and building to see what you want to build I think you're doing it wrong

Aparna Chennapragada · 23:00

demos before memos

Aparna Chennapragada · 24:00
#prototyping#prds#product-development#ai-tools
Hot Take26:00

Coding Isn't Dead — We'll Just Have 'SOs' Instead of SWEs

Aparna strongly disagrees with the 'coding is dead, don't study CS' narrative. Programming has always added higher layers of abstraction — from assembly to C and beyond — and AI is just the next layer. It democratizes building, producing an order of magnitude more software operators, but understanding computer science as a way of thinking still matters.

  • AI is another layer of abstraction, like moving off assembly and C
  • You'll tell the computer what to do at a higher level, democratizing software
  • Expect an order of magnitude more 'software operators' (SOs) than SWEs
  • Computer science remains a valuable mental model and way of thinking

I strongly disagree with the whole like coding is dead

Aparna Chennapragada · 26:30

instead of SWES maybe we'll have souls

Aparna Chennapragada · 26:30
#coding#computer-science#abstraction#ai
Hot Take28:00

Why the PM Role Gets More Important, Not Dead

Aparna argues the opposite of 'PMs are dead': when coding is easy and ideas become abundant, the taste-making and editing function matters more. Process-only 'TPS report' PMs are at risk, but the bar rises for real product judgment — and gatekeeping by title disappears, replaced by unlocking latent good ideas from engineers, researchers, and designers.

  • Process- and project-management-only PMs face a real 'what's the value-add' question
  • The taste-making and editing function becomes far more important as prototype supply explodes
  • Editing authority must now be earned, not granted by title
  • AI unlocks latent good ideas from smart engineers, researchers, and designers

you have to earn it now you just don't get it because of this title

Aparna Chennapragada · 29:30
#product-management#taste#editing#ai
Hot Take45:00

Why Microsoft 'Lost' AI Coding — and Why It's Fine

Asked what happened when Cursor and others outran GitHub Copilot despite Microsoft's early lead, Aparna reframes it: code generation unlocked a huge space of new products, not a single winner-take-all market. GitHub is a system, not just a product — repo, context, autocomplete, chat, and agent mode — that scales from expert coders to vibe coders. All roads lead to GitHub.

  • Code generation is a tool that unlocks many products, not one zero-sum market
  • GitHub is a system built around your repo and context, not a single feature
  • It scales across expert coders and casual coders with the right assistance
  • Everyone ends up in GitHub regardless of which tool they build with

so it's a system not just a product or a set of features

Aparna Chennapragada · 46:30

All roads uh lead to GitHub.

48:00
#github#ai-coding#competition#copilot

Explainer· 4

Explainer07:30

What's Actually Different About Building for Enterprise

Moving from consumer companies to Microsoft, Aparna found that enterprise product work always carries a second use case alongside the first: the feature must work AND be governed. The failure modes are disregarding governance entirely or over-crippling the user experience by leaning too hard on it.

  • In consumer you have a playbook to make a feature work and delightful; enterprise adds governance every time
  • Even sharing a document link must be frictionless yet secure, auditable, and safe
  • The trap is either ignoring the governance side or crippling the UX by overweighting it

in the enterprise, you almost have every time you you think you have one use case, you really do, which is how do you make…

Aparna Chennapragada · 08:00
#enterprise#governance#product-management#ux
Explainer16:00

The Three Things That Actually Define an AI Agent

Cutting through the hype, Aparna frames agents as tools built on stochastic models and answers 'what is an agent' with three product principles: autonomy (a spectrum of delegation), complexity (multi-step, not one-shot), and natural asynchronous interaction. She notes the magic isn't time saved but new insights that fire synapses she didn't have.

  • Autonomy is a spectrum of how much you can delegate, not a zero-or-one switch
  • Complexity means going beyond one-shot tasks to 'build me this prototype' or 'help me nail this meeting'
  • Agents interact naturally and asynchronously — they work when you're not working
  • The real value is new insights and 'superpowers', not just summarizing or saving time

So, when I think about agents, I think about three things.

Aparna Chennapragada · 16:00

It works when you're not working, right?

Aparna Chennapragada · 17:00
#agents#ai#autonomy#product-principles
Explainer41:30

The 'Why Now' Test: Two of Three Inflection Points

For a great zero-to-one product, Aparna looks for at least two of three inflection points: a step-function shift in technology, a shift in consumer behavior, and a shift in the business model. She illustrates each with Google Lens (deep learning + free storage making everyone photograph everything) and Robin Hood (mobile finance + zero-fee monetization).

  • Tech inflection: a step function like deep learning, speech recognition, or today's reasoning models
  • Consumer-behavior inflection: e.g. free storage turning cameras into a keyboard for the real world
  • Business-model inflection: second-price auctions, SaaS, and now usage or outcome-based AI monetization
  • You want at least two of the three to line up for a strong product

you do want to look for at least two out of these three factors uh inflection points here

Aparna Chennapragada · 42:00
#why-now#inflection-points#zero-to-one#product-strategy
Explainer48:30

Why 5,000 Startups Can't Kill Excel

Aparna relays what an OG Excel product person told her: Excel is proof that non-coders also program, giving powerful programming ability to people who don't write code. She points to world Excel championships as evidence of its depth, and draws a broader lesson — some tools have upfront learning friction precisely because they're powerful and deep to use.

  • Excel proves non-coders also program — it hands powerful programming ability to non-coders
  • World Excel championships showcase the tool's surprising depth
  • Some tools trade a harder initial learning curve for long-term power and depth
  • Decades of attention to deep user signal compound into a lasting moat

Excel is a proof that non-coders also have to program

Aparna Chennapragada · 49:00

some tools are harder to learn perhaps in the beginning there's friction in terms of learning but great to use

Aparna Chennapragada · 49:30
#excel#no-code#product-depth#microsoft

Story· 4

Story04:30

How Standup Comedy Makes Her a Better Product Builder

Aparna does open-mic standup semi-seriously and sees a direct parallel to product building: both are tight, live iteration loops where you ship an unpolished first version, get brutally clear micro-feedback, and build resilience. She ties it to Reid Hoffman's rule that if your first version doesn't embarrass you, you shipped too slow.

  • Open mics are real live experiments that give clear, tough micro-feedback from users
  • Product builders need the resilience to launch a strong vision whose first version isn't quite there
  • Both comedy and products need PMF: product-market fit and punchline-market fit

if you don't launch the first version and are not embarrassed you're doing it too slow

Aparna Chennapragada · 06:00
#product-building#iteration#resilience#comedy
Story10:30

The 'Frontier' Program: Institutionalizing Living One Year in the Future

Aparna describes a Microsoft effort to operationalize her personal habit of living one year in the future. They spun up a small internal team — and even a fake external company — where early adopters get cutting-edge experimental agents in their hands, without forcing the whole company to change at once.

  • The mistake to avoid is holding back early adopters while the rest of the org catches up
  • Frontier rolls out experimental features to willing users in a trusted way, alongside slower change management
  • The goal is to imagine a frontier product, team, and way of working, not just frontier models

I want to kind of institutionalize and operationalize my personal model of like living one year in the future

Aparna Chennapragada · 11:00
#ai-adoption#early-adopters#microsoft#change-management
Story35:30

Working Closely With Both Satya and Sundar

Having served as technical advisor to Sundar Pichai at Google and now working with Satya Nadella at Microsoft, Aparna contrasts the two. Sundar was a master at staying calm, measured, and thoughtful across complex ecosystems like phones, search, and publishers. Satya stands out for his appetite to learn, fine-tune his mental models, and fluidly zoom between macro strategy and micro insight.

  • Both are 99.99th percentile across intellect, empathy, leadership, and product strategy
  • Sundar excelled at calm, thoughtful navigation of complex ecosystems
  • Satya has a striking appetite for learning and updating his mental models
  • Satya moves fluidly between macro strategy and specific micro insights, often ahead of others

really calm and measured and thoughtful

Aparna Chennapragada · 36:30

the appetite he has for learning and fine-tuning his mental models

Aparna Chennapragada · 37:00
#leadership#satya-nadella#sundar-pichai#careers
Story50:30

Google Now and the Lesson That Being Early Is Being Wrong

Aparna's pivotal career moment came after a year failing to make search personalization work, which led her to build Google Now — a product that pushed proactive, contextual content on the phone. Though the product eventually went away, it taught her that she loves seeing around corners, that being early is the same as being wrong, and that talent density is everything.

  • A failed year on search personalization led to the pivot to Google Now
  • Google Now became a foundation for Google Assistant and later paid off with Gemini's LLM step function
  • The interface was great but the intelligence wasn't there yet — being early equals being wrong
  • A small, dense group of very smart people can produce outsized results

being early is the same as being wrong

Aparna Chennapragada · 52:30

the interface was great the intelligence wasn't there

Aparna Chennapragada · 54:00
#google-now#careers#timing#failure

Takeaway· 2

Takeaway31:30

Update Your Priors: There's Alpha in Demanding More of AI

The hardest and most valuable habit, Aparna says, is updating your priors on what AI can do — even for people living in the space. Capabilities that failed months ago (image spelling, reasoning, data analysis) suddenly work, so scar tissue from past attempts becomes a liability. Ignoring what you learned and demanding more of today's models is a real arbitrage.

  • Reflexive AI usage is hard because updating your priors is hard
  • Models that couldn't do something a year ago can now — the baby grew up fast
  • Scar tissue from old failed attempts causes builders to under-ask
  • Cutting against the grain and demanding more of AI today unlocks an edge

the baby just grew up to be a 15year-old in a month

Aparna Chennapragada · 32:30

There's a lot of alpha in in doing that.

Aparna Chennapragada · 33:00
#ai-adoption#mental-models#arbitrage#priors
Takeaway37:30

Solve Before Scale: Get Comfortable With the Chaos

Aparna's most repeated counterintuitive lesson is 'solve before scale' — resist the temptation to rush to scale a zero-to-one product. The solve stage demands wide lurches (Google Lens started as plant detection and pivoted to translation), and you should have an appetite for that chaos rather than prematurely fixing on one local hill you later can't climb off.

  • Solving and scaling are different postures and modes, not one continuous motion
  • Solve mode requires wide, seemingly chaotic pivots — Lens went from plant detection to translation
  • Prematurely locking onto one local hill traps companies for years
  • You should have an appetite for the chaos of the solve stage, not just tolerate it

So I've always said uh to my teams solve before scale

Aparna Chennapragada · 38:00

the last thing you want is prematurely like you know fix on one local hill

Aparna Chennapragada · 39:00
#zero-to-one#product-strategy#scaling#google-lens