LLenny's Podcast
← All episodes
Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter)10 April 2025

OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more

7Frameworks
15Insights

Episode overview

OpenAI CPO Kevin Weil explains how rapidly improving models force teams to plan lightly, deploy iteratively, and build products near the edge of current capabilities. He argues that custom evals, fine-tuning, model ensembles, and close collaboration among product, engineering, design, and research are becoming core product-building skills. The conversation also covers startup opportunities in specialized verticals, AI-assisted coding and creativity, personalized tutoring, and lessons from Facebook's unsuccessful Libra payments initiative.

Key ideas

  • AI product teams must expect the underlying model capabilities to change every few months, making agility more valuable than rigid long-range roadmaps.
  • Evals function like tests for models and help teams measure, fine-tune, and improve performance on specific product use cases.
  • Startups retain large opportunities in industry- and company-specific applications because foundation-model companies cannot own every vertical or access every private dataset.
  • OpenAI favors iterative deployment, empowered teams, lightweight planning, and products built near the frontier of model capability.
  • Fine-tuned models and ensembles of specialized model calls can outperform a single generic model call on complex workflows.
  • Product teams should increasingly prototype through AI-assisted coding, while machine-learning or research expertise becomes embedded across teams.
  • Chat remains a powerful AI interface because natural language supports broad, nuanced, open-ended communication.
  • Kevin emphasizes curiosity, independence, self-confidence, and clear thinking as durable human skills in an uncertain AI-shaped future.

Transcript available · source text is retained privately and is not published

Frameworks in this episode

People & resources mentioned

Attributed to the moment in the episode. Timestamps are approximate.

People · 11

  • Kevin WeilMentionsWe got connected through Kevin Wheel, who is former CPO at OpenAI, now head of science at OpenAI.

    Today my guest is Kevin Wheel. Kevin is chief product officer at Open AI

  • Lenny RachitskyMentionstoday we've got another very special compilation episode something I've been pulling on more and more with the podcast and the newsletter

    Kevin, thank you so much for being here and welcome to the podcast.

  • Elizabeth WeilMentionsElizabeth, my wife

    My wife Elizabeth sent me one of hers. So, I'm I'm right there with you.

  • Christina CacioppoMentionsI am very excited to have Christina Copo CEO and co-founder vanta joining me

    I'm excited to chat with Christina Gilbert, the founder of One Schema, one of our longtime podcast sponsors.

  • Dario AmodeiMentionsI was sitting next to Dario yesterday and he's like I keep making these predictions and people keep laughing at me

    Dario at Enthropic said that at the same time you guys are all hiring engineers like crazy

  • Julia VillagraMentionsour chief people officer Julia was telling me the other day she vibecoded an internal tool

    our chief people officer Julia was telling me the other day she vibecoded an internal tool

  • Peter ZeihanRecommendsThe Accidental Superpower by Peter Zion. Uh, very good if you're interested in geopolitics

    The Accidental Superpower by Peter Zion. Uh, very good if you're interested in geopolitics

  • Ethan MollickRecommendsCo-intelligence by Ethan Mullik. A really good book about AI and how to use it in your daily life

    Co-intelligence by Ethan Mullik. A really good book about AI and how to use it in your daily life

  • John MaloneMentionsthe biography of John Malone. Just fascinating if you like business

    the biography of John Malone. Just fascinating if you like business

  • Mark ZuckerbergCoinedit was sort of a random decision for Mark Zuckerberg to move from a uh monthly active to a daily active

    sometimes it's not any one thing. It's just good work consistently over a long period of time.

  • Alex KomoroskeMentionsAlex spent 13 years at Google where he worked on search double click he led Chrome's open web platform team for8 years

    I had a guest, Alex Kamaroski, come on the podcast. He's from Stripe and writes weekly reflections

Resources · 49

  • WhatsAppMentionssoftware · Meta

    It was integration into WhatsApp and Messenger. I would be able to send you 50 cents in WhatsApp for free.

  • MessengerMentionssoftware · Meta Platforms

    It was integration into WhatsApp and Messenger.

  • OpenAIMentionscompany

    Kevin is chief product officer at Open AI

  • TwitterMentionswebsite · Twitter

    He was previously head of product at Instagram and Twitter.

  • The Nature ConservancyMentionscompany

    He's also on the boards of Planet and Strava and the Black Product Managers Network and the Nature Conservancy.

  • Black Product Managers NetworkMentionscompany · Jules Walter and co-founders

    He's also on the boards of Planet and Strava and the Black Product Managers Network and the Nature Conservancy.

  • FacebookMentionscompany · Meta

    He was co-creator of the Libra cryptocurrency at Facebook which we chat about.

  • StravaMentionssoftware

    He's also on the boards of Planet and Strava and the Black Product Managers Network and the Nature Conservancy.

  • PlanetMentionscompany

    He's also on the boards of Planet and Strava and the Black Product Managers Network and the Nature Conservancy.

  • SuperhumanRecommendssoftware · Superhuman

    you get a year free of Perplexity Pro, Linear, Notion, Superhum, and Ranola.

  • NotionRecommendssoftware · Notion Labs

    you get a year free of Perplexity Pro, Linear, Notion, Superhum, and Ranola.

  • GranolaRecommendssoftware · Granola

    you get a year free of Perplexity Pro, Linear, Notion, Superhum, and Ranola.

  • LinearRecommendssoftware · Linear

    you get a year free of Perplexity Pro, Linear, Notion, Superhum, and Ranola.

  • Perplexity ProRecommendssoftware · Perplexity AI

    you get a year free of Perplexity Pro, Linear, Notion, Superhum, and Ranola.

  • Lenny's NewsletterCoinednewsletter · Lenny Rachitsky

    If you become an annual subscriber of my newsletter, you get a year free of Perplexity Pro, Linear, Notion, Superhum, and Ranola.

  • YouTubeRecommendswebsite · Google

    don't forget to subscribe and follow it in your favorite podcasting app or YouTube.

  • EppoRecommendssoftware · Eppo

    EPO is a next generation AB testing and feature management platform built by alums of Airbnb and Snowflake for modern growth teams.

  • PersonaRecommendscompany · Persona

    Persona, the adaptable identity platform that helps businesses fight fraud, meet compliance requirements, and build trust.

  • InstagramUsessoftware · Meta

    This was true, by the way, when we were building stories at Instagram.

  • Instagram StoriesMentionsproduct · Instagram

    when we were building stories at Instagram. More than anything else in my career, we could feel it was going to work

  • Deep ResearchMentionsproduct · OpenAI

    let's take our deep research product, which is one of my favorite things that we've released maybe ever.

  • OpenAI APIMentionssoftware · OpenAI

    We have 3 million developers using our API.

  • AnthropicMentionscompany

    kudos to Anthropic. They've built very good coding models.

  • GoogleMentionscompany · Google

    there are going to be different places where, you know, the Google models are really good

  • OperatorMentionssoftware · OpenAI

    more agentic tools like operator where it's browsing for you and doing things for you on the web

  • DeepSeekMentionscompany · DeepSeek

    And then DeepSeek launched and there were it was a lot

  • OneSchema FileFeedsMentionsproduct · OneSchema

    we just launched one schema file feeds, which allows you to build an integration with any system in 15 minutes

  • OneSchemaRecommendssoftware · Christina Gilbert

    if you want to learn more, head on over to one schema.c co that's one schema.co

  • ChatGPTUsessoftware · OpenAI

    every one of us is in chat GPT all the time summarizing docs using it to help write docs

  • FigmaMentionssoftware · Figma

    instead of showing stuff in like Figma, we should be showing prototypes that people are vibe coding

  • AlexaUsessoftware · Amazon

    they have full conversations with chat GPT and Alexa and everything else.

  • Khan AcademyMentionscompany · Sal Khan

    Khan Academy does great things. They're a wonderful partner of ours.

  • Star WarsMentionsfilm · George Lucas

    think of like Star Wars and you've got some scene where there's a there's a plane zooming into some Death Star-L like thing.

  • SoraUsessoftware · OpenAI

    now I can use Sora our video model and I can get 50 different variations of this cut scene

  • LibraMentionsproduct · Facebook

    Libra is probably the biggest disappointment of my career.

  • Mysten LabsMentionscompany

    Aptos and Miston are two companies that are built off of this tech.

  • AptosMentionscompany

    Aptos and Miston are two companies that are built off of this tech.

  • The Accidental SuperpowerRecommendsbook · Peter Zeihan

    The Accidental Superpower by Peter Zion. Uh, very good if you're interested in geopolitics

  • Co-IntelligenceRecommendsbook · Ethan Mollick

    Co-intelligence by Ethan Mullik. A really good book about AI and how to use it in your daily life

  • Cable CowboyRecommendsbook · Mark Robichaux

    I really enjoyed Cable Cowboy. I don't know who the author is, but uh the biography of John Malone.

  • The Wheel of TimeMentionsbook · Robert Jordan

    when I was a kid, I read the Wheel of Time series.

  • The Wheel of TimeMentionsbook · Robert Jordan

    now Amazon has it uh as they're in like the third season of it. So, I I want to watch that.

  • Top Gun: MaverickRecommendsfilm · Joseph Kosinski

    Top Gun 2 was an awesome movie.

  • WaymoRecommendscompany · Alphabet

    Maybe the other one is Whimo. Uh every chance I get, I'll take a Whimo. It's just a better way of riding

  • WindsurfRecommendssoftware · Codeium

    I think like vibe coding with with products like Windsorf is just super fun. Um I'm I'm having a great time doing that.

  • StripeMentionscompany · Patrick Collison and John Collison

    He's from Stripe and writes weekly reflections on what's happening in the world.

  • Apple PodcastsRecommendssoftware · Apple Inc.

    you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.

  • SpotifyRecommendssoftware · Spotify

    you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.

  • Lenny's PodcastCoinedpodcast · Lenny Rachitsky

    You can find all past episodes or learn more about the show at lennispodcast.com.

Spot an error or want something removed? Request a correction or removal.

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Hot Take· 4

Hot Take00:00

Today's AI Is the Worst You'll Ever Use

Weil's core mental model for reasoning about AI: whatever model you touch today is the least capable version you will ever experience, because a materially better one arrives every couple of months. Internalizing this changes how you build — you should design for capabilities that are 'almost there' rather than today's limits.

  • Every two months computers can do something they've never done before
  • Unlike traditional software, the technology you build on is not fixed — it keeps improving underneath you
  • The right posture is 'model maximalism' — build for where the puck is going, not where it is

The AI models that you're using today is the worst AI model you will ever use for the rest of your life.

Kevin Weil · 00:00

Every 2 months, computers can do something they've never been able to do before, and you need to completely think differently about what you're doing.

Kevin Weil · 00:00
#ai#product#mindset#openai
Hot Take31:00

Model Maximalism: Build on the Edge of What's Possible

Weil describes OpenAI's 'model maximalism' philosophy — don't over-invest in scaffolding around a model's current weaknesses, because in two months a better model will erase them. His advice to developers: if your product barely works because it's at the edge of model capabilities, keep going, because it will soon 'sing.'

  • Avoid heavy scaffolding around model limitations that the next model will fix anyway
  • Products sitting right at the edge of capability are a sign you're doing something right
  • A couple months of model improvement can turn a barely-working product into a great one

our general mindset is in two months there's going to be a better model and it's going to blow away whatever you know the current…

Kevin Weil · 31:30

keep going because you're doing something right because you give it another couple months and the models are going to be great and suddenly the…

Kevin Weil · 32:00
#ai#product#philosophy#openai
Hot Take41:00

Why Chat Is Actually the Ideal AI Interface

Against the common view that chat is a placeholder for something better, Weil argues chat is an amazing, uniquely versatile interface because it mirrors how humans naturally communicate — unstructured, high-bandwidth language. It only works now because LLMs can finally handle the nuance of human speech, so chat is precisely fit to their power.

  • Chat is versatile because it's the low-restriction way humans already communicate
  • More rigid interfaces would cap the range of what you could express to a model
  • Chat never worked before because no model understood the complexity of human speech; more prescribed interfaces still make sense for high-volume, narrow tasks

I actually think chat is an amazing interface because it's so versatile.

Kevin Weil · 41:30

So to me it's like an interface that's exactly fit to the power of these things.

Kevin Weil · 42:30
#interface#chat#llm#product
Hot Take1:06:00

Why a Billion-Kid AI Tutor Should Already Exist

Weil says personalized tutoring may be one of the most important things AI can do, and he's genuinely surprised no one has built a two-billion-kid AI tutoring product yet. The models are good enough now, ChatGPT is free, Android devices are everywhere, and research consistently shows combining classroom learning with personalized tutoring yields multiple-standard-deviation gains.

  • Combining classrooms with personalized tutoring produces multiple standard deviation improvements in learning speed
  • Models are already good enough and ChatGPT is free and globally accessible
  • Weil is surprised no massive personalized-tutoring product exists yet and wants it to

when you combine that with personalized tutoring, you get like multiple standard deviation improvements in learning

Kevin Weil · 1:07:00

the models are good enough. Like it still just kind of blows my mind that there isn't something amazing out there

Kevin Weil · 1:07:30
#education#tutoring#ai#impact

Explainer· 4

Explainer17:30

Model Accuracy Dictates What Product You Can Build

Weil explains that LLMs break the classic software contract of defined inputs producing defined outputs — they take fuzzy inputs and give fuzzy, non-deterministic outputs. Because of that, the model's success rate on a use case fundamentally changes the product you should build around it.

  • Traditional computers give the same output every time; LLMs give 'spiritually the same answer' but not the same words
  • A use case the model gets right 60% of the time needs a totally different product than one it gets right 99.5% of the time
  • You have to get into the weeds on your specific use case and its evals to know what to build

If the model gets it right 60% of the time, you build a very different product than if the model gets it right 95% of…

Kevin Weil · 18:00
#ai#product#llm#evals
Explainer18:30

Writing Evals Is Becoming a Core Product Skill

Weil explains what an eval is — essentially a quiz or unit test that measures how good a model is at a specific task — and why writing them is quickly becoming a must-have skill for product managers. Evals aren't just static measurement; you design them alongside the product and 'hill climb' on them to make the model better at your hero use cases.

  • An eval is like a quiz or unit test that gauges a model's capability on a set of tasks
  • OpenAI designed deep-research evals in parallel with the product, then fine-tuned the model against them
  • Evals turn model quality into a continuous, measurable learning process rather than a static hope

I I think the easiest way to think about it is almost like a a quiz for a model

Kevin Weil · 19:00
#evals#product-management#ai#skills
Explainer55:30

What Vibe Coding Actually Means

Weil explains vibe coding (a term he credits to Andrej Karpathy): with tools like Cursor, Windsurf, and Copilot getting better, you increasingly 'let go of the wheel' — accepting the model's suggestions, pasting errors back in, and telling it to keep going. It's not yet for tight production code, but it's ideal for proofs of concept and demos.

  • Vibe coding = accepting model suggestions rapidly instead of hand-editing every step
  • When code doesn't compile you paste the error back and say 'go go go'
  • Not for production-tight code yet, but great for demos and proofs of concept — teams should be doing it more instead of Figma mockups

as the models are getting better, and as people are getting more used to it, you can kind of just like uh let go of…

Kevin Weil · 56:00

And it's not that you would do that for production code that needed to be super uh tight today yet, but for so many things.

Kevin Weil · 56:30
#vibe-coding#coding#tools#ai
Explainer1:00:00

Break Problems Down and Ensemble Specialized Models

Weil reveals OpenAI uses ensembles of models far more than people realize — solving 10 problems might mean 20 model calls across differently-sized and fine-tuned models with custom prompts, chosen by latency and cost needs. His advice: break a problem into specific tasks and use specialized models for each, rather than throwing a single generic prompt at one big model.

  • OpenAI solves problems with many specialized model calls, not one generic model
  • Model choice per sub-task is driven by latency and cost (e.g. 4o-mini for fast cheap checks)
  • Weil's analogy: a company is itself an ensemble of fine-tuned 'models' — people with different skills combined for better output

you want to break the problem down into more specific tasks versus some broader set of highle tasks and then you can use models very…

Kevin Weil · 1:00:30

in general it's like specific models for specific purposes and then you you you ensemble them together to solve problems

Kevin Weil · 1:02:30
#ai#architecture#fine-tuning#ensembles

Story· 3

Story06:00

When Internal Usage Explodes, You Know It'll Work

Weil describes a repeated career signal for a winning product: when the team can't stop using it internally. It happened with Instagram Stories and again with the new image model, where an internal gallery generated non-stop buzz. Especially for social products, if it isn't taking off inside the company, question what you're doing.

  • The Instagram Stories team felt it would work because they used it themselves over weekends
  • OpenAI's image model had 'non-stop buzz' in an internal generation gallery before launch
  • For social products, weak internal adoption is a warning sign

More than anything else in my career, we could feel it was going to work because we were all using it internally

Kevin Weil · 06:00
#product#launch#signal#instagram
Story1:18:00

Libra: The Biggest Disappointment of Weil's Career

Weil calls the Facebook cryptocurrency project Libra the biggest disappointment of his career, because it solved very real problems — regressive remittance fees where people pay ~20% and wait days to send money home. In hindsight he'd have introduced the change more gently rather than launching a new blockchain, a currency basket, and WhatsApp/Messenger integration all at once during Facebook's reputational low point.

  • Remittances are regressive — people pay outrageous fees (~20%) and wait days to send money home
  • Libra tried too much at once: new blockchain, basket of currencies, and messaging integration
  • Bad timing amid Facebook's reputation low; Weil owns the decisions but wishes it existed today

Libra is probably the biggest disappointment of my career.

Kevin Weil · 1:18:30

Why can't you send money as immediately, as cheaply, as simply as you send a text message?

Kevin Weil · 1:19:00
#libra#crypto#facebook#remittances
Story1:24:30

The Life Motto: Good Work, Consistently, Over a Long Time

Weil's guiding philosophy came from a Mark Zuckerberg answer on a Facebook earnings call — asked what one thing drove huge growth, Zuckerberg said sometimes it isn't any one thing, just good work consistently over a long period of time. Weil made it into a poster; people look for silver bullets, but excellence is showing up daily and letting the compounding gains add up.

  • People too often hunt for a silver bullet instead of consistent daily work
  • Small daily improvements are unnoticeable week to week but compound massively over years
  • Weil, an ultramarathon runner, frames excellence as grinding and getting a little better every day

sometimes it's not any one thing. It's just good work consistently over a long period of time.

Kevin Weil · 1:25:00

a lot of like excellence is actually showing up day in and day out doing good work getting a little bit better every single day

Kevin Weil · 1:25:30
#philosophy#motto#consistency#career

Tool· 1

Tool1:27:00

Two Prompting Tricks: Examples and Personas

Weil first wants to kill the idea that you must be a good prompt engineer — over time models should make that unnecessary. But today, two techniques work: include examples of question-and-good-answer pairs in your prompt ('poor man's fine-tuning'), and assign the model a persona ('you are the world's greatest brand marketer') to shift it into a useful mindset.

  • Including example question/good-answer pairs in the prompt acts as 'poor man's fine-tuning'
  • People don't provide in-prompt examples nearly often enough
  • Persona framing ('you are Einstein', 'the world's greatest marketer') shifts the model into a more positive mindset — a human analog to how we frame people

you can do like, you know, effectively poor man's fine-tuning by including examples in your prompt

Kevin Weil · 1:27:30

You can also say things like I want you to be Einstein. Now answer this physics problem for me. Or you are the world's greatest…

Kevin Weil · 1:28:30
#prompting#tips#llm#tools

Takeaway· 3

Takeaway24:30

Where Startups Can Win That OpenAI Won't Go

For founders worried OpenAI will crush them, Weil argues the opposite: most of the world's valuable data and processes live behind company and industry walls, and OpenAI can't and doesn't want to build for every vertical. Their focus on a great API (3M+ developers) is a bet that most AI value will be built by others.

  • There are far more smart people outside your company's walls than inside them
  • Most valuable data is industry- or company-specific and not in the training set
  • OpenAI intentionally doesn't want to build every vertical — immense opportunity exists for startups to own them

there are way more smart people outside your walls than there are inside your walls.

Kevin Weil · 25:00

And there are immense opportunities in every industry and every vertical in the world to go build AI based products that improve upon the the…

Kevin Weil · 26:00
#startups#moats#api#strategy
Takeaway36:00

The Counterintuitive Trick: Reason About AI Like a Human

Weil's most surprising lesson from building AI products: you can often figure out how an AI feature should work — or why an AI behavior occurs — by reasoning about what an equivalent human would do. It shaped the reasoning-model UI (little status updates instead of silence or a firehose of thoughts) and the use of model ensembles that mimic group brainstorming.

  • The reasoning model's 'thinking' UI was designed by asking how a human would behave while pausing to think
  • Ensembles of models attacking one problem then integrating outputs mirror human brainstorming
  • Treating the model like a person gives useful intuition for designing AI experiences

there's just like all these situations where you can actually kind of reason about it like a group of humans or an individual human and…

Kevin Weil · 39:00
#ai#product-design#reasoning#ux
Takeaway48:00

Stay PM-Light and Hire for High Agency

OpenAI runs with surprisingly few PMs (~25) on purpose — Weil believes too many PMs fill the world with decks instead of execution and tempt micromanagement. He'd rather have PMs stretched across too many product-focused engineers, and he hires for high agency and comfort with extreme ambiguity, which makes it a hard fit for junior PMs.

  • OpenAI has roughly 25 PMs and deliberately keeps the ratio PM-light
  • Too many PMs produces decks and micromanagement rather than shipping
  • Hiring bar is high agency, comfort with ambiguity, and leading through influence — hard for early-career PMs who want defined scope

my personal belief is that you want to be pretty PM light as an organization just in general.

Kevin Weil · 48:00

high agency, very comfortable with ambiguity, ready to come in and help execute and move really quickly.

Kevin Weil · 51:00
#product-management#hiring#org-design#openai