LLenny's Podcast
← All episodes
Marily Nika (Meta, Google)05 February 2023

AI and product management

8Frameworks
13Insights

Episode overview

Marily Nika argues that product managers should avoid the “shiny object trap” and begin with a genuine user problem before choosing AI. She explains how PMs can use ChatGPT in daily work, collaborate with research scientists, assess data and model quality, and manage the uncertainty of AI product development. The conversation also covers practical learning resources, no-code model training, and her three-week AI product-management course.

Key ideas

  • AI should solve a validated user pain point, not be added merely because it is fashionable.
  • Product managers can use ChatGPT to refine mission statements, explore user segments, and generate ideas after forming their own initial judgment.
  • Future product teams will increasingly require PMs to collaborate comfortably with research scientists amid uncertain outcomes and longer research cycles.
  • AI is usually inappropriate for an MVP; teams can prototype the intended experience first and invest in models after validating demand.
  • Useful AI products require appropriate data, explicit quality thresholds, and a decision about whether proprietary training creates meaningful differentiation.
  • AI PMs often manage the problem as much as the product, bridging research with viable, desirable, monetizable user applications.
  • Hands-on learning—including coding fundamentals, shadowing researchers, and building an end-to-end AI product—helps PMs develop confidence and judgment.

Transcript available · source text is retained privately and is not published

Frameworks in this episode

Productivity4 steps

Draft-First LLM Augmentation

Never ask the model to do your job — write your version first, then have it improve it.

Read →
Innovation4 steps

Fake the AI Before You Build It

Never train a model for an MVP — prototype the AI's output and test demand first.

Read →
Influence4 steps

The Adjacent-Precedent De-Risk Pitch

Win leadership buy-in for a big AI bet by anchoring it to a past bet that already worked.

Read →
Self-Mastery5 steps

The AI PM Upskilling Path

Learn the fundamentals, shadow a research scientist an hour a week, and build one model end to end.

Read →
Leadership4 steps

The Four Challenges of AI Product Management

Uncertainty, pivots, data scarcity and a broken promo path — the four taxes of the AI PM role.

Read →
Strategy4 steps

The Model Launch Bar

With probabilistic products, the PM — not the scientist — decides what accuracy is good enough to ship.

Read →
Strategy4 steps

The Shiny Object Trap (Problem-First AI)

A regular PM ships the right product; an AI PM solves the right problem.

Read →
Entrepreneurship6 steps

Treat Your Course Like a Product

Hypothesise the audience, interview them, iterate the ICP, and run three weeks — not one.

Read →

People & resources mentioned

Attributed to the moment in the episode. Timestamps are approximate.

People · 6

  • Marily NikaMentionsmarily teaches the most popular course on Maven on AI and product management she's currently product lead at meta

    “today my guest is marily NAA”

    ▶ Watch at ~0:30approx.
  • Lenny RachitskyMentionstoday we've got another very special compilation episode something I've been pulling on more and more with the podcast and the newsletter

    “marily welcome to the podcast”

    ▶ Watch at ~3:00approx.
  • Tyler CowenMentionshe has this awesome blog marginal Revolution and he's really good at sharing insights from research papers

    “when I think that I always think of Tyler Cohen he has this awesome blog marginal Revolution”

    ▶ Watch at ~35:00approx.
  • Steve JobsMentionsSteve Jobs is just uh the bar he held for the company and for technical talent and for excellence was not uh wavering.

    “I love what Steve Job said where he he used to say well users don't know what they want until you show it to them”

    ▶ Watch at ~36:00approx.
  • Marty CaganCoinedMarty has been helping product teams and product managers improve their craft processes and careers for over 20 years

    “inspired it taught me it's all about how to create products people love by Marty Kagan”

    ▶ Watch at ~44:00approx.
  • Andrew BosworthMentionsIdentified as Meta's CTO and formerly Facebook's head of ads; he encouraged Deb Liu to publish monthly.

    “I like boses podcast I don't know if you're aware of it BOS is the CEO of Facebook he has a great podcast”

    ▶ Watch at ~45:00approx.

Resources · 46

  • Lenny's PodcastCoinedpodcast · Lenny Rachitsky

    “welcome to Lenny's podcast where I interview world-class product leaders and growth experts”

    ▶ Watch at ~0:00approx.
  • MetaMentionscompany

    “she's currently product lead at meta focusing on metaverse avatars and identity”

    ▶ Watch at ~0:30approx.
  • MavenUseswebsite · Maven

    “marily teaches the most popular course on Maven on AI and product management”

    ▶ Watch at ~0:30approx.
  • GoogleMentionscompany · Google

    “prior to meta she was at Google for over eight years”

    ▶ Watch at ~0:30approx.
  • AmplitudeMentionssoftware · Amplitude

    “this episode is brought to you by amplitude”

    ▶ Watch at ~1:00approx.
  • EppoMentionssoftware · Eppo

    “EPO is a Next Generation AB testing platform”

    ▶ Watch at ~1:30approx.
  • TLDRRecommendsnewsletter

    “I am a big big big fan of the download by NP technology review or tldr”

    ▶ Watch at ~4:00approx.
  • The DownloadRecommendsnewsletter · MIT Technology Review

    “I am a big big big fan of the download by NP technology review or tldr”

    ▶ Watch at ~4:00approx.
  • ChatGPTUsessoftware · OpenAI

    “obviously CH is amazing I'm using it day to okay”

    ▶ Watch at ~5:00approx.
  • ChatGPTUsessoftware · OpenAI

    “I literally goild the CH and I say rewrite this mission statement for me”

    ▶ Watch at ~6:00approx.
  • Stranger ThingsMentionstv show · The Duffer Brothers

    “after you you watch white L like stranger things you will want something similar to watch”

    ▶ Watch at ~8:30approx.
  • NetflixMentionscompany · Netflix

    “you cannot watch to Netflix you cannot even watch a movie without needing that”

    ▶ Watch at ~8:30approx.
  • FigmaRecommendssoftware · Figma

    “fake it create a little figma product type and just show it some users”

    ▶ Watch at ~15:00approx.
  • MidjourneyMentionssoftware · Midjourney, Inc.

    “use something that's already out there like say gbt or mid Journey or all those guys”

    ▶ Watch at ~17:00approx.
  • Google I/OMentionsevent · Google

    “they had a video on last year's Google IO”

    ▶ Watch at ~21:30approx.
  • Google GlassMentionsproduct · Google

    “the glass would take as an input the audio that came from that other person and it would transcribe it”

    ▶ Watch at ~22:00approx.
  • GPT-6Mentionssoftware · OpenAI

    “do you think chat GPT or just say GPT 4 or J GPT 5 GPT 6”

    ▶ Watch at ~22:30approx.
  • GPT-5Mentionssoftware · OpenAI

    “do you think chat GPT or just say GPT 4 or J GPT 5 GPT 6”

    ▶ Watch at ~22:30approx.
  • GPT-4Mentionssoftware · OpenAI

    “do you think chat GPT or just say GPT 4 or J GPT 5 GPT 6”

    ▶ Watch at ~22:30approx.
  • PandoMentionsproduct · Pando

    “this episode is brought to you by Pando the always on employee performance platform”

    ▶ Watch at ~25:00approx.
  • Stanford UniversityMentionsplace

    “there is an amazing one actually introduction to AI by Stanford”

    ▶ Watch at ~26:30approx.
  • CourseraRecommendswebsite

    “there are some people that like learn offline so just go corsera there are so many courses”

    ▶ Watch at ~26:30approx.
  • Introduction to Artificial IntelligenceRecommendscourse · Stanford University

    “there is an amazing one actually introduction to AI by Stanford that they we encourage people to take a look”

    ▶ Watch at ~26:30approx.
  • PythonMentionssoftware · Guido van Rossum

    “I was actually gave talks ages ago called Dojo about Python”

    ▶ Watch at ~27:00approx.
  • CareerFoundryRecommendscourse

    “there is something called Career fundry which is a fantastic online coding school”

    ▶ Watch at ~27:00approx.
  • General AssemblyRecommendscompany

    “then I recommend these resources so there is something called Career fundry which is a fantastic online coding school general assembly and then coding Dojo”

    ▶ Watch at ~27:00approx.
  • Coding DojoRecommendscourse

    “general assembly and then coding Dojo”

    ▶ Watch at ~27:00approx.
  • arXivRecommendswebsite

    “there's a website called archive where you can see new papers come up”

    ▶ Watch at ~33:30approx.
  • Marginal RevolutionRecommendswebsite · Tyler Cowen and Alex Tabarrok

    “he has this awesome blog marginal Revolution and he's really good at sharing insights from research papers”

    ▶ Watch at ~35:00approx.
  • AI Product Management course by Marily NikaCoinedcourse · Marily Nika

    “my course is three week long it's meant for people that are either aspiring or current PMS”

    ▶ Watch at ~35:30approx.
  • Google CloudMentionsproduct · Google

    “this is offered by ro cloud and essentially it allows you to train high quality custom machine learning models”

    ▶ Watch at ~39:00approx.
  • AutoMLRecommendssoftware · Google Cloud

    “one of the tools I would like to recommend to people is actually autoing mail this is offered by ro cloud”

    ▶ Watch at ~39:00approx.
  • YouTubeMentionswebsite · Google

    “a great application I had to see there's actually a YouTube video about this”

    ▶ Watch at ~39:30approx.
  • DiscordUsessoftware · Discord Inc.

    “get to know each other on like an offline Discord Community”

    ▶ Watch at ~41:30approx.
  • InspiredRecommendsbook · Marty Cagan

    “inspired it taught me it's all about how to create products people love by Marty Kagan”

    ▶ Watch at ~44:00approx.
  • You Look Like a Thing and I Love YouRecommendsbook · Janelle Shane

    “you look like a thing and I love you and I have it right here it's a great thing super super cool”

    ▶ Watch at ~44:30approx.
  • Adventures of Women in Tech WorkbookCoinedbook · Marily Nika

    “it's a workbook for women in Tech trying to navigate working in Tech Tech it's called Adventures of women Tech workbook”

    ▶ Watch at ~44:30approx.
  • AmazonMentionscompany · Jeff Bezos

    “Amazon amazing”

    ▶ Watch at ~45:00approx.
  • Boz's podcastRecommendspodcast · Andrew Bosworth

    “I like boses podcast I don't know if you're aware of it”

    ▶ Watch at ~45:00approx.
  • The White LotusRecommendstv show · Mike White

    “oh my God the White Lotus people were talking about this thing”

    ▶ Watch at ~45:30approx.
  • LensaRecommendssoftware · Prisma Labs

    “well the lens out was pretty cool too right we all uploaded our photos”

    ▶ Watch at ~46:00approx.
  • Marily Nika's newsletterCoinednewsletter · Marily Nika

    “I'm also kicking off a newsletter”

    ▶ Watch at ~47:00approx.
  • Marily Nika's YouTube product channelCoinedwebsite · Marily Nika

    “I also have a product Channel on YouTube that you can check out I just started it”

    ▶ Watch at ~47:00approx.
  • InstagramUsessoftware · Meta

    “people can find me on Instagram”

    ▶ Watch at ~47:00approx.
  • SpotifyMentionssoftware · Spotify

    “you can subscribe to the show on Apple podcast Spotify or your favorite podcast app”

    ▶ Watch at ~47:30approx.
  • Apple PodcastsMentionssoftware · Apple Inc.

    “you can subscribe to the show on Apple podcast Spotify or your favorite podcast app”

    ▶ Watch at ~47:30approx.

Spot an error or want something removed? Request a correction or removal.

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Myth Buster· 2

Myth Buster04:30

AI Enhances Us, It Doesn't Steal From Us

Responding to writers fearful that AI will replace them, Nika pushes back on the idea that ChatGPT steals jobs. She frames it as both over-hyped and under-hyped at once, and argues technology enhances human worth rather than replacing it, pointing to lesser-known applications like light detection.

  • ChatGPT is simultaneously over-hyped and under-hyped
  • Fear that online writing will die is misplaced
  • Technology enhances human worth rather than stealing from it
  • AI has many under-discussed uses beyond chat, e.g. light detection

“tring B and technology is enhancing our worth it's enhancing us it does not steal from us”

Marily Nika · 05:00
#ai#hype#future-of-work
Myth Buster22:30

Will ChatGPT Replace Product Managers? Absolutely Not

Against the Twitter chorus declaring product management dead, Nika says AI absolutely won't replace PMs. Instead it frees up time by handling repetitive work like writing PRDs, letting PMs focus on strategy. She argues it will make PMs smarter and unlock new areas of product management.

  • AI won't replace PMs — it makes everything better
  • It frees up time by handling repetitive tasks like writing PRDs
  • Lets PMs focus on the more strategic side of the work
  • It will make PMs smarter and unlock new areas of product management

“oh absolutely not as I said like it makes everything better if anything it's going to free out time for me to to do other…”

Marily Nika · 23:00
#ai#product-management#future-of-work

Hot Take· 1

Hot Take08:30

Every Product Manager Will Be an AI PM

Nika predicts that all product managers will become AI product managers, because every product will need personalization, good recommender systems, and automation. She says PMs will need to get comfortable partnering with research scientists and, importantly, comfortable with the uncertainty that research carries.

  • All products will need personalized experiences and good recommender systems
  • PMs must get comfortable working with research scientists on their team
  • Research means tolerating uncertainty, unlike predictable ship cycles
  • Cross-functional teams may each end up with a research scientist

“I believe that old product managers will be AI product managers in the future and this is because we see all products needing to have…”

Marily Nika · 08:30

“whereas when you're working with research it's more like we're GNA try this and then in a year if it doesn't work out we're GNA…”

Marily Nika · 10:00
#ai#product-management#future-of-work#research

Explainer· 3

Explainer06:00

How Nika Uses ChatGPT in Her Day-to-Day PM Work

Nika shares three concrete ways she uses ChatGPT as a PM: rewriting mission statements so they inspire every discipline, generating user segments with motivations and pain points she wouldn't have thought of, and sparking new ideas. Crucially, she uses it after she already has a mission in mind, not as a replacement for her own thinking.

  • Rewrites mission statements to be understood and inspiring across all disciplines
  • Generates user segments plus their motivations and pain points
  • Used after she already has a direction, not to do the job for her
  • Example prompt: 'who would be interested in a fitness band that doesn't have a screen'

“it helps me create user segments in a fantastic way it will think of user segments that your mind wouldn't even go there like it…”

Marily Nika · 06:30

“but I'm not making it do my job for me I'm asking it after I have already had a mission in my head and what…”

Marily Nika · 07:00
#chatgpt#ai#product-management#tools
Explainer18:30

What a Model Is, Explained Like a Kid's Brain

Nika explains models using her three-year-old learning animals: repeat 'this is a rhino' enough times and the child recognizes one on the street. A model is like that brain — it takes an input (image, text, or voice) and outputs a recognition plus a probability of how certain it is. Training is feeding thousands of labeled examples until the model finds patterns on its own.

  • A model works like a child's brain trained by repetition
  • It takes an input and outputs a recognition with a confidence probability
  • Inputs can be images, text (as with ChatGPT), or voice (as with Whisper)
  • Training feeds thousands of labeled examples so the model learns patterns itself

“the model is like a kid's brain it has the ability to take an input which means it has the ability to take an image…”

Marily Nika · 19:30
#ai#machine-learning#explainer
Explainer31:30

How to Get Buy-In for a Risky ML Bet

To convince leadership to fund a big ML bet, Nika de-risks it two ways: she points to an adjacent product the company already shipped that succeeded and was AI-first, framing her proposal as similar to that once-crazy idea. Then she proposes a small commitment with an explicit rollback plan and a capped, minimal downside so the bet effectively starts at zero risk.

  • De-risk by citing an adjacent AI-first product the company already launched successfully
  • Frame the new bet as similar to a past idea that once seemed crazy but worked
  • Propose a small commitment with an explicit rollback plan
  • Cap the maximum negative impact so the downside stays minimal

“then I propose a little C CL like hey if that doesn't work out here's the roll back plan here's kind of the Maximum Impact…”

Marily Nika · 32:30
#ai#leadership#buy-in#product-management

Story· 1

Story21:30

The Google Glass Demo That Translated Speech in Real Time

Nika describes the AR/VR team at Google demoing Google Glass at Google IO: one person speaks a language, and the glasses take that audio as input, transcribe it, translate it, and display it on screen in the wearer's language. She calls it one of the most impactful things she has seen and insists it isn't science fiction anymore.

  • Google Glass took a speaker's audio, transcribed it, translated it, and displayed it on screen
  • It let two people speaking different languages communicate directly
  • Demoed on stage at Google IO
  • The technology exists now — it's about connecting the pieces, not science fiction

“the glass would take as an input the audio that came from that other person and it would transcribe it it would translate it and…”

Marily Nika · 22:00

“there's no science fiction anymore these things are real the technology is here it's just a matter of connecting the pieces to the puzzle in…”

Marily Nika · 22:30
#ai#google-glass#translation#ar

Q&A· 1

Q&A15:30

How Much Data Do You Actually Need for AI?

Asked for a rule of thumb on data volume, Nika says it depends entirely on the task. Classifying a cat versus a dog might work with 15-20 labeled photos, but voice recognizers or complex NLP need thousands. She notes teams sometimes synthesize fake data to have something to train and test on.

  • Data needs depend heavily on the task
  • Simple image classification may work with 15-20 labeled photos
  • Voice recognition or complex NLP needs thousands of examples
  • Teams sometimes synthesize fake data to train and test

“if you're trying to classify if the photo is a cat or a dog obviously even if you have I don't know like 15 20…”

Marily Nika · 16:00
#ai#data#machine-learning

Tool· 1

Tool39:00

AutoML + a Wind Turbine Drone Example for No-Code Model Training

Nika recommends AutoML (offered by Roboflow) for training custom machine learning models with no coding — you supply corrected photos and it does the rest. Her favorite example: a company inspecting wind turbines swapped manual ladder climbs for drones that photographed the machines, uploaded them to AutoML, and identified which needed maintenance, cutting the job from three weeks to a few hours.

  • AutoML (via Roboflow) trains custom ML models with no code
  • You provide already-corrected photos; it won't collect data for you
  • A wind turbine company used drones to photograph machines, then AutoML to flag maintenance needs
  • Inspection time dropped from about three weeks to a few hours

“one of the tools I would like to recommend to people is actually autoing mail this is offered by ro cloud and essentially it allows…”

Marily Nika · 39:00

“I think they reduced time from like three weeks of work to like a few hours of knowing which need maintenance and just be able…”

Marily Nika · 40:00
#autoies-ml#tools#no-code#machine-learning

Takeaway· 4

Takeaway00:00

Avoid the Shiny Object Trap: Don't Do AI for AI's Sake

Marily Nika warns against adopting AI just because it's trendy. The starting point must always be a real problem or pain point, not the technology. Only after identifying the problem and a high-level solution should you figure out how to implement AI.

  • There's a 'shiny object trap' of doing AI for the sake of doing AI
  • Start with a real problem or pain point that needs solving
  • Identify the problem and high-level solution before reaching out to implement

“hey don't do AI for the sake of doing AI make sure there is a problem there make sure there is a pain point that…”

Marily Nika · 00:00
#ai#product-management#strategy
Takeaway13:30

A PM Builds the Right Product; an AI PM Solves the Right Problem

Nika reframes the AI PM role: where a traditional PM helps their team ship the right product, an AI PM helps the team solve the right problem. Getting into AI product management means identifying the problem a data scientist will build a model to solve, and confirming there's a real audience and pain point for it.

  • Traditional PM: help the team build and ship the right product
  • AI PM: help the team solve the right problem
  • There must be a defined problem, an audience, a user, and a pain point before building a model

“I usually say that generally PM helps their team and their company build and ship the right product but the aipm helps their team company…”

Marily Nika · 13:30
#ai#product-management#role
Takeaway14:00

Don't Use AI for Your MVP — Fake It Instead

Nika argues that building a real model for an MVP makes zero sense because training takes weeks and wastes data scientists' time. To get buy-in for an AI feature, fake it with a Figma prototype and show users what the AI would do. Save real AI investment for when you already have data, or adjacent data you can leverage.

  • Don't train a model just to validate demand for an MVP
  • Fake the AI with a Figma prototype and test it on users
  • Model training can take weeks on powerful machines
  • Invest in AI only where you already have usable data or adjacent data

“don't do it for your MVB it makes zero sense do not waste time of data scientists that can train models”

Marily Nika · 14:30

“fake it create a little figma product type and just show it some users and just F what the AI is going to be doing”

Marily Nika · 15:00
#ai#mvp#product-management#validation
Takeaway29:30

The Real Challenges of Being an AI PM

Nika calls out the hard parts of AI product management: tolerating uncertainty when trained models don't match your hypothesis, keeping the team motivated as its captain, sourcing good data creatively, and a trickier career trajectory. Because PMs usually get ahead by launching often, research-heavy work means fewer launches — so clarify with hiring managers upfront how you'll be assessed.

  • Model results may not match your original hypothesis — expect uncertainty
  • The AI PM is the captain keeping the team motivated through setbacks
  • Good data is hard; you may need creative collection methods
  • Fewer launches in research work, so clarify assessment criteria with hiring managers early

“from a career trajectory usually product managers get ahead the more they launch but if you're in in a research or you're not going to…”

Marily Nika · 30:30
#ai#product-management#career#research