The Four Challenges of AI Product Management
Uncertainty, pivots, data scarcity and a broken promo path — the four taxes of the AI PM role.
- Difficulty
- Advanced
- Time to result
- ~ongoing to results
- Steps
- 4
- Confidence
- 90%
Before taking an AI PM role, Nika insists you go in with eyes open on four structural differences from regular product management. Each has a specific countermeasure: research uncertainty needs you to act as morale captain; forced direction changes need leadership skill; data scarcity needs creative collection; and the launch-based promotion path breaks, so you must renegotiate your assessment criteria with your hiring manager before you accept.
Origin
Marily Nika's own experience across eight years at Google (Glass, computer vision, speech) and Meta, taught as the 'challenges' module of her AI PM course.
Core principles
- 01Research does not resolve on a launch calendar — 'we'll try this and in a year if it doesn't work we shut it down' is a normal outcome.
- 02The PM is the captain of the ship through model results that contradict the hypothesis.
- 03Good data is hard; you must be willing to do anything to get it.
- 04PMs are usually promoted on launches, and AI PMs launch less often — so the assessment criteria must be renegotiated in advance.
How to run it
- 1
Accept the uncertainty tax
Expect that after all the research, hypotheses and ideas, the trained model's results may simply not be optimal or answer your hypothesis. Plan the project as a bet, not a sequence of launches.
Watch out PMs used to a rhythm of 'do this, launch, do this, launch' are the ones who burn out fastest in research settings.
- 2
Take the morale-captain role deliberately
When results disappoint, you are the one who has to encourage the team and keep it moving. Treat that as an explicit part of the job description, not an accident.
- 3
Get creative and relentless about data
Assume good data will not be sitting there. Be willing to invent collection methods — up to and including going out on the street and asking people to contribute data, or synthesising fake data to have something to train and test with.
Pro tip The amount needed scales with the problem: a cat-vs-dog classifier may work on 15-20 labelled photos; voice recognisers or complex NLP need thousands.
- 4
Renegotiate your promotion criteria before you start
Ask the hiring manager explicitly, early: what does progress mean in this role, and how will I be assessed given research work doesn't launch on a normal cadence?
Pro tip Do this during the interview, not at your first performance review — by then the launch-count metric is already being applied to you.
Watch out Taking an AI/research PM role on standard launch-based promotion criteria is a career trap.
In the wild
Nika describes research-adjacent product work as 'we're gonna try this and then in a year if it doesn't work out we're gonna shut everything down' — a rhythm that PMs accustomed to continuous launches find deeply uncomfortable.
→ PMs who pre-accept this cadence, and who pre-negotiate how they'll be assessed, survive research orgs; those who don't, complain and stall.
Common mistakes
Assuming your launch-based promo path still works
Product managers usually get ahead the more they launch. In a research org you launch far less, so without a renegotiated definition of progress you will underperform against a metric that doesn't fit your work.
Waiting for good data to appear
Getting good data is hard and rarely solved by waiting. Nika expects the AI PM to invent the collection route — street collection, adjacent products, or synthesised data — rather than treating scarcity as a blocker.
Is it for you?
Best for
PMs considering or newly moved into an AI/ML or research-adjacent product role.
Not ideal for
PMs on conventional feature teams where model uncertainty and research timelines aren't part of the job.
From the transcript
“there are few challenges that people need to be aware of number one and I kind of mentioned it before is the uncertainty”
“usually product managers get ahead the more they launch but if you're in in a research or you're not going to launch as often so…”
“you may get on the street and ask for people to actually contribute data for what is you're doing”
From the episode
AI and product management
Marily Nika (Meta, Google)