Think First, Then Challenge with AI
Reverse the default AI workflow: form your own answer, then use the model to attack it.
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 4
- Confidence
- 90%
Cagan publicly reversed his own advice on using generative AI for product work. He used to tell people to start with ChatGPT and improve from there; he watched people take the model's output too literally, head in the wrong direction, and then optimize that wrong direction. His new instruction is to think the answer through first, write it down, and only then bring the model in — to challenge the argument, tighten it, and try to beat it.
Origin
Marty Cagan (SVPG), 2024 — an explicit reversal of his own earlier 'start with ChatGPT' guidance after observing the failure pattern in practice. Pairs with the Leslie Lamport line he quotes on writing as thinking.
Core principles
- 01Starting with the model anchors you to its answer, and you then optimize a wrong direction.
- 02The model is best used adversarially — as a challenger of a position you already hold.
- 03'If you're thinking without writing you just think you're thinking' — the written artefact is the precondition for the AI step.
- 04The right question is not what generative AI can be used for (almost everything) but what it is good for.
- 05Judgment-heavy work (viability, ethics, probabilistic vs deterministic risk) is where humans still carry the load.
How to run it
- 1
Form your own answer first
Before opening any model, think the problem through and put something down — a strategy, a spec, a PRD, an argument. The act of writing is what forces the thinking.
Pro tip Cagan attributes the underlying idea to Leslie Lamport: if you're thinking without writing, you just think you're thinking.
Watch out Skipping this step is the whole failure mode — you will end up refining the model's direction instead of your own.
- 2
Bring the model in as a challenger
Hand your written position to the model and ask it to improve it, challenge it, find the holes, and make the argument tighter — rather than asking it to produce the answer.
- 3
Discount the output deliberately
Treat what comes back as material to evaluate, not as a result. Cagan's stated reason for the reversal is that people trust the results too much.
Watch out The techniques are a moving target — what works depends on the system and the week; do not build a rigid process on top of them.
- 4
Reserve viability judgment for yourself
For decisions involving probabilistic versus deterministic software — legal exposure, ethics, mission-critical correctness — recognise these as value and viability questions, which land squarely on the product manager and cannot be delegated to the model.
Pro tip Cagan expects viability to become MORE important, not less, as generative AI spreads.
In the wild
Cagan originally told product people to start with ChatGPT and then improve from there. He kept seeing people take what they got too literally, too seriously, treat it as too valuable, head off in a wrong direction — and then optimize that direction.
→ He now recommends the opposite order: think it through, write it down, then use the model to try to improve or challenge it.
Cagan describes being on many calls about the implications of probabilistic versus deterministic software: lawyers weighing in, ethical questions, and the mission-critical question of whether a probabilistic answer is acceptable at all.
→ He classifies these as value and viability questions, meaning more responsibility lands on the product manager than in the past — the opposite of the assumption that AI shrinks the PM job.
Common mistakes
Starting with the model
Prompting first anchors your thinking to the model's framing, and the subsequent work compounds a wrong direction rather than correcting it.
Asking what AI can be used for
Almost nothing is off-limits, so the question carries no information. The harder and more useful question is what it is actually good for.
Assuming AI reduces the PM's judgment load
For an empowered PM, viability — legal, compliance, ethics, mission-criticality of probabilistic outputs — becomes more important, not less.
Is it for you?
Best for
Product managers and knowledge workers using LLMs for strategy documents, specs, and arguments who notice their output converging on the model's first draft
Not ideal for
Pure generation tasks with no judgment content (boilerplate, format conversion) where anchoring costs nothing
From the transcript
“what I kept seeing was people taking what they get too literally too seriously too much value and they were heading off in a wrong…”
“now I've been recommending to people that they think through the answer first”
“then use chat GPT to see if you can't improve on that to see if you can't challenge that”
“it's hard to think of something you can't use it for the harder question is what is it good for”
“if you're thinking without writing you just think you're thinking”
From the episode
Product management theater
Marty Cagan (Silicon Valley Product Group)