LLM as Disconfirmation Engine

Core principles

• 01Straight-up general LLMs are good enough for most of this work
• 02Ask where your strategy does NOT fit the customer data, not where it does
• 03Public documents are usually a derivative of the underlying strategy, so models can infer it
• 04Value only emerges when you provoke the problems you don't want to hear
• 05LLMs collapse weeks of manual summarization into a fast, structured pass

How to run it

1. 1

   Test your strategy against customer voice

   Paste a batch of customer interviews and your strategy into the LLM and ask it to find where the strategy does not fit what customers talked about.

   Pro tip Explicitly instruct: 'tell me where my strategy does not fit' — the negative framing is the whole point.

   Watch out People spend far too much time looking for what they hope to see, not what they're missing.

2. 2

   Check competitor fit against your customers

   Copy a competitor's positioning documents into the LLM and ask whether their positioning is a better fit for what your customers said than your own.

   Pro tip This surfaces where a rival's story would resonate more with your base — a direct threat signal.

3. 3

   Reverse-engineer competitor strategy from public docs

   Feed a competitor's public documents and ask the model to summarize their probable product strategy. Because public materials derive from real strategy, the output is often 'creepily' accurate about what they'll likely do next.

   Pro tip Ask for ranked likelihoods ('more likely they'll do this than that') to prioritize your response.

   Watch out Treat the output as a hypothesis about a moving target, not a fixed fact — information has a decay rate.

In the wild

Common mistakes

Is it for you?

From the transcript