Model Introspection Harness Repair
When an AI agent misbehaves, ask it why — its explanation reveals the harness gap to fix
- Difficulty
- Moderate
- Time to result
- ~days to results
- Steps
- 4
- Confidence
- 90%
Rather than guessing why an AI model made a wrong decision, Cat asks the model to introspect on its own behavior. The model's explanation of what confused it — a muddled system prompt, an unclear task boundary, an unchecked sub-agent — points directly at the harness defect to repair. It reframes debugging an agent from black-box trial-and-error to a curiosity-driven diagnostic conversation.
Origin
Cat Wu's personal practice for understanding and improving Claude Code's behavior.
Core principles
- 01The model can often diagnose its own failure better than you can guess it
- 02Curiosity about the model's reasoning surfaces what misled it
- 03The fix usually lives in the harness (prompts, task definition, sub-agent handoffs), not the model weights
- 04This is how you elicit maximum capability from the current model
How to run it
- 1
Notice unexpected behavior
Watch for the model doing something surprising — e.g. making a front-end change and running tests but never actually using the UI to verify.
Pro tip Spend real time using and watching the model so you can spot these deviations at all.
- 2
Ask the model to reflect on why
Prompt the model to introspect on why it took that action instead of assuming the cause. Let it name the confusion.
- 3
Read the explanation for the harness gap
The model may reveal a confusing system prompt, an unrecognized task requirement, or a sub-agent it delegated to and never checked. Each is a concrete, fixable harness defect.
Watch out The model's self-report is a hypothesis, not ground truth — confirm the pattern in real data before over-fitting a fix to one anecdote.
- 4
Fix the harness to close the gap
Patch the prompt, clarify the task boundary, or add a verification check so the failure mode can't recur.
In the wild
Cat noticed the model would make a front-end change and run tests but not actually use the UI. Asking it to reflect surfaced causes like a confusing system prompt, not realizing front-end verification was part of the task, or delegating verification to a sub-agent whose work it never checked.
→ Identified the specific harness gap misleading the model so it could be fixed to close the verification hole.
Common mistakes
Guessing the cause instead of asking the model
Assuming why an agent failed leads to fixes aimed at the wrong defect. The model's introspection often points straight at the real confusion.
Treating one self-report as proof
A single introspection is a hypothesis. Without checking whether it reflects a larger pattern in the data, you risk patching a one-off and missing the systemic issue.
Is it for you?
Best for
PMs and engineers building on agentic LLM harnesses who need to debug why an agent behaves unexpectedly
Not ideal for
Failures rooted in model weights or capability ceilings that no harness change can fix, or teams without access to the agent's reasoning traces
From the transcript
“One of the things I really like to do is to ask the model to introspect on its own behaviors.”
“A lot of times just like being very curious about why the model made the decision that it did will show you what misled it,…”
From the episode
How Anthropic’s product team moves faster than anyone else
Cat Wu (Head of Product, Claude Code)