Diagnose Agent Failure as Structural, Not Stupidity
When an agent does the wrong thing, check its context, tools, and scope — not its intelligence.
- Difficulty
- Moderate
- Time to result
- ~days to results
- Steps
- 3
- Confidence
- 90%
When an agent underperforms, the instinct is to think it's dumb — but in an agentic system the real cause is almost always structural: missing context, unclear tool access, or fuzzy scope. Because the agent's knowledge lives in an inspectable file system, you can literally check whether it has the information and permissions to do the job, then fix the structure rather than blaming the model.
Origin
Claire Vo applies the 'waterline model' from Molly Graham's guest post (via Lenny's) — that most team problems are structural (unclear role, goal, or ownership), not the person — to AI agents.
Core principles
- 01Agent failures are usually missing context, tools, or scope — not lack of intelligence
- 02The file system makes the structural gap literally inspectable
- 03Communication is lossy; what you told it once, or weeks ago, it forgets
- 04Fixing the agent is a mirror of how good your own documentation and tasking are
How to run it
- 1
Inspect the file system for the gap
When the agent does the wrong thing, open its files and ask concretely: does it have this information? Should it be able to do this job? The answer is usually a visible missing piece.
Pro tip Unlike managing a human, the 'does it know this?' question is literally answerable by reading the agent's own files.
- 2
Check the tools document, not just memory
The most-overlooked failure is the agent forgetting what tools it has and how to use them ('I can't read that email' when it can). Edit the tools.md by hand to spell out available tools and how you want them used.
Pro tip Claire avoids hand-editing the agent's 'soul' but does hand-edit the tools file, because tool usage has nuances (how to read a calendar, which task system to use) worth encoding precisely.
Watch out Symptoms like 're-authenticate' or 'can't access that file' are usually tool-context gaps, not real capability limits.
- 3
Maintain operational hygiene to prevent context loss
Before a long topic gets compacted, do a check-in: 'write this to memory, update the to-do list, capture the action items.' Treat it like taking minutes at the end of a meeting.
Pro tip OpenClaw hooks can automate writing memory and action items so context survives compaction.
In the wild
Claire repeatedly heard agents claim they couldn't read an email address they demonstrably could access. The cause wasn't incapacity but a gap in the tools.md describing what the agent could use and how — a structural documentation fix, not a model problem.
→ Editing the tools document by hand resolved the recurring 'can't access' complaints without changing the model.
Common mistakes
Blaming the model's intelligence
Assuming a misbehaving agent is 'dumb' leads you to distrust the tool instead of fixing the real cause — missing context, unclear scope, or undocumented tools — which is where the fix actually lives.
Venting frustration at the agent
Typing angry messages ('I just told you this!') is as ineffective on an agent trained on human data as it is on an employee; it fixes nothing. The productive move is to correct the structural gap in its files.
Is it for you?
Best for
Managers and operators debugging why an agent underperforms who want a systematic diagnosis instead of frustration
Not ideal for
Failures that are genuinely model-capability limits (e.g. unreliable browser use) where no amount of context fixes the underlying tool
From the transcript
“if your bot is doing the wrong thing, it's not that it's a dumb, it just doesn't have the context”
“You can actually go into its file system and say does it have this information? Should it be able to do this job?”
“the thing that I find it forgets the most that people don't think about enough is what tools it can use and how”
From the episode
From skeptic to true believer: How OpenClaw changed my life
Claire Vo