Context-First Agent Debugging
When a coding agent won't do what you want, treat it as a missing-context problem, not a model problem.
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 4
- Confidence
- 90%
OpenAI's internal experience is that most cases where a coding agent fails to complete a task are caused by under-specification or missing information, not model incapability. The fix is to encode the tribal knowledge in your head into the repository itself — through code comments, code structure, and text/MD files (skills, agents.md) — so the agent has what it needs to succeed on the next attempt.
Origin
Learnings from an internal OpenAI team running an experiment to maintain a 100% Codex-written codebase, shared by Sherwin Wu.
Core principles
- 01Most agent failures are context failures, not capability failures
- 02Tribal knowledge in a human's head is invisible to the agent until encoded in the repo
- 03The codebase itself is the durable place to store the context an agent needs
- 04Removing the human 'escape hatch' forces you to actually solve the context gap
How to run it
- 1
Diagnose the failure as a context gap
When the agent won't complete a task, first assume you underspecified it or there isn't enough information available to the agent about how to do it.
Watch out Don't immediately conclude the model can't do it — that reflex leads you to abandon the agent prematurely.
- 2
Locate the missing tribal knowledge
Identify what you know in your head about how this task should be done that the agent has no way to access.
- 3
Encode it into the repository
Add the knowledge as code comments, better code structure, or text files (MD files, skills, agents.md) inside the repo so the model can use it.
Pro tip Persist context in the codebase rather than re-explaining it per prompt, so future agent runs inherit it automatically.
- 4
Re-run and iterate
Have the agent retry with the added context and repeat the loop, steering rather than taking over.
In the wild
An internal OpenAI team maintaining a fully Codex-written codebase removed the usual fallback of dropping to tab-complete or hand-coding when the agent got stuck. Forced to make the agent succeed, they found failures were usually context problems and had to encode missing knowledge into the repo.
→ A set of best practices and paradigms fell out, slated for a public blog post.
Common mistakes
Reaching for the escape hatch too early
Rolling up your sleeves to hand-code the moment the agent struggles hides the real (context) problem and prevents you from building the durable documentation that makes agents reliable.
Blaming the model instead of the prompt/repo
Attributing failures to model incapability leads you to under-invest in specification and repo context, so the same failures recur across tasks.
Is it for you?
Best for
Engineers and teams leaning heavily on coding agents who hit repeated agent failures
Not ideal for
Genuinely novel tasks that exceed current model capability, where more context won't help
From the transcript
“when the coding agent is not doing what you want, it's usually a problem with context”
“encode more tribal knowledge that's in your head somehow into the codebase”
“either via you know code comments itself or code structure itself or via text files like you know MD files skills”
“this team uh uh for the experiment this team doesn't have that escape hatch”
From the episode
“Engineers are becoming sorcerers”
The future of software development with OpenAI’s Sherwin Wu