Build for Where the Models Are Going
Design products for the capability one to two years out, not today's model ceiling.
- Difficulty
- Advanced
- Time to result
- ~months to results
- Steps
- 4
- Confidence
- 92%
Because frontier models improve so fast, they routinely absorb the scaffolding and workarounds developers build around their current limitations — 'the models will eat your scaffolding for breakfast.' The winning move is to build a product that is ~80% possible today so it 'kind of works,' then have it click into something incredible as the next model lands, rather than over-engineering logic for a capability the model will soon do natively.
Origin
Sherwin Wu, applying Rich Sutton's 'bitter lesson' to product-building; the 'eat your scaffolding for breakfast' phrasing is credited to Nicholas, founder of the startup FinTool.
Core principles
- 01Models tend to disrupt themselves, especially in the tooling and scaffolding space
- 02As models get better, removing logic and trusting the model beats hand-built architecture (the bitter lesson)
- 03Today's fashionable scaffolding (agent frameworks, vector stores, skills files) may be obsolete tomorrow
- 04Customer feedback reflects a local maximum, so balance it against where the models are heading
How to run it
- 1
Identify the ideal end-state capability
Define the product experience assuming the model can do the task well, not around what it can barely do today.
- 2
Ship at ~80% readiness
Build the product for a capability that is roughly 80% of the way there today, so it works imperfectly now but is positioned to unlock fully on the next model release.
Pro tip You often don't need to wait long — models improve so quickly that the unlock can arrive within months.
- 3
Minimize scaffolding, maximize trust in the model
Strip out hand-built logic and give the model simple tools (e.g. any search backend, files on disk) instead of assuming one heavyweight architecture is the permanent answer.
Pro tip Treat scaffolding as disposable — vector stores gave way to file-based context and skills as models improved.
Watch out Don't assume the current scaffolding is the only scaffolding you'll ever need; it is a moving target.
- 4
Weight customer feedback against the trend line
Listen to customers but discount requests that only optimize today's local maximum (e.g. 'a better vector store') against where model capability is heading.
Watch out Blindly chasing exact customer feature requests can lock you into a local maximum the models will render obsolete.
In the wild
Around 2023 the field assumed you had to embed your entire corpus into a vector store and heavily optimize vector search to feed context in. As models improved, a better approach became stripping that logic out and just handing the model a set of search tools — which could be plain files on a filesystem (skills, agents.md).
→ The elaborate vector-store scaffolding and ecosystem largely gave way to simpler file-based context management.
Common mistakes
Over-architecting around today's limits
Building elaborate frameworks to compensate for a current model weakness wastes effort because a smarter model soon does the task natively and 'eats' the scaffolding — OpenAI's own API team admits taking wrong turns this way.
Building only for today's static capability
Assuming the model is static produces a product capped at today's ceiling; you miss the compounding unlock and end up with a worse experience than a competitor who designed for the trajectory.
Is it for you?
Best for
AI product builders and startup founders deciding how much scaffolding to invest in on top of frontier models
Not ideal for
Regulated or safety-critical systems that require deterministic, fixed behavior and can't ride an improving capability curve
From the transcript
“make sure you're building for where the models are going and not where they are today”
“the models will eat your scaffolding for breakfast”
“a better approach is actually to take out a lot of that logic and trust the model and give it a set of tools for…”
“if you just blindly listen to your customers they'll be like yeah I want a better vector store”
From the episode
“Engineers are becoming sorcerers”
The future of software development with OpenAI’s Sherwin Wu