Complement-the-Frontier Model Strategy
Don't rebuild foundation models — train small specialty models that attack their weaknesses in speed, cost, and niche tasks.
- Difficulty
- Expert
- Time to result
- ~months to results
- Steps
- 4
- Confidence
- 85%
Cursor uses an ensemble: the biggest foundation models for high-level reasoning, plus their own small, fast, specialty models where the frontier models are too slow, too expensive, or simply not built for the task. The strategy is to be pragmatic — never reinvent the wheel (start from the best pre-trained/open-source models), and deploy custom models only at the frontier models' weak points and around their input and output.
Origin
Michael Truell explaining Anysphere/Cursor's model stack; Truell notes OpenAI CPO Kevin Weil calls the same idea an 'ensemble of models.'
Core principles
- 01Don't replicate what the big providers already do well (e.g. pre-training from scratch) — it's the wrong place to spend effort.
- 02Start from the best existing pre-trained models, often open source, and post-train them.
- 03Deploy custom models specifically where foundation models are weak — cost, speed, or a niche task they were never built for.
- 04Wrap the big models: custom models on the input (retrieving relevant code) and on the output (turning high-level sketches into full diffs fast).
How to run it
- 1
Map foundation-model strengths and weaknesses
Identify where the best foundation models are excellent (leave those alone) and where they fail on cost, latency, or task fit.
Pro tip Pick your spots carefully — focus on their weaknesses, not their strengths.
Watch out Don't anthropomorphize models — being great in one area doesn't mean they're good everywhere; they have massive issues.
- 2
Use frontier models for high-level reasoning
Let the smartest, most expensive models spend a few tokens doing the high-level thinking and sketch the changes.
- 3
Train specialty models for speed/cost-bound tasks
Where a foundation model can't serve (e.g. sub-300ms autocomplete on every keystroke), train your own fast, cheap model — here specialized to autocomplete a series of diffs, not generic text.
Pro tip Start from the best open-source pre-trained models, sometimes tuning closed-source ones in collaboration with providers, rather than training from scratch.
Watch out Replicating pre-training (teaching a network the whole internet) is enormously costly and the wrong place to focus.
- 4
Wrap the big models on input and output
Put custom models on the input to retrieve the relevant parts of the codebase (a mini codebase-specific search) and on the output to turn the big model's high-level change sketches into full code diffs quickly.
In the wild
Cursor's next-edit prediction must return a completion within ~300ms on every keystroke and predict diffs across multiple files — a specialty, latency-bound task no foundation model serves well, so Cursor trained its own model for it with no foundation model involved.
→ A core, heavily-used Cursor capability; every 'magic moment' in Cursor now involves a custom model in some way.
Common mistakes
Trying to replicate pre-training from scratch
Teaching a neural net the whole internet is astronomically expensive and duplicates what excellent providers already offer — Truell ran the GPT-4 training cost calc and knew it was the wrong place to compete.
Anthropomorphizing models into general competence
Seeing a model do something magical in one area and assuming it's therefore great everywhere ignores that these models have massive issues; it leads to over-trusting them where they're weak.
Is it for you?
Best for
AI application builders deciding how much model development to do in-house versus relying on foundation-model APIs.
Not ideal for
Teams with no ML capacity, or products where off-the-shelf foundation models already meet all cost/latency/quality needs.
From the transcript
“picking your your spots carefully, not trying to reinvent the wheel, not trying to focus on places maybe where the the the best foundation models…”
“at this point every magic moment in Kerserv involves a custom model in some way”
“They need to give you a completion within 300 milliseconds”
“we take the sketches of the changes that these models are suggesting you make with that codebase. And then, you know, we have models that…”
From the episode
The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using
Michael Truell (co-founder and CEO)