Build Only Where You Can Be Best (Model Sourcing)
Build in-house only where your unique data or position lets you beat the frontier; otherwise buy
- Difficulty
- Expert
- Time to result
- ~months to results
- Steps
- 4
- Confidence
- 88%
A build-vs-buy decision rule for AI-native companies choosing which model components to train versus which to outsource to frontier providers. The test is not ego but defensibility: build only where you have a coherent argument (usually proprietary data or a hard technical position) for being better than the frontier; for everything else, use the best available open-source or third-party model.
Origin
Varun Mohan explaining how Windsurf composes its stack — using Anthropic's Sonnet for high-level planning while building proprietary in-house models for retrieval, edits, and autocomplete.
Core principles
- 01Your only goal is to build the best possible product and raise the ceiling as high as possible
- 02Never build something just for the sake of ego when open source or a frontier model is better
- 03Proprietary usage data you uniquely possess is the strongest reason to build
- 04If you can't articulate a coherent argument for why you'd be best, that's a signal to buy
How to run it
- 1
Decompose the task into components
Break the workflow into distinct model jobs — e.g. high-level planning, codebase retrieval, code edits, autocomplete — and evaluate each separately rather than the system as a whole.
- 2
For each component, ask 'can we coherently argue we'd be best?'
Test whether you have a real edge — proprietary data or a technical advantage. For pure high-level planning, Mohan can't construct that argument, so they use Sonnet/GPT. For retrieval and edits, their usage data gives them one.
Pro tip The tell is your own data looking unlike internet data. Windsurf's autocomplete preference data captures code mid-typing (incomplete), which frontier models rarely see — a defensible edge.
Watch out If your inputs are just high-level web elements everyone has, the frontier models will likely match or beat you; don't build there.
- 3
Build in-house where you have the edge, buy where you don't
Train proprietary models for the defensible components (Windsurf pre-trains its retrieval model fully in-house and post-trains edit/autocomplete models on open source). Use frontier or open-source models for the rest.
Pro tip Building your own edit model has side benefits: lower latency and the ability to hold more of the codebase in context than the frontier provider.
Watch out You physically can't send a 100M-line (1.5B+ token) codebase to a frontier model — cost, latency, and context limits force smart retrieval regardless.
- 4
Compound the data advantage over time
Capture preference and acceptance signals (tens of millions of feedback events per hour) and feed them back into the components you own, widening the moat competitors can't replicate.
Pro tip Use acceptance signals — did the user accept the change after a retrieval? — as a label for whether retrieval was good.
In the wild
Windsurf uses Anthropic's Sonnet for high-level planning because they can't argue they'd beat it there. But retrieval over 100M-line codebases and fast code edits run on models they trained themselves, ranking code snippets across thousands of GPUs in parallel.
→ They ship high-quality large-codebase understanding and faster edits than sending everything to a frontier model, while spending in-house effort only where it's defensible.
Common mistakes
Building models for ego
Training a model you can't beat the frontier at wastes effort; Mohan explicitly refuses to build 'just for the sake of ego.'
Trying to differentiate at too high a plane
If you only consume high-level web elements and sit on top of Sonnet doing what every wrapper does, there's no clear place to keep differentiating.
Is it for you?
Best for
AI-native product teams deciding which parts of their model stack to train versus outsource to frontier providers
Not ideal for
Teams without proprietary data or a technical edge, for whom building models is pure cost with no defensible payoff
From the transcript
“what could we be best at and we will go ahead and trade. But if we're not going to be best at it, we should…”
“if something is just purely like code planning there's not a great reason why we would be the best at that”
“every hour we get probably tens of millions of pieces of feedback from our users”
“we are uniquely well positioned to actually build a good model that can complete code even when it's in an incomplete state”
From the episode
Building a magical AI code editor used by over 1 million developers in four months: The untold story of Windsurf
Varun Mohan (co-founder and CEO)