Adapt the Model, Don't Build One
Post-training is the new pre-training — steer an off-the-shelf model to your outcome instead of pre-training your own
- Difficulty
- Advanced
- Time to result
- ~months to results
- Steps
- 4
- Confidence
- 90%
A decision rule for how to invest AI budget: above roughly 30 billion parameters, the capex to pre-train your own foundation model stops making economic sense. Instead, take a strong off-the-shelf model and adapt it via post-training — fine-tuning or reinforcement learning on your own, purchased, or synthetic data — to optimize for a specific outcome.
Origin
Asha Sharma, drawing on Nathan Lambert's study of top leaderboards showing the ~30B-parameter post-training threshold, and survey data that ~50% of developers now fine-tune. She frames post-training spend eventually exceeding pre-training spend.
Core principles
- 01Above ~30B parameters, pre-training your own model rarely makes economic sense
- 02You get more economic and 'taste' leverage by steering an existing model than building from scratch
- 03Adaptation data can be your own, purchased, synthetic, or a mix
- 04Same industrial logic as ranking — never ship what's off the shelf as-is; tailor it to your use case
- 05Prefer a model system / ensemble over 'one model to rule them all'
How to run it
- 1
Default to adapting, not building
Assume you will start from a strong off-the-shelf model rather than invest heavily in pre-training your own, given the compute, science expertise, and capex required.
Pro tip Rising scientist costs and scarce expertise make DIY pre-training even less attractive — let someone else pay that capex.
- 2
Choose the model(s) per use case
Select from the many good domain models by what each task needs — latency, thinking time, retrieval speed, quality — favoring a model system or ensemble over a single model.
Pro tip Different models suit different jobs (e.g. one is great where you want quick retrieval, another where thinking time is fine); match model to task.
- 3
Decide your adaptation data source
Pick the data to steer the model: your own proprietary data (strongest), purchased data, synthetic data, or a combination.
- 4
Post-train toward the outcome and run the full loop
Apply fine-tuning or reinforcement learning to optimize the model for price, performance, or quality — and go through the full loop rather than stopping at basic fine-tuning.
Pro tip Fine-tuning alone helps, but running the complete rewards-and-iteration loop yields materially better results.
In the wild
GitHub uses an ensemble of models fine-tuned across ~30 countries and their languages, iterating in a loop for next-edit suggestions and code completions.
→ Tailored, multi-language code assistance from adapted models rather than one monolithic pre-trained model.
Common mistakes
Investing to pre-train your own large model
Above ~30B parameters the capex and scarce science expertise rarely pay off versus adapting an existing model.
Shipping the off-the-shelf model as-is
Like taking raw ranking output or default React components, an un-adapted model isn't tailored to your specific users or use cases.
Is it for you?
Best for
Founders and product leaders deciding how to spend AI budget who have a specific outcome and access to relevant data
Not ideal for
Frontier labs whose entire business is pushing the pre-training frontier, or teams with no differentiating data to adapt on
From the transcript
“once a model hits 30 billion parameters the capex to actually train a model and put you know billions of tokens into a a kind…”
“I believe we will see you know just as much money spent on post-training uh as we will on pre-training and in the future more…”
“more and more companies and organizations kind of start to think about how do I adapt a model rather than how do I take something…”
“I'm much more in the like model system uh rather than one model to rule them all”
From the episode
How 80,000 companies build with AI: products as organisms, the death of org charts, and why agents will outnumber employees by 2026
Asha Sharma (CVP of AI Platform at Microsoft)