Eval-First AI Product Development
Define what 'correct' looks like as evals first; the spec becomes the scorecard, not the build instructions
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 4
- Confidence
- 88%
As AI writes more of the implementation, the leverage in product work shifts from specifying HOW to build to specifying what CORRECT looks like. This framework turns that into a concrete practice: build a prompted-model baseline, encode desired behavior as deterministic pass/fail evals plus human win-rate evals, and use a prompted-baseline-scores-lowest test to know your eval is actually robust.
Origin
Articulated by Karina Nguyen based on teaching product managers at OpenAI (and drawing on Anthropic practice) how to think about evaluations for AI features.
Core principles
- 01Product development is shifting from 'here's a PRD, let's build it' to 'here's what correct looks like, AI build it.'
- 02A prompted model is your baseline — the floor you must beat, and a fast way to prototype the target behavior.
- 03A robust eval is one where the prompted baseline scores the lowest; if the baseline already aces it, the eval isn't discriminating.
- 04Optimizing one behavior must not regress other areas of intelligence — always watch for collateral damage.
- 05Two eval types cover most cases: deterministic pass/fail with ground-truth labels, and human win-rate comparisons across model completions.
How to run it
- 1
Stand up a prompted-model baseline
Before training anything, prompt an existing model to attempt the behavior. This is both your prototype and the baseline score every future model must beat.
- 2
Build deterministic evals with ground-truth labels
For behaviors with a clear right answer, create labeled cases — e.g. a Google Sheet with tabs for current behavior, ideal behavior, and notes. 'Under this conversation it should trigger Canvas; under this one it should not.' Score pass/fail.
Pro tip The same labeled spreadsheet can double as training input — hand it to a model and it can figure out how to teach itself the good behavior.
- 3
Add human win-rate evals for quality judgments
For subjective quality (which edit or comment is better), show trainers or internal people a prompt plus several model completions and have them pick the winner. Track continuous win rates so each new model should beat the prior one.
- 4
Validate the eval with the baseline-lowest test
Confirm your eval is discriminating: the prompted baseline should get the lowest score. If a trained model then hill-climbs on that eval, you know it genuinely improved.
Watch out Watch that hill-climbing on one eval isn't regressing other intelligence evals — this happens constantly in every lab.
In the wild
For the Tasks feature, correct scheduling was hard for the model, so the team wrote deterministic evals: if the user says 7pm, the model must output 7pm. Pass or fail, no ambiguity.
→ A concrete, testable target that let them ship correct schedule extraction.
Product managers and model designers went through user conversations and labeled ground truth — which conversations should trigger Canvas and which should not — creating a bi-deterministic eval for the trigger behavior.
→ A measurable trigger boundary the team could optimize against.
Common mistakes
Building an eval your baseline already passes
If a prompted baseline scores well on your eval, the eval isn't discriminating and can't tell you whether training helped. Design evals where the baseline scores lowest so improvement is visible.
Optimizing a single eval blind to regressions
Pushing one behavior's score up can quietly degrade other areas of the model's intelligence. Track a broader eval suite, not just the target metric, or you ship a model that's better at one thing and worse overall.
Is it for you?
Best for
Product managers and designers building AI features who need to shift their craft from writing specs to defining and measuring correctness
Not ideal for
Early open-ended exploration where the desired behavior isn't yet known well enough to label — prototype first, then formalize evals
From the transcript
“you can have a deterministic evals with this like pass or fail”
“you can have prompted model as a baseline already”
“which model is the best, which model produce the highest quality comment or edit, and then you can have like continuous win rates”
“like you kind of don't want to like bring damage in like other areas of intelligence”
From the episode
OpenAI researcher on why soft skills are the future of work
Karina Nguyen (Research at OpenAI, ex-Anthropic)