Build-With-The-Tools Assessment
Don't test people on doing work without AI — hand them the tools and score what they can build in an hour
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 3
- Confidence
- 80%
Foody's approach to assessing talent (and, by extension, what skills to hone) in the AI era. Fighting the tools is the calculator mistake — don't hand people arithmetic homework and forbid the calculator. Instead, tell candidates to use whatever AI tools they want and measure the product they can ship in a fixed window. The skill being measured is leveraging AI to do far more, which is the same skill worth developing in any vertical.
Origin
Brendan Foody, CEO of Mercor, drawing on Mercor's origins curating AI interviews and assessments, and conversations with teachers about assessing students in the AI era.
Core principles
- 01Don't fight people using the models — it's the calculator mistake repeated
- 02Assess the output people can produce with the tools, not their unaided ability
- 03Give a real build task and a fixed time box
- 04The measured skill — leveraging AI to do more — is the durable skill in any vertical
How to run it
- 1
Give a concrete build task, tools allowed
Set a real deliverable (e.g. 'build a website') and explicitly permit any AI tool — ChatGPT, Codex, Claude Code, Cursor.
Pro tip Frame it exactly as 'use whatever tools are available' so candidates optimize for output, not tool-avoidance.
- 2
Fix the time box
Constrain to a short window — Foody uses one hour — so the assessment measures leverage and judgment under time pressure.
- 3
Score the product, not the process
Evaluate what they actually shipped, treating AI-leverage as the competency being measured.
Pro tip This mirrors the skill worth honing personally: how to leverage AI to do far more in your own industry.
Watch out Testing unaided ability selects for a skill AI is erasing, not the one that will matter.
In the wild
Mercor gives candidates interviews that say: use ChatGPT, Codex, Claude Code, Cursor, whatever tools are available, and build a website — let's see what product you can build in an hour.
→ An assessment that measures AI-leverage, the competency Foody argues predicts future value.
Foody advises teachers not to give arithmetic homework engineered to prevent calculator use, but to say 'use the tools and let's see what you can do' — the same logic he applies to hiring.
→ Assessment reframed around tool-augmented output rather than tool-forbidden recall.
Common mistakes
Forbidding the tools in the test
Testing unaided performance is the calculator mistake — it measures a skill AI is making obsolete.
Scoring process over shipped product
What matters is what the person can actually build with leverage, not how 'purely' they did it.
Is it for you?
Best for
Hiring managers and educators designing assessments in the AI era
Not ideal for
Roles or exams where the point is to certify unaided fundamental competence (e.g. safety-critical manual skills)
From the transcript
“you don't want to fight against them using the models”
“we'll give people interviews where we say use chatbt and codeax use cloud code use whatever tool cursor and whatever tools are available to build…”
“it's sort of similar to like when the calculator come out you came out you don't want to give people all this this arithmetic homework”
From the episode
Why experts writing AI evals is creating the fastest-growing companies in history
Brendan Foody (CEO of Mercor)