Define Success Before You Prompt
The clearer your definition of success and failure, the better the work you get from people or AI.
- Difficulty
- Moderate
- Time to result
- ~days to results
- Steps
- 3
- Confidence
- 90%
Zhuo's single highest-leverage skill for working with AI is being crystal clear on the outcome — boiling a high-level human intention down into something an agent can objectively recognize as success or failure. She links this directly to why evals, metrics, and KPIs matter: they are all attempts to make 'what good looks like' objective. This is the same skill that resolves alignment problems on human teams.
Origin
Julie Zhuo, drawing on her work at Sundial automating data analysis and her management experience at Facebook.
Core principles
- 01Vague success criteria produce mediocre work from both humans and AI
- 02Humans naturally think at high levels; the work is boiling that down into concrete, testable criteria
- 03Evals, metrics and KPIs exist to make success objective — it is an art, not a science
- 04Alignment failures on teams usually trace to people holding different pictures of success
How to run it
- 1
State the general intention
Write the high-level thing you want — 'I want lots of people to hear this and take something away' — as your starting point.
Watch out Stopping here is the mistake; this is too general for anyone or any agent to execute against.
- 2
Boil it down to objective criteria
Get progressively more specific until you could tell without question whether the result hit the mark.
Pro tip Treat writing evals as the discipline of forcing yourself to name objective success and failure conditions.
- 3
Encode it into the prompt or brief
Feed that concrete definition of success into the agent or teammate so they optimize toward the right target.
Pro tip If output is weak, suspect an unclear success definition before blaming the model.
In the wild
Zhuo notes that saying 'I want lots of people to hear it and take away things' is very general, and the real work is getting specific enough 'that we know without question whether we've hit it or not' — the same problem her data company solves with metrics.
→ Clarity that lets an agent understand what success and failure look like, which she calls 'a lot of the game.'
Common mistakes
Leaving the goal at a human, high level
If you are unclear about what success looks like in the prompt, 'you're probably not going to get the most amazing work' — the agent optimizes toward an ambiguous target.
Treating success definition as a science
Choosing which measure represents success and interpreting whether a change is good is interpretation and art; pretending it is objective creates false precision.
Is it for you?
Best for
Anyone briefing AI agents or teammates on non-trivial creative or analytical work
Not ideal for
Tasks where the success criteria are already obvious and shared
From the transcript
“being really really crystal clear on what does success look like”
“figuring out how to boil it down so that an agent can really understand what success and failure looks like is a lot of the…”
“that's why we have to write eval and that's why they're so important because they're helping us understand what is the objective criteria”
“if you're really unclear about what success looks like the prompt, you're probably not going to get the most amazing work”
From the episode
From managing people to managing AI: The leadership skills everyone needs now
Julie Zhuo (Facebook VP, Sundial CEO, The Making of a Manage