Objective-Data Alignment Test
Before betting on an AI capability, check that your training data is the same shape as your desired output.
- Difficulty
- Moderate
- Time to result
- ~days to results
- Steps
- 4
- Confidence
- 86%
A diagnostic for judging how hard an AI/ML problem will be: compare the objective function (what you want the model to output) against the form of the training data you actually have. When they are the same modality, progress is fast; when they diverge, you must manufacture bridging data first. Use it to sanity-check any ML project, roadmap, or investment before committing.
Origin
Fei-Fei Li used it to explain why language models advanced so much faster than robotics. Language researchers had 'perfect alignment' — text tokens in, text tokens out. Robotics wants actions in 3D worlds as output, but its available data (web videos) contains no actions, forcing teams to supplement with teleoperation and synthetic data.
Core principles
- 01Name the objective function (the exact form of the output you want) explicitly.
- 02Name the form your available training data actually takes.
- 03The larger the gap between the two, the more bridging data you must synthesize — and the further off the result is.
- 04A perfect input-output modality match is a massive, often invisible, structural advantage.
How to run it
- 1
State the desired output
Write down precisely what the model must produce — words, images, robot actions in 3D space, etc. This is the objective function.
Pro tip Be specific about modality and dimensionality; 'actions in 3D worlds' is very different from '2D video frames'.
- 2
Inventory the data you can actually get
Describe the form of the training data available at scale — not the data you wish existed.
Watch out Do not count data you cannot realistically acquire; 'there is web data' is not the same as 'web data of the right shape'.
- 3
Measure the gap
Compare output form to data form. Same modality means a clean path; a mismatch means the data lacks the very thing your output requires.
- 4
Plan the bridge or reset expectations
If aligned, scale data/compute. If misaligned, budget for supplementary data (teleoperation, synthetic environments) — or lower your timeline expectations accordingly.
Pro tip World-modeling and simulation can generate the missing action/3D data to close the gap.
In the wild
Language models train on words that become tokens and output words — a perfect objective-to-data match, so they scaled quickly. Robots must output actions in 3D worlds, but web-video training data 'lacks actions in 3D worlds', so teams must fit a square peg in a round hole with teleoperation and synthetic data.
→ Explains why conversational AI matured years ahead of embodied AI despite similar compute.
Common mistakes
Assuming 'big data exists' solves the problem
Volume is irrelevant if the data is the wrong shape. Web video is abundant but contains no action labels, so it cannot directly teach a robot to act.
Is it for you?
Best for
AI/ML founders, PMs, and researchers deciding which capabilities are near-term feasible versus multi-year bets.
Not ideal for
Problems where labeled data already matches the output modality — the test just confirms what you know.
From the transcript
“So you have this perfect alignment between what you hope to get which we call objective function and what your training data looks like. But…”
“You hope to get actions out of robots. But your training data lacks actions in 3D worlds.”
From the episode
The Godmother of AI on jobs, robots, and why world models are next
Dr. Fei-Fei Li