The Golden Recipe of Modern AI
Every AI breakthrough sits on three legs: big data, the right neural architecture, and enough compute.
- Difficulty
- Easy
- Time to result
- ~weeks to results
- Steps
- 4
- Confidence
- 82%
A mental model for what any modern AI capability actually requires. Fei-Fei Li identifies the recurring trio behind both the 2012 ImageNet/AlexNet moment and the ChatGPT moment: large-scale data, a neural network architecture, and GPUs. Use it as a checklist to diagnose why an AI effort is or isn't working, or what is missing.
Origin
Li traces the birth of modern AI to 2012, when Geoff Hinton's Toronto team combined ImageNet's big data, a neural network, and two Nvidia GPUs to crack object recognition. She notes ChatGPT uses the same three ingredients at vastly larger scale — internet-scale text, a more complex architecture, and far more GPUs.
Core principles
- 01Data, architecture, and compute are complementary — a weakness in any one caps the whole system.
- 02The ingredients stayed constant across a decade; only their scale changed.
- 03Historically the field over-focused on models while starving them of data — data was the overlooked ingredient.
How to run it
- 1
Audit the data leg
Do you have large-scale, clean, well-labeled data matched to the task? This was the ingredient the field originally neglected.
Pro tip Curation and a good taxonomy matter as much as raw volume — ImageNet paired 15M images with a 22,000-concept taxonomy.
- 2
Audit the architecture leg
Is the model architecture suited to the data and objective? Neural networks remain the core, but architecture complexity has grown enormously.
- 3
Audit the compute leg
Do you have enough GPU capacity to train at the needed scale?
Watch out Under-provisioned compute silently caps what the other two legs can achieve.
- 4
Find and fix the weakest leg
Locate which ingredient is limiting results and invest there rather than assuming a bigger model alone will help.
In the wild
A Toronto team used ImageNet's big data plus a neural network plus two consumer Nvidia GPUs to make a huge leap on object recognition — the moment widely seen as the birth of modern deep learning.
→ Established the data + neural net + GPU trio as the template all subsequent breakthroughs followed.
Common mistakes
Chasing model cleverness while starving it of data
Li's field spent years perfecting models that 'don't have data to be trained on'. The breakthrough came from recognizing data as the missing, co-equal ingredient.
Is it for you?
Best for
Anyone scoping or debugging an applied AI project who needs a fast structural diagnosis of what's missing.
Not ideal for
Teams needing a novel algorithmic breakthrough — the recipe names ingredients, not the frontier research itself.
From the transcript
“that combination of the trio technology uh big data neuronet network and GPU was kind of the golden recipe for modern AI”
“these models don't have data to be trained on”
From the episode
The Godmother of AI on jobs, robots, and why world models are next
Dr. Fei-Fei Li