“everyone's been pointing to these academic evals of PhD level reasoning with GPQA, humanities last exam or Olympiad math.”
← All resourcespaper
GPQA
1 recommend/use · 3 sourced episodes
Every sourced reference
Short attributed excerpts only. Timestamps are approximate.
“A great example is a public paper called like GPQA.”
“GPQA, which is like a Google proof question answering like PhD level.”
Related frameworks
Strategy3 steps
Build for the Future Model
Design product ideas for the model capability that is coming, not the one you have today
Read →
Peak Performance3 steps
Build-With-The-Tools Assessment
Don't test people on doing work without AI — hand them the tools and score what they can build in an hour
Read →
Innovation5 steps
Core-Behavior Synthetic Training
Ship an AI feature by naming its 3-4 core behaviors and teaching them with model-generated data
Read →
Strategy3 steps
Elastic-Demand Career Bet
Invest in domains where making people 10x more productive increases demand rather than reducing it
Read →
Productivity4 steps
Eval-First AI Product Development
Define what 'correct' looks like as evals first; the spec becomes the scorecard, not the build instructions
Read →
Leadership3 steps
Patient-Then-Floor-It Hiring
Be obsessively patient hiring the founding team; step on the gas the moment demand exceeds what you can handle
Read →
Innovation3 steps
Prompting-as-Prototyping
Validate a product idea by prompting a model in a local browser before building anything
Read →
Entrepreneurship4 steps
Pull-Based Product-Market Fit
Be stubborn on your thesis but open on its form — chase the customer who is surprisingly easy to sell, not the one you must force
Read →
Innovation3 steps
RLAIF Reward Design
Have an expert define success criteria and a rubric once, then let AI reinforce the capability — more scalable than labeling examples
Read →
Strategy3 steps
The Access-to-an-Audience Moat
In human data the only durable moat is a trusted, targetable audience — not recruiters or ads
Read →
Entrepreneurship5 steps
The Company-Inside-a-Company Incubation Playbook
Spin a 0-to-1 business inside a mature company by ring-fencing it completely from the core
Read →
Innovation4 steps
The Middleman-Signal Disintermediation Play
When middlemen and end-customers both come to you directly, that's the signal to build the business yourself
Read →
Strategy3 steps
The Quality-Volume-Speed Priority Stack
What high-stakes data buyers actually rank: quality first, then volume, then speed
Read →
Leadership3 steps
Three Tenets: Can-Do, High Standards, Intensity
The three cultural tenets Foody credits for the fastest revenue ascent in history
Read →
Innovation3 steps
Value-Chain Eval
Before applying AI to your business, build a systematic test that measures how well it automates your core value chain
Read →
People in these episodes
Related resources
- AlphaSights
- Amazon
- Anthropic
- Apple Podcasts
- Atlassian
- Benchmark
- Character.AI
- ChatGPT
- Claude
- Claude Artifacts
- Claude Code
- CodeRabbit
- Codex
- Conductor
- Cursor
- Discord
- Dropbox
- ElevenLabs
- Enterpret
- Game of Thrones
- General Catalyst
- GLG
- Goldman Sachs
- Google Sheets
- Handshake
- High Output Management
- How to Start a Startup
- Jira
- Jira Product Discovery
- Lenny's Newsletter
- Lenny's Podcast
- Lenny's Podcast website
- Lennybot
- Loom
- Massachusetts Institute of Technology
- McKinsey & Company
- Meta
- Midjourney
- Neuralink
- No Priors
- OpenAI
- OpenAI o1
- Operator
- Oppenheimer
- Palantir
- Princeton University
- Scale AI
- Shoe Dog
- Slack
- SNOO
- Spotify
- Square
- Stanford University
- Suits
- Surge AI
- SWE-bench
- Tesla
- The Hard Thing About Hard Things
- The New York Times
- Tuple
- Uber
- University of California, Berkeley
- Vanta
- Visual Studio Code
- Warrant
- Westworld
- Wisdom
- WorkOS
- xAI
- Zanzibar
- Zero to One
Spot an error or want this page removed? Request a correction or removal.