The popular fear is that models will stop improving because they've exhausted the internet. Karina argues pre-training teaches a model to compress knowledge and model the world, but the real scaling now happens in post-training via reinforcement learning. Because you can teach a model an effectively infinite number of tasks, there is no data wall.

Pre-training is really about learning to compress knowledge and model the world
Post-training via reinforcement learning is not hitting a wall
There is an effectively infinite supply of tasks to teach the model

“we we went from like raw data sets from from pre-trained models to infinite amount of tasks that you can teach the model in the…”
Karina Nguyen · 10:00

“there's no data wall or whatever because there will be infinite amount of tasks. And that's how the model becomes extremely super intelligent.”
Karina Nguyen · 10:30

#synthetic-data#reinforcement-learning#post-training#scaling

Hot Take· 3

Hot Take10:30

The Real Bottleneck Now Is Evals, Not Data

Karina points out that models are saturating existing benchmarks — PhD-level tests like GPQA are being solved at human-expert levels. The limiting factor for progress is no longer data or intelligence, but the lack of hard enough evaluations to measure the frontier.

Models are saturating nearly all existing benchmarks
GPQA (PhD-level Q&A) is reaching the ~60-70% humans score
The frontier bottleneck is building good evaluations

“we are actually getting saturated on all benchmarks. So, I think the bottom neck is actually in evaluations.”
Karina Nguyen · 10:30

#evals#benchmarks#ai-research

Hot Take42:00

Soft Skills, Not Hard Skills, Are the Future of Work

Karina's core thesis: models are getting great at the 'hard' skills — coding, synthesis, writing — but struggle with aesthetics, taste, and genuine creativity. The durable human advantages are creative thinking, listening, prioritization and management. She even argues AI research progress itself is bottlenecked by management and compute allocation.

Teaching a model aesthetics, visual design and real creativity is very hard
Creative thinking, listening and prioritization become more valuable
AI research progress is bottlenecked by management and compute allocation
Hard skills (coding, design, writing) are the ones AI is taking over

“I think it's actually a really, really hard to teach the model how to be aesthetic or like do like visual or really good like…”
Karina Nguyen · 46:00

“AI research progress is bottlenecked by like management like research managers because you have like constrained set of compute”
Karina Nguyen · 46:00

#soft-skills#future-of-work#management#creativity

Hot Take49:30

AI Will Be Incredibly Good at Strategy

Lenny argues strategy is really just taking all available inputs, understanding the world, and forming a plan to win — exactly what a smart LLM should excel at. Karina agrees: models are strongest at connecting the dots across user feedback, dashboards and other inputs to co-create plans and recommendations, and self-improvement isn't far off.

Strategy is synthesizing all inputs into a plan to win
Models excel at connecting the dots across disparate sources
AI can aggregate feedback and dashboards to co-create plans and recommendations
Self-improving product development loops aren't far away

“isn't strategy just take all the inputs, all the data you have available, understand the world around you and come up with a plan to…”
Lenny Rachitsky · 49:30

“what models are really good at is like um like connecting the dots”
Karina Nguyen · 51:00

#strategy#ai-capabilities#data-analysis#self-improvement

Explainer· 4

Explainer06:30

Why Model Training Is More Art Than Science

Karina explains that the biggest misconception about how models are built is that it's a precise science. In practice, ensuring the highest-quality data for a desired behavior is one of the most important and least deterministic parts of the job. Debugging a model, though, looks a lot like debugging software.

Data quality is one of the most important levers in model training
Training is closer to a craft than a repeatable science
Debugging model behavior resembles debugging software

“Model training is more an art than a science and in a lot of ways”
Karina Nguyen · 06:30

“the way you debug models is actually very similar the way you debug software.”
Karina Nguyen · 06:30

#model-training#data-quality#ai-research

Explainer38:00

The Cost of Intelligence Is Collapsing

Karina names a core trend: the cost of reasoning and intelligence is dropping fast, and small models are becoming smarter than older large ones thanks to distillation. She saw this first-hand with Claude 3 Haiku, which was smarter than the much larger Claude 2 — pointing to a world of cheap, fast, highly capable small models.

The cost of reasoning and intelligence is dropping sharply
Distillation makes small models smarter than older large models
Claude 3 Haiku was smarter than the much bigger Claude 2

“the cost of reasoning and intelligence is drastically going down.”
Karina Nguyen · 38:00

“Smart small models are becoming even smarter than like large models and that's because of like the distillation research.”
Karina Nguyen · 38:30

#distillation#small-models#cost-of-intelligence#claude

Explainer39:00

AI Will Unblock Work That Was Bottlenecked by Intelligence

As intelligence gets cheap and abundant, Karina argues work that was previously limited by access to expertise gets unblocked — healthcare, education, and scientific research. Lenny cites a New York Times study where ChatGPT alone outperformed both doctors and doctors using ChatGPT.

Cheaper AI gives builders and developers far more access to intelligence
Feeding symptoms to ChatGPT can approximate access to a doctor
A NYT study found ChatGPT alone beat both doctors and doctors-with-ChatGPT
Education and scientific research see massive implications

“all the work that has been like bottlenecked by the intelligence will be kind of like unblocked.”
Karina Nguyen · 39:00

“there's a New York Times story about that where they compared doctors to doctors using ChatGPT to just ChatGPT and just just ChatGPT was the…”
Lenny Rachitsky · 40:00

#healthcare#education#ai-access#automation

Explainer48:00

Why Models Still Can't Do Taste

Karina explains that idea generation and discriminating good visual design or art are areas where models haven't learned enough from people to judge well. The deeper reason: there aren't many people with exceptional taste whose judgment is accessible to models to learn from. Lenny sums it up: 'there's not enough of you yet.'

Models haven't learned enough human examples to discriminate good design/art
New reasoning paradigms may improve writing over time
The scarcity of accessible taste-makers to learn from is the core limit

“when it comes down to like idea generation or like um discriminating of like what is a good like visual design and art? I feel…”
Karina Nguyen · 48:30

“Basically, there's not enough of you yet.”
Lenny Rachitsky · 49:00

#taste#creativity#visual-design#model-limits

Story· 3

Story07:00

The Model That Got Confused About Having No Body

During Claude 3 training, the team taught the model self-knowledge (it has no physical body) while also teaching it function calls like setting an alarm. The conflicting signals confused the model about whether it could actually perform physical-world actions, sometimes causing it to over-refuse. It illustrates the constant trade-off between being helpful and being safe.

Self-knowledge data ('you have no body') collided with function-call data ('this is how you set an alarm')
The confusion sometimes made the model over-refuse requests
Training balances helpfulness against harm across diverse scenarios

“the model would get like extremely confused uh about like whether it it can set an alarm in the but it doesn't have a body…”
Karina Nguyen · 07:30

#claude-3#model-behavior#anthropic#refusals

Story13:00

How Canvas Broke the Wall Between Research and Engineering

Karina describes pitching Canvas at OpenAI and watching a cross-functional team form almost out of nothing. It was, in her telling, the first OpenAI project where researchers and applied engineers worked together from the very start of the product cycle — moving ChatGPT from a chatbot toward a genuine collaborator.

Canvas turned ChatGPT from a chatbot into a collaborative agent
A team of engineers, designers, PMs and researchers formed rapidly
It was OpenAI's first product built with research + engineering together from the beginning

“So, going from like being a chatbot to more of a collaborative agent and a collaborator”
Karina Nguyen · 13:00

“Canvas is like one of the I would say like the first project at OpenAI where researchers and applied engineers started working together from the…”
Karina Nguyen · 13:30

#canvas#openai#product-development#collaboration

Story57:00

Claude in Slack: One of the First Tool-Using Products

Karina recalls early Anthropic days when Claude in Slack was one of the first tool-using products — Claude could operate in your workplace, summarize threads on demand, and every Monday auto-summarize entire channels. It created a social, Midjourney-on-Discord-like element where people learned to prompt together, before being sunsetted post-ChatGPT.

Claude in Slack was one of the first tool-using products
It could summarize threads and, weekly, whole channels automatically
It created a social, community learning element around prompting
It was sunsetted around 2023 as focus shifted to consumer/enterprise

“Claude and Slack was actually one of the first like tool using like products.”
Karina Nguyen · 57:30

“every Monday Claude would just like summarize the entire channel.”
Karina Nguyen · 58:00

#claude#slack#anthropic#agents

Q&A· 1

Q&A53:30

Anthropic vs OpenAI: Craft vs Risk-Taking

Having worked at both labs, Karina says they're more similar than different but with real cultural nuances. Anthropic taught her deep care and craft around moral training and hardcore prioritization; OpenAI is more innovative, risk-taking and bottoms-up. She frames Claude's 'librarian' personality as a reflection of its creators.

Anthropic emphasizes moral craft and hardcore prioritization
OpenAI is more innovative, risk-taking and bottoms-up
A model's personality reflects its creators (Claude as a 'librarian')
The two labs are one big community, not enemies

“I think what I have learned from Anthropic is this like real care and craft towards like moral behavior, moral craft, moral training.”
Karina Nguyen · 54:00

“OpenAI is like much more innovative and much more like risk takers in terms of like product or like research”
Karina Nguyen · 56:00

#anthropic#openai#company-culture#claude

Tool· 1

Tool67:30

Operator: An Agent With Its Own Virtual Computer

Karina describes Operator, an OpenAI agent that completes tasks inside its own virtual computer — e.g. ordering a book on Amazon by driving a browser itself. She explains why this is genuinely hard: the models operate on pixels rather than language, and visual perception is much harder to scale than text.

Operator completes tasks inside its own virtual environment/browser
It can carry out real tasks like buying a book on Amazon
It's hard because agents act on pixels, not language
Deriving human intent and knowing when to ask follow-ups is a key challenge

“it's basically an agent that can complete the task in its own like virtual computer like in its own virtual environment.”
Karina Nguyen · 67:30

“Much of it is like because right now the models it operating on like pixels instead of like language or what not.”
Karina Nguyen · 70:00

#operator#computer-use#agents#openai

Takeaway· 2

Takeaway24:30

Prompting Is the New Way to Prototype Products

While building the 100K-context file-upload feature at Anthropic, Karina prototyped it just by prompting the model in a local browser — and people immediately wanted the API. That's when it clicked that prompting is a genuine product-development and prototyping tool for designers and PMs, not just a way to chat.

The 100K-context file upload was prototyped purely by prompting in a browser
Prompting lets designers and PMs prototype features without heavy engineering
Small 'micro experiences' (like personalized starter prompts and titles) can be built via prompting

“prompting is like a new way of like product development and like prototyping for designers and for like product managers.”
Karina Nguyen · 24:30

#prompting#prototyping#product-development#anthropic

Takeaway43:30

Build for the Model You'll Have, Not the One You Have

Karina's advice for builders: design product ideas for where the models are going, not where they are today. It doesn't matter if the model is good enough right now — build so that by the time models get really good, your product works really well. Claude artifacts and early Canvas are examples of ideas that landed as capability caught up.

Design for future model capability, not current limitations
A product built ahead of the curve pays off as models improve
Fast-iterating startups win by inventing new ways to train and listening to users

“you kind of want to like build for the future. So it's like it doesn't necessarily matter whether the model is good or not good…”
Karina Nguyen · 43:30

#product-strategy#building#future-planning

Frameworks in this episode

Build for the Future Model

Core-Behavior Synthetic Training

Eval-First AI Product Development

Prompting-as-Prototyping

Insights & moments

⚡Myth Buster· 1

Why There Is No 'Data Wall'

◆Hot Take· 3