Simon Willison02 April 2026

An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines

7Frameworks

15Insights

Episode overview

Lenny Rachitsky talks with Simon Willison about the late-2025 coding-model inflection point and why reliable coding agents are rapidly reshaping software work. Willison distinguishes low-stakes vibe coding from professional agentic engineering, explores dark software factories and AI-driven testing, and explains practices such as red/green test-driven development, reusable project templates, and retaining prior experiments as agent context. They also examine burnout, changing career leverage, prompt injection’s “lethal trifecta,” and why a secure version of OpenClaw remains a major unsolved opportunity.

Key ideas

GPT-5.1 and Claude Opus 4.5 crossed a practical reliability threshold in November 2025, making coding agents useful for substantial end-to-end work.
Vibe coding is appropriate for low-risk prototypes, while agentic engineering applies professional review, testing, and quality standards to AI-generated production software.
Dark software factories aim to produce trustworthy software without humans writing or reading code, using automated validation such as persistent swarms of simulated users.
As code generation becomes cheap, product discovery, evaluation, security, and human cognitive capacity become more important bottlenecks.
Experienced engineers can amplify accumulated judgment with agents, while mid-career workers may face greater disruption; agency and ambitious experimentation are key adaptive skills.
Red/green test-driven development and thin starter templates give coding agents executable feedback and patterns they can reliably extend.
Reusable repositories of working tools and code-backed research let agents combine proven techniques across future projects.
Prompt injection becomes especially dangerous when an agent combines private-data access, exposure to malicious instructions, and an exfiltration channel; breaking one leg limits the blast radius.

Transcript available · source text is retained privately and is not published

Frameworks in this episode

Productivity4 steps

Hoard What You Know How To Do

Keep a searchable backlog of verified, working experiments so agents can recombine them into new solutions.

Read →

Productivity4 steps

Red/Green TDD for Coding Agents

Force agents to write the test first, watch it fail, then pass — the compressed prompt is just 'red/green TDD'.

Read →

Innovation4 steps

The Dark Factory Software Model

Ship production software no human writes or reads — replace review with simulated-user QA swarms.

Read →

Strategy4 steps

The Lethal Trifecta

Any AI agent that combines three capabilities can be tricked into stealing your data — cut one leg.

Read →

Productivity3 steps

The Thin Skeleton Template

Start every project from a minimal template — agents copy its style far better than they follow prose instructions.

Read →

Mindset4 steps

The Vibe-Coding Blast-Radius Rule

Vibe-code freely when only you get hurt by bugs; stop and take responsibility the moment others could.

Read →

Innovation3 steps

Three-Prototype Ideation

Since prototypes are now free, build a feature three ways and let real use pick the winner.

Read →

People & resources mentioned

Attributed to the moment in the episode. Timestamps are approximate.

People · 8

Simon WillisonMentionsHe co-created Django, the web framework that powers Instagram, Pinterest, Spotify, and thousands of other platforms.
“Today, my guest is Simon Wilson.”
▶ Watch at ~1:00approx.
Lenny RachitskyMentionstoday we've got another very special compilation episode something I've been pulling on more and more with the podcast and the newsletter
“Today, my guest is Simon Wilson.”
▶ Watch at ~1:00approx.
Simon WillisonCoinedHe co-created Django, the web framework that powers Instagram, Pinterest, Spotify, and thousands of other platforms.
“And amongst his hundred plus open source projects, he created Datasette, a data analysis tool that has become a staple of investigative journalism.”
▶ Watch at ~1:30approx.
Simon WillisonCoinedHe co-created Django, the web framework that powers Instagram, Pinterest, Spotify, and thousands of other platforms.
“He coined the term prompt injection, popularized the ideas of AI slop and agentic engineering.”
▶ Watch at ~1:30approx.
Boris ChernyMentionsShe oversees both Boris Cherney and Kat Wu, both of whom who have been on the podcast
“Yeah, I had Boris Turning on the podcast and he's doing the same thing.”
▶ Watch at ~8:00approx.
David PlacekMentionsDavid is the founder of Lexicon branding which pioneered the field of brand naming
“That reminds me of a chat I had with David Place like he's a expert naming person.”
▶ Watch at ~25:30approx.
Sander SchulhoffMentionsI'm an artificial intelligence researcher. I've been doing AI research for the last probably like seven years now
“the last episode we had on this with Sander Schulhoff, he does professional red teaming where they test models”
▶ Watch at ~1:26:30approx.
Alfred MolinaMentionsmentioned
“if you Alfred Molina, legendary Spider-Man villain, I I like that.”
▶ Watch at ~1:34:00approx.

Resources · 47

DjangoCoinedsoftware · Simon Willison and collaborators
“He co-created Django, the web framework that powers Instagram, Pinterest, Spotify, and thousands of other platforms.”
▶ Watch at ~1:30approx.
DatasetteCoinedsoftware · Simon Willison
“he created Datasette, a data analysis tool that has become a staple of investigative journalism.”
▶ Watch at ~1:30approx.
Simon Willison's WeblogRecommendswebsite · Simon Willison
“And as he's leaned into this new way of building, he's been sharing everything he's learning in real time. There's incredible blog.simonwillson.net.”
▶ Watch at ~2:00approx.
Lenny's NewsletterMentionsnewsletter · Lenny Rachitsky
“available exclusively to Lenny's newsletter subscribers.”
▶ Watch at ~2:30approx.
Lenny's Product Hunt PassRecommendswebsite · Lenny Rachitsky
“Don't forget to check out Lenny's Product Hunt Pass.com for an incredible set of deals available exclusively to Lenny's newsletter subscribers.”
▶ Watch at ~2:30approx.
AnthropicMentionscompany
“Both Anthropic and OpenAI spent the whole of 2025 focusing all of their training efforts on coding.”
▶ Watch at ~3:30approx.
OpenAIMentionscompany
“Both Anthropic and OpenAI spent the whole of 2025 focusing all of their training efforts on coding.”
▶ Watch at ~3:30approx.
OpenAI o1Mentionssoftware · OpenAI
“Like OpenAI's O1 was the first model to exhibit that.”
▶ Watch at ~4:00approx.
GPT-5.1Mentionssoftware · OpenAI
“in November we had what I call the inflection point where GPT-5.1 and Claude Opus 4.5 came along.”
▶ Watch at ~4:30approx.
Claude Opus 4.5Mentionssoftware · Anthropic
“in November we had what I call the inflection point where GPT-5.1 and Claude Opus 4.5 came along.”
▶ Watch at ~4:30approx.
StrongDMMentionscompany
“There's a company called StrongDM has been pushing this and doing some really interesting experiments around this.”
▶ Watch at ~13:30approx.
Mozilla FirefoxMentionssoftware · Mozilla
“Anthropic had discovered a hundred like potential vulnerabilities in Firefox and responsibly reported them to Mozilla, who then fixed them.”
▶ Watch at ~19:30approx.
Claude Opus 4.6Mentionssoftware · Anthropic
“Opus 4.6 still can't do this particular thing”
▶ Watch at ~28:30approx.
ThoughtworksMentionscompany
“ThoughtWorks, um the big um like a IT consultancy, did a offsite a few about a month ago”
▶ Watch at ~29:30approx.
CloudflareMentionscompany
“Cloudflare and Shopify both said they were hiring a thousand interns over the course of 2025”
▶ Watch at ~29:30approx.
ShopifyMentionscompany · Shopify
“Cloudflare and Shopify both said they were hiring a thousand interns over the course of 2025”
▶ Watch at ~29:30approx.
ChatGPTUsessoftware · OpenAI
“I've been using AppleScript for like two and a half years now because ChatGPT knows AppleScript”
▶ Watch at ~32:00approx.
AppleScriptUsessoftware · Apple
“I never used AppleScript because AppleScript is a whole programming language you have to learn.”
▶ Watch at ~32:00approx.
ClaudeUsessoftware · Anthropic
“I've been using Claude, it turns out, excellent chef”
▶ Watch at ~32:30approx.
NetflixMentionscompany · Netflix
“It's supposed to let us sit around and watch Netflix and do all the create wealth and productivity in the world.”
▶ Watch at ~35:00approx.
LinearMentionssoftware · Linear
“I was talking to the founder of Linear the other day and this idea of the factory.”
▶ Watch at ~37:00approx.
VantaRecommendssoftware · Christina Cacioppo
“Learn more at vanta.com/lenny. And as a listener of this podcast, you get $1,000 off Vanta.”
▶ Watch at ~48:00approx.
Claude CodeUsessoftware · Anthropic
“I do a huge amount of work using Claude code”
▶ Watch at ~48:30approx.
OpenAI CodexUsessoftware · OpenAI
“So I've been leaning on GPT 5.4 a lot more this month. Um and OpenAI Codex”
▶ Watch at ~50:30approx.
GPT-5.4Usessoftware · OpenAI
“So I've been leaning on GPT 5.4 a lot more this month.”
▶ Watch at ~50:30approx.
Nano BananaUsessoftware · Google
“for image generation, I'm using Gemini because of Nano Banana”
▶ Watch at ~54:30approx.
GPT-5.4 miniMentionssoftware · OpenAI
“The other day, OpenAI released GPT 5.4 mini and nano at five different thinking levels”
▶ Watch at ~56:30approx.
GPT-5.4 nanoMentionssoftware · OpenAI
“The other day, OpenAI released GPT 5.4 mini and nano at five different thinking levels”
▶ Watch at ~56:30approx.
Gemini 3.1Mentionssoftware · Google
“Gemini 3.1 just a few weeks ago, they had a video which featured a pelican riding a bicycle like animated.”
▶ Watch at ~57:30approx.
simonw/researchCoinedsoftware · Simon Willison
“Then the other one I have is simonw/research on GitHub, which are AI-driven research projects.”
▶ Watch at ~1:03:00approx.
GitHubUsessoftware · GitHub
“I use GitHub as a backup system.”
▶ Watch at ~1:05:00approx.
DropboxRecommendscompany · Drew Houston and Arash Ferdowsi
“So, the easiest one would be like a folder synced to Dropbox or something like that.”
▶ Watch at ~1:05:30approx.
Tesseract OCRUsessoftware
“I'd also written some code that used Tesseract, which is an OCR library that can run in your browser”
▶ Watch at ~1:07:00approx.
The Challenger Launch Decision: Risky Technology, Culture, and Deviance at NASAMentionsbook · Diane Vaughan
“there's this fantastic paper that came out of the the Space Shuttle Challenger disaster called the normalization of deviance.”
▶ Watch at ~1:23:30approx.
Space Shuttle Challenger disasterMentionsevent
“The reason I think about the Challenger disaster is there's this fantastic paper that came out of the the Space Shuttle Challenger disaster”
▶ Watch at ~1:23:30approx.
Defeating Prompt Injections by DesignMentionspaper · Google DeepMind
“There was a paper that Google DeepMind put out a couple of years ago, the the camel paper”
▶ Watch at ~1:26:30approx.
Google DeepMindCoinedcompany · Google
“There was a paper that Google DeepMind put out a couple of years ago, the the camel paper”
▶ Watch at ~1:26:30approx.
AI.comMentionswebsite
“in the Super Bowl, there was an ad for AI.com, which was effectively a vaporware white-labeled open claw hosting provider.”
▶ Watch at ~1:28:30approx.
Super BowlMentionsevent
“in the Super Bowl, there was an ad for AI.com”
▶ Watch at ~1:28:30approx.
OpenClawMentionssoftware · Peter Steinberger
“Open Claw demonstrates that people want a personal digital assistant so much that they are willing to not just overlook the security side of things”
▶ Watch at ~1:29:30approx.
DockerUsessoftware · Docker
“I don't run it myself outside of a Docker container where I set it up to safely poke it and see what it could do.”
▶ Watch at ~1:31:00approx.
OpenClawUsessoftware · Peter Steinberger
“I don't run it myself outside of a Docker container where I set it up to safely poke it and see what it could do.”
▶ Watch at ~1:31:00approx.
Mac miniUsesproduct · Apple
“I got one running right here on my Mac mini.”
▶ Watch at ~1:31:30approx.
Spider-Man 2Mentionsfilm · Sam Raimi
“The other thing I love about the name claw is there's a Spider-Man 2 reference, right?”
▶ Watch at ~1:33:30approx.
New ZealandMentionsplace
“There is a rare parrot in New Zealand called the kakapo parrot.”
▶ Watch at ~1:38:00approx.
Kākāpō Recovery ProgrammeRecommendsother
“there are webcams where you can watch them sitting on their nests. It's a really really good time”
▶ Watch at ~1:38:30approx.
Lenny's PodcastRecommendspodcast · Lenny Rachitsky
“You can find all past episodes or learn more about the show at lennyspodcast.com.”
▶ Watch at ~1:39:30approx.

Spot an error or want something removed? Request a correction or removal.

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Myth Buster· 2

Myth Buster08:30

When Vibe Coding Is Fine and When It's Irresponsible

Willison distinguishes vibe coding (not looking at, caring about, or understanding the code) from agentic engineering (professionals shipping reviewed, production-ready code with coding agents). He loves that non-programmers can now build little tools, but draws a hard line: vibe code for yourself all you want, but the moment your bugs could harm someone else you need to step back. He argues calling all AI-assisted work 'vibe coding' devalues a useful term.

Vibe coding = you don't look at, understand, or care about the code, per Karpathy's original definition
Safe to vibe code anything where you're the only person hurt by bugs
Once other people rely on it (e.g. scraping, security-adjacent code), vibe coding is irresponsible
Knowing what's responsible is itself an expert-level skill
He prefers 'agentic engineering' for professional agent-driven work to keep 'vibe coding' meaningful

“if you're vibe coding something for yourself where the only person who gets hurt if it has bugs is you,”
Simon Willison · 09:30

#vibe coding#agentic engineering#responsibility#prototyping

Myth Buster1:22:00

Why 97% Effective Prompt-Injection Filters Are a Failing Grade

Willison rejects the instinct that you can just tell the AI not to fall for tricks. Filters top out around 97% effectiveness, which he calls a failing grade, because 3 in 100 attacks still steal all your data. Since prompts can be written in any human language, it's an unwinnable deny-list problem: you can always invent a new sequence of characters to slip through. The real defense is limiting blast radius, not perfecting filters.

You can't just instruct the model to refuse trickery; attackers route around it
Filters reach ~97% effectiveness, which still lets 3% of attacks steal everything
It's a deny-list problem: any new sequence of characters can trick the model
You can filter 'ignore previous instructions' in English but not every language
The real defense is limiting the blast radius of what a compromised agent can do

“you can get to like 97% effectiveness on those filters. I think that's a failing grade.”
Simon Willison · 1:22:00

#prompt injection#security#filters#guardrails

Hot Take· 5

Hot Take29:30

AI Comes for the Middle: Mid-Career Engineers Are Most at Risk

Citing a ThoughtWorks offsite of engineering VPs, Willison says AI amplifies experienced engineers and dramatically speeds up onboarding for brand-new ones (Cloudflare and Shopify each hired ~1,000 interns in 2025). The group in trouble is mid-career engineers who lack deep expertise to amplify but no longer get the beginner onboarding boost. He notes AI keeps attacking 'the middle' of many things at once.

Experienced engineers get their skills amplified by agents
New engineers onboard far faster; interns are useful in a week instead of a month
Cloudflare and Shopify each said they hired ~1,000 interns over 2025
Mid-career engineers are the group ThoughtWorks judged most at risk
AI is hitting the 'middle' of the product process and of seniority alike

“The problem is the people in the middle.”
Simon Willison · 30:00

#careers#junior engineers#hiring#seniority

Hot Take26:30

The Productivity Paradox: The Most AI-Pilled People Are the Most Exhausted

Willison describes firing up four agents in parallel and being cognitively wiped out by 11 a.m., because there's a hard limit on how much a human can hold in their head at once. He warns of a gambling/addiction dynamic where people wake at 4 a.m. to set off more agent runs. AI gives him more time but leaves his brain exhausted, and he worries good management needs to protect its best people from burnout.

Running four agents in parallel is mentally exhausting, wiping him out by 11 a.m.
Human cognition caps how much you can hold in your head, and it's easy to 'pop that stack'
There's a gambling/addiction feel to it; people lose sleep launching extra agent runs
AI grants more free time but leaves the brain fried from work intensity
Good companies must avoid burning out their best employees for short-term output

“I can fire up like four agents in parallel and have them work on four different problems and by like 11:00 a.m., I am wiped…”
Simon Willison · 26:30

#burnout#productivity#cognition#working habits

Hot Take52:30

Why Willison Turns Off AI Memory Features

Memory is meant to be the stickiest feature the labs offer, but Willison hates it and turns it off wherever he can, because as an AI researcher he needs to see what everyone else sees when prompting, not results skewed by his prior conversations. He recounts how, during the OpenAI-military controversy, Anthropic's onboarding let users export ChatGPT memories with a single prompt and paste them into Claude, an amusingly easy migration of something previously thought hard to extract.

Memory is considered the stickiest feature the labs are betting on
He disables it so he sees the same baseline output everyone else does when prompting
Personalized memory can make something look like it 'works' only for him
Anthropic's onboarding exported ChatGPT memories via a single copy-paste prompt
A whole migration between assistants happened just by prompting for the stored info

“and I hate those features and I turn them off wherever I can”
Simon Willison · 52:30

“a a whole export like move from one to the other just by prompting it to to give you the information you needed.”
Simon Willison · 53:30

#memory#personalization#anthropic#openai

Hot Take1:23:30

The Coming 'Challenger Disaster of AI' and Normalization of Deviance

Willison invokes the Space Shuttle Challenger disaster and the 'normalization of deviance' paper: engineers knew the O-rings were unreliable, but every successful launch made them more institutionally confident. He argues AI is doing the same with prompt injection, using systems in increasingly unsafe ways because no headline-grabbing million-dollar breach has happened yet. He predicts a big disaster will eventually force the field to change, while candidly admitting he's made this prediction every six months for three years and been wrong.

The 'normalization of deviance' paper explains how repeated safe launches bred false confidence
Each Challenger launch that survived faulty O-rings increased institutional overconfidence
AI is normalizing deviance around prompt injection because no huge breach has hit the news
He predicts a 'Challenger disaster of AI' that will eventually force a reckoning
He's made this prediction every 6 months for 3 years and it hasn't happened yet

“every single time you get away with launching a space shuttle without the O-rings failing, you institutionally feel more confident in what you're doing.”
Simon Willison · 1:24:00

“my prediction is that we're going to see a Challenger disaster.”
Simon Willison · 1:24:30

#challenger disaster#normalization of deviance#security#predictions

Hot Take1:28:30

Open Claw Is a Tamagotchi, and 'Safe Open Claw' Is the Biggest Opportunity in AI

Willison marvels that Open Claw went from first line of code on November 25th to a Super Bowl ad in about three and a half months, despite being exactly the insecure personal-assistant product he argues against, with people losing Bitcoin wallets. Its runaway success proves enormous demand for a personal digital assistant that the big labs wouldn't build over security fears. He calls building a safe version, one that won't leak data or delete files, the biggest opportunity in AI, and relays a friend's framing of it as a Tamagotchi living in a Mac mini 'aquarium.'

Open Claw went from first code (Nov 25) to a Super Bowl ad in ~3.5 months
It's the insecure personal-assistant product Willison most argues against; users lost Bitcoin wallets
Its success proves huge demand the big labs wouldn't touch over security concerns
A safe Open Claw that won't leak data or delete files is the biggest opportunity in AI
A friend calls it a Tamagotchi, with the Mac mini as the aquarium the digital pet lives in

“Open Claw is basically it's a it's a Tamagotchi, right? It's a digital pet and you buy the Mac mini as an aquarium.”
Simon Willison · 1:31:30

“if you can deploy a version of Open Claw that does all the things people love about it and won't randomly leak people's data and…”
Simon Willison · 1:30:30

#open claw#personal assistants#security#opportunity

Explainer· 2

Explainer04:30

The November Inflection Point: When Coding Agents Crossed a Threshold

Simon Willison argues that both Anthropic and OpenAI spent all of 2025 pointing their reinforcement-learning and reasoning efforts at code, because code is verifiably right or wrong. In November, GPT-5.1 and Claude Opus 4.5 crossed a threshold where agents went from 'mostly works but watch it closely' to 'almost always does what you told it to.' He frames software engineers as a bellwether for how AI will hit other knowledge work next.

All of 2025, Anthropic and OpenAI focused their training almost entirely on coding
Reasoning models (starting with OpenAI's o1 in late 2024) turned out to be great for code
November's GPT-5.1 and Claude Opus 4.5 were only incrementally better but crossed a usability threshold
Code is the easiest agent problem because it is obviously right or wrong when you run it
Software engineers are a bellwether for how agents will reshape other information work

“in November we had what I call the inflection point where GPT-5.1 and Claude Opus 4.5 came along.”
Simon Willison · 04:30

#ai coding#coding agents#inflection point#software engineering

Explainer1:20:30

The Lethal Trifecta: The Three Ingredients of an AI Data Breach

Willison explains the lethal trifecta, a subset of prompt injection: an agent becomes dangerous when it combines access to private information, exposure to malicious instructions, and a way to exfiltrate data back to an attacker. His email-assistant example shows how an incoming email can instruct the agent to forward private data. Because LLMs can't distinguish trusted instructions from pasted-in untrusted text, the only reliable fix is to cut off one of the three legs, usually exfiltration.

The trifecta: access to private data + exposure to malicious instructions + an exfiltration path
Example: an attacker emails your assistant and asks it to forward private sales projections
LLMs can't tell the difference between your instructions and text pasted from others
The only fix is to cut off one leg, most easily the exfiltration path
He named it so the term can't be guessed, unlike 'prompt injection'

“You have a lethal trifecta anytime your agent has three things.”
Simon Willison · 1:20:30

“The only way to fix it is to cut off one of those three legs.”
Simon Willison · 1:21:30

#lethal trifecta#prompt injection#security#exfiltration

Story· 3

Story12:30

The 'Dark Factory': StrongDM's $10k-a-Day Simulated QA Swarm

Willison describes visiting a demo from StrongDM, a security-access company experimenting with 'dark factory' software where nobody writes or reads the code. To test it, they ran a swarm of simulated employees in a simulated Slack channel, 24 hours a day, requesting access to Jira and Slack, spending around $10,000 a day on tokens. Crucially, they had agents build fake versions of Slack, Jira, and Okta from the public API docs so they could run 10,000 simulated users without hitting real rate limits.

The 'dark factory' idea: factories so automated you can turn the lights off, applied to software
StrongDM adopted policies of 'nobody writes code' and later 'nobody reads the code'
They simulated a QA department as a 24/7 swarm of agent 'employees' requesting access
Roughly $10,000/day in tokens to robustly test the software like a QA team that never sleeps
They had agents build their own simulations of Slack, Jira, and Okta from public API docs to avoid rate limits, running as a cheap Go binary

“if your factory is so automated that you don't need any people there, you can turn the lights off.”
Simon Willison · 13:00

“they were spending $10,000 a day on tokens, I think, simulating all of these end users.”
Simon Willison · 16:00

#dark factory#strongdm#qa#testing#agents

Story39:00

Why Labs Are Paying Big for Pre-2022 Human-Written Code

Lenny shares that data-labeling companies are buying old GitHub repos of handwritten code, paying a lot for 'artisanal' human-written code from before ChatGPT (around 2022). Willison connects it to low-background steel: metal salvaged from pre-nuclear-era shipwrecks that lacks the radiation baked into everything made after the first atomic tests. Human code from before the AI era is the software equivalent of uncontaminated source material.

Data-labeling companies buy old GitHub repos of handwritten code to train models
They pay a premium for 'artisanal' human-written code, targeting code from before ~2022
Willison likens it to low-background steel dug from pre-nuclear-explosion shipwrecks
Post-AI code is 'contaminated' the way post-atomic-test metal carries radiation

“Data labeling companies are buying old GitHub repos of handwritten code to train their models on and they're paying a lot of money for like…”
Lenny Rachitsky · 39:00

#training data#human code#low-background steel#github

Story55:00

The Pelican-on-a-Bicycle Benchmark and Why It Predicts Everything

Frustrated that numeric benchmarks didn't tell him anything, Willison invented a joke benchmark: ask a model to generate an SVG of a pelican riding a bicycle. It tests the text model (which outputs SVG code), not image models, and models are usually terrible at it because spatial reasoning via vectors is hard. Strangely, the quality of the pelican drawing correlates strongly with how good the model is at everything else, and it has become a meme the labs now compete on.

He started the benchmark to make fun of meaningless numeric benchmark scores
It's a test of text models outputting SVG code, not of image-generation models
Models are usually terrible because drawing via plotted vectors needs spatial reasoning
Pelican-drawing quality strongly correlates with overall model quality, and nobody can explain why
It became a meme; labs now relish showing off their pelicans (GPT 5.4 x-high drew the best)

“basically to make fun of the benchmarks, I started my own benchmark which was generate an SVG of a pelican riding a bicycle.”
Simon Willison · 55:30

“there appears to be a very strong correlation between how good their drawing of a pelican riding a bicycle is and how good they are…”
Simon Willison · 56:00

#benchmarks#svg#pelican#model evaluation

Tool· 1

Tool48:30

Simon Willison's AI Coding Stack: Claude Code for Web, YOLO Mode, GPT 5.4

Willison mostly uses Claude, and favors Claude Code for web (the hosted version accessible via the iPhone Claude app's code tab) over the local version. Running on Anthropic's servers lets him use 'dangerously skip permissions' / YOLO mode safely, so he can run four agents at once and step away, since the worst case is leaking his already-open-source code. He notes GPT 5.4 is now on par with Claude Opus 4.6 and cheaper, and uses models like Claude and ChatGPT for web research over Google.

Primarily Claude; prefers hosted Claude Code for web over the local install
The iPhone Claude app has a code tab that runs agents on Anthropic's servers
Hosted execution makes YOLO / 'dangerously skip permissions' mode safe to run
In safe mode he runs multiple agents and walks away; worst case is leaking open-source code
GPT 5.4 is on par with Claude Opus 4.6 and cheaper; he uses LLMs for parallel web search over Google

“there's Claude code for web, which is their hosted version of Claude code. And I use that one more than the one on my own…”
Simon Willison · 48:30

#tools#claude code#yolo mode#gpt-5.4#ai stack

Takeaway· 2

Takeaway37:30

Tests and Docs No Longer Signal Quality; You Need 'Proof of Usage'

Willison can now build a library with docs and tests in an hour that looks like weeks of work, but he doesn't believe in it because he rushed and, most importantly, hasn't actually used it. The signal that polished tests and documentation once gave (that software is good) is gone. His workaround is to label such software 'alpha' to signal he hasn't battle-tested it, and he and Lenny reframe the new signal as 'proof of usage.'

He can produce documented, tested software in an hour that looks like weeks of effort
He doesn't trust it because he rushed and hasn't used it in practice
When using others' software, he most wants to know they've run it for months
Tests + docs used to mean 'good'; that signal is now gone
He marks unused software 'alpha'; the new signal is 'proof of usage,' not proof of work

“So, I've got some very cool software that I built that I've never used.”
Simon Willison · 38:00

“if you see my software and it says it's an alpha, that probably means I haven't actually used it yet”
Simon Willison · 38:30

#software quality#trust signals#alpha software#craft

Takeaway31:30

Invest in Your Own Agency and Be More Ambitious

Asked what mid-career workers should do, Willison says to lean into the tools and use them to amplify your own skills rather than let them atrophy. He argues agents have no agency because they lack human motivations, so the durable human edge is deciding what problems to take on. His own New Year's resolution inverted years of 'focus more, do less' into taking on more and being more ambitious, since people underestimate what they can now do.

Lean into the tools and be mindful so your skills amplify rather than atrophy
Agents have no agency because they lack human motivation to choose what matters next
The durable human edge is agency: deciding which problems to take on
The only universal skill now is being able to roll with constant change
His resolution flipped from 'focus more' to taking on more and being more ambitious

“invest invest in your own agency and invest in how do I use this technology to get better at what I do and to do…”
Simon Willison · 33:30

#agency#career advice#ambition#mindset

Key ideas

Frameworks in this episode

Hoard What You Know How To Do

Red/Green TDD for Coding Agents

The Dark Factory Software Model

The Lethal Trifecta

The Thin Skeleton Template

The Vibe-Coding Blast-Radius Rule

Three-Prototype Ideation

People & resources mentioned

People · 8

Resources · 47

Insights & moments

⚡Myth Buster· 2

When Vibe Coding Is Fine and When It's Irresponsible

Why 97% Effective Prompt-Injection Filters Are a Failing Grade

◆Hot Take· 5

AI Comes for the Middle: Mid-Career Engineers Are Most at Risk

The Productivity Paradox: The Most AI-Pilled People Are the Most Exhausted

Why Willison Turns Off AI Memory Features

The Coming 'Challenger Disaster of AI' and Normalization of Deviance

Open Claw Is a Tamagotchi, and 'Safe Open Claw' Is the Biggest Opportunity in AI

✶Explainer· 2

The November Inflection Point: When Coding Agents Crossed a Threshold

The Lethal Trifecta: The Three Ingredients of an AI Data Breach

❝Story· 3

The 'Dark Factory': StrongDM's $10k-a-Day Simulated QA Swarm

Why Labs Are Paying Big for Pre-2022 Human-Written Code

The Pelican-on-a-Bicycle Benchmark and Why It Predicts Everything

✦Tool· 1

Simon Willison's AI Coding Stack: Claude Code for Web, YOLO Mode, GPT 5.4

▲Takeaway· 2

Tests and Docs No Longer Signal Quality; You Need 'Proof of Usage'

Invest in Your Own Agency and Be More Ambitious

Myth Buster· 2

Hot Take· 5

Explainer· 2

Story· 3

Tool· 1

Takeaway· 2