LLenny's Podcast
← All episodes
Logan Kilpatrick (head of developer relations)08 February 2024

Inside OpenAI

7Frameworks
15Insights

Frameworks in this episode

Productivity5 steps

Context Is All You Need Prompting

Treat the model as a brilliant stranger with zero context, and supply what a colleague would already know.

Read →
Leadership4 steps

High Agency, High Urgency Hiring Filter

Hire for two traits only — people who see a problem and go, and people who go now.

Read →
Mindset3 steps

Measure in Hundreds

If your unit of measurement is one hundred attempts, five failures means you have effectively tried zero times.

Read →
Innovation4 steps

The Better Tool, Same Problems Lens

Every model leap gets normalised within months — build for the boring future, not the euphoric one.

Read →
Leadership4 steps

The Constrained-Resource Headcount Test

When the bottleneck isn't people, each new hire is a net productivity loss unless they uplevel everyone.

Read →
Strategy5 steps

The Platform Encroachment Test

Build where the platform's mission says it will never go — the general layer is not yours to own.

Read →
Strategy4 steps

The Proxy Goal Ladder

Every metric you chase is a proxy — climb the ladder to the mission before you optimise it.

Read →

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Hot Take· 4

Hot Take06:30

Why Logan is grateful the OpenAI crisis happened now, not later

Reflecting on the board drama, Logan says one of his takeaways is gratitude that it happened while the stakes were relatively low. Today OpenAI's collapse would hurt customers who built businesses on it, but on a world scale someone else would continue the march toward general intelligence. Had the same event happened in five or ten years, the outcome could have been far worse.

  • Customers have built businesses on OpenAI, so a failure would hurt them
  • On a world scale, another lab would still build the models if OpenAI disappeared
  • The same crisis five to ten years later could have been much worse
  • Lower current stakes made this the better time for it to happen

“one of my takeaways was I'm actually very grateful that this happened when it happened”

Logan Kilpatrick · 06:30

“somebody else will build a a model if open AI disappeared”

Logan Kilpatrick · 07:00
#openai#agi#risk
Hot Take08:00

2024 is the year AI moves beyond the chat box

Logan is most excited by new interfaces to AI beyond chat, citing the Rabbit R1 hardware device and TL Draw's infinite canvas. He argues chat is only the predominant interface today, and that humans often make more sense of information laid out on a canvas than listed in a chat thread. He frames 2024 as the year of multimodal AI and of pushing new UX paradigms.

  • Rabbit R1 and TL Draw's infinite canvas are early examples of non-chat AI interfaces
  • A canvas the AI fills in with files, videos, and references can beat a chat list
  • 2024 is the year of multimodal AI and new UX paradigms
  • Chat is predominant today but not the natural end state

“I think like 2024 is the year of multimodal AI but it's also the year that people really push the boundaries of um some of…”

Logan Kilpatrick · 09:30
#ai-interfaces#multimodal#ux#product
Hot Take39:00

At OpenAI, revenue is just a proxy for compute

Logan explains that OpenAI's stated goals are often abstractions for something deeper. Even revenue isn't the real goal — it's a proxy for buying more compute, which buys more GPUs, which trains better models, which advances the mission. He warns that hearing a goal like revenue in a vacuum misleads people into thinking OpenAI just wants to make money.

  • Stated goals are often intermediate abstractions toward the mission
  • Revenue is a proxy for compute, which means more GPUs and better models
  • Hearing 'revenue' out of context misreads OpenAI as just chasing money
  • The real end goal is achieving the mission, not the metric

“even if revenue is a goal it's like revenue is not actually the goal revenue is a proxy for getting more compute which is then…”

Logan Kilpatrick · 39:30
#openai#strategy#metrics#compute
Hot Take1:06:30

AI won't replace you — humans using AI will

Logan closes on the narrative he endorses: it isn't AI that replaces humans, it's other humans who are augmented by AI tools and thereby more competitive in the job market. He frames now as the best time to learn these tools, since using them makes you more productive and empowered in your work.

  • It's not AI replacing humans, it's AI-augmented humans out-competing others
  • AI-augmented workers are more competitive in the job market
  • Now is the best time to learn these tools
  • Using AI makes you more productive and empowered in your job

“it's not AI That's going to replace humans it's like other humans that are being augmented and like using AI tools that are like going…”

Logan Kilpatrick · 1:06:30
#future-of-work#ai#careers#advice

Explainer· 3

Explainer13:30

Engineering is the highest-leverage use of AI today

Logan argues software engineering is one of the highest-leverage tasks to apply AI to right now, estimating at least a 50% improvement, especially on lower-hanging fruit. He notes he ships faster himself using ChatGPT and points to GitHub Copilot studies as an analogy for the gains. He also references a Harvard Business School study on the order-of-magnitude efficiency gains for AI users.

  • Engineering is one of the highest-leverage tasks to apply AI to today
  • At least a 50% improvement, especially on lower-hanging-fruit tasks
  • GitHub Copilot studies are a useful analogy for the productivity gains
  • An HBS study measured order-of-magnitude efficiency gains for AI users

“I think engineering is actually like one of the highest leverage things that you could be using AI to do today and like really unlocking…”

Logan Kilpatrick · 14:30
#productivity#engineering#chatgpt#copilot
Explainer18:30

Prompt engineering is really about giving context

Logan reframes prompt engineering as a fundamentally human act: models, like people, give generic answers when they lack context. The model has human-level intelligence but knows nothing about you, your goals, or what you'll ask, so generic prompts get generic responses. His core advice is that context is the thing that matters most for getting value from a language model.

  • Models default to generic answers because they're trained to just answer the question
  • Treat the model as human-level intelligence with zero context about you
  • Generic outputs come from users forgetting to supply context
  • 'Context is all you need' — the single biggest lever on output quality

“imagine a human human level intelligence but like literally no context like it has no idea what you're going to ask it it's never met…”

Logan Kilpatrick · 20:30

“context is is all you need context is the only thing that matters”

Logan Kilpatrick · 24:00
#prompt-engineering#chatgpt#context#llms
Explainer43:00

Why adding a researcher can slow the whole team down

Logan relays a counterintuitive point from an OpenAI researcher: in a GPU-constrained world, each new researcher can be a net productivity loss for the group. Unless the new hire dramatically uplevels everyone, adding someone on a different research direction forces everyone to share GPUs, slowing all experiments. This is why OpenAI keeps its research team intentionally small, unlike product teams where more engineers means more output.

  • In a GPU-constrained world, a new researcher can be a net productivity loss
  • A new person on a different direction means everyone shares GPUs and slows down
  • The exception is a hire who profoundly uplevels the whole group
  • OpenAI keeps research small on purpose; product teams scale differently

“if you just add somebody who's going to go and like tackle some completely different research Direction you now have to share your gpus with…”

Logan Kilpatrick · 44:00
#research#gpus#openai#scaling

Story· 1

Story04:00

What the Sam Altman board weekend was like from the inside

Logan describes the shock inside OpenAI when the board removed Sam Altman over Thanksgiving weekend, a week the company had planned as a rare full reset. Despite deep trust in leadership being suddenly tested, what surprised him most was how fast everyone returned to work. He believes many companies would have been derailed for a long time by an event like this.

  • The changes landed Friday afternoon of a rare company-wide break week
  • Employees had deep, longstanding trust in Sam and Greg, making it very surprising
  • OpenAI's transparent culture meant people usually hear about problems early, but not this time
  • By Monday the team was laser-focused and back to work, not derailed

“I think the thing that surprised me the most was like just how quickly everybody got back to business”

Logan Kilpatrick · 05:30
#openai#leadership#culture#sam-altman

Q&A· 1

Q&A10:00

How to build so OpenAI won't disrupt you

Asked how founders should think about where OpenAI won't go, Logan says OpenAI is focused on very general capabilities like reasoning, coding, and writing. Vertical and domain-specific applications, like Harvey for legal work, are safe because OpenAI's general models will never be as tuned as a specialist's custom build. He warns that anyone building a general-purpose assistant should expect OpenAI to eventually compete there.

  • OpenAI targets general reasoning, coding, and writing, not verticals
  • Harvey (legal AI) is a safe example: custom models OpenAI won't match
  • Expect OpenAI to launch general-purpose agent products (it already does via GPTs)
  • OpenAI won't build verticalized products like an AI sales agent

“we're not going to launch like some of these like very verticalized products like we're not going to launch like an AI sales agent”

Logan Kilpatrick · 11:30
#product-strategy#startups#openai#moats

Tool· 2

Tool17:00

The private planning GPT Logan built for himself

Logan describes a personal GPT he built to force better quarterly planning. He took best-practice planning tips from an online article, loaded them into the GPT, and now runs his plans through it to generate timelines, metrics, success criteria, and important cross-functional stakeholders. He says it forces him to consider the things people typically miss or are bad at during planning.

  • Built from planning best-practice tips found in an online article
  • Generates timelines, specific metrics, and success criteria for a plan
  • Surfaces cross-functional stakeholders to include in the process
  • Forces consistency and catches what people usually miss in planning

“the private gbt that I use myself that like helps with some of the like planning stuff for for different quarters”

Logan Kilpatrick · 17:00
#gpts#planning#productivity#workflow
Tool52:30

The new V3 embedding models: cheaper and multilingual

Logan previews OpenAI's third-generation embedding models, the tech behind question-answering over your own documents (like Lenny's own lennybot.com). The big improvements are much stronger non-English performance, opening embeddings to many more languages, and roughly five times lower cost. He notes you can embed about 62,000 pages of text for a dollar.

  • V3 embeddings power Q&A grounded in your own corpus of knowledge
  • Non-English performance improved significantly, unlocking many more languages
  • Roughly five times cheaper than before
  • About 62,000 pages of text can be embedded for one dollar

“I think now it's like you can embed I'm pretty sure was like 62,000 pages of text for $1”

Logan Kilpatrick · 55:00

“it's like five times cheaper as well which is wonderful”

Logan Kilpatrick · 54:30
#embeddings#openai-api#pricing#rag

Takeaway· 4

Takeaway24:30

Why a smiley face can make ChatGPT answer better

Logan explains the odd prompt tricks that work, like adding a smiley face or telling the model to take a break, because the training corpus is human-to-human communication where those cues signal effort and positivity. He cautions these give only a small lift, on the order of one or two percent, which may be imperceptible on short answers but can matter materially on long generations.

  • Adding a smiley face or telling the model to 'take a break' can raise performance
  • It works because models are trained on human-to-human messages
  • The gain is small, roughly one or two percent
  • Likely imperceptible on short answers, but material on long text

“there's a lot of like really small silly things like adding a smiley face increases the performance of the model”

Logan Kilpatrick · 24:30
#prompt-engineering#chatgpt#tips
Takeaway32:00

The two traits OpenAI hires for: high agency and urgency

Logan says if he were hiring five people today, the top two traits he'd look for are high agency and working with urgency. High-agency people don't need to gather fifty people's consensus; they hear a customer problem and immediately start pushing on the solution. He contrasts this with traditional companies that stall by checking with seven different departments.

  • High agency and urgency are the top two traits he'd hire for
  • High-agency people act without needing broad consensus
  • They hear a customer challenge and immediately push on a solution
  • Traditional companies stall by routing feedback through many departments

“folks are so high agency like they see a problem and they go and tackle it like they hear something from our customers about a…”

Logan Kilpatrick · 34:00
#hiring#culture#openai#agency
Takeaway47:30

GPT-5 will feel normal fast — plan for that, not magic

Logan pushes back on wild GPT-5 expectations. Like GPT-4, it will be an extremely effective tool that solves the same problems better and faster, not a system doing backflips while writing your code. His edge-giving insight: powerful tools become normal very quickly, so planning for rapid user habituation is smarter than assuming the model will change everything.

  • GPT-5 will be a very effective tool similar to GPT-4, not a magic leap
  • The same real-world problems remain; you just get a better tool for them
  • Powerful AI tools become 'very normal very quickly'
  • Planning for fast habituation is an edge; expecting it to change everything is the wrong framing

“it is just going to be this like very effective tool very similar to gbd4 and it's also going to become like very normal very…”

Logan Kilpatrick · 50:00
#gpt-5#product-strategy#expectations#ai
Takeaway1:03:00

Logan's life motto: measure in hundreds

Logan keeps a note behind his camera that reads 'measure in hundreds.' The idea is that when people say they tried something and it didn't work, they've usually tried a handful of times, which on a scale of hundreds is effectively zero. Everything compounds over many attempts, so if you don't try enough times you'll never succeed.

  • 'Measure in hundreds' is his life motto, kept on a note behind his camera
  • Five failed attempts, measured against hundreds, is basically zero tries
  • Success is built on compounding and many attempts
  • Too few attempts guarantees you never succeed at something

“and it's measure in hundreds I love this idea of measuring things in hundreds”

Logan Kilpatrick · 1:03:00

“if I measure in hundreds the five times that you failed at something youve failed tried zero times”

Logan Kilpatrick · 1:03:00
#mindset#persistence#advice