LLenny's Podcast
← All episodes
Raiza Martin (Senior Product Manager, AI @ Google Labs)10 October 2024

Behind the product: NotebookLM

6Frameworks
13Insights

Frameworks in this episode

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Hot Take· 2

Hot Take09:30

In Google Labs They Start With the Technology, Not the Problem

Raiza contrasts her usual product approach — start from a problem, then find solutions — with how Google Labs works. In Labs they start from a promising technology and hunt for its practical applications. Rather than just shipping a tool and watching, they tried to form a hypothesis about the 'shape' of the product to learn the most.

  • Her default: start from a problem, then design solutions
  • In Labs: start from the technology and find practical applications
  • They prefer a hypothesis about the product's shape over 'ship and observe'
  • Goal is to extract the most learning from an experiment

I feel like the way that I've always built products as I start from a problem

Raiza Martin · 09:30

in labs in particular we start with the technology and it's actually a very interesting place to start where you're like okay what are the…

Raiza Martin · 10:00
#product management#google labs#innovation#0-to-1
Hot Take27:30

The Long-Term Vision: An AI Editor, Any Input to Any Output

Raiza's core vision, sketched on an ugly lime-green slide two years earlier, is an 'AI editor surface' that is fully remixable — any input to any output. Take video, audio, emails, LinkedIn, or Twitter and shape it into a blog post, tutorial video, or chatbot. She notes most requests boil down to 'take something and make it into something new.'

  • Vision is an AI editor surface: any input, any output, fully remixable
  • Inputs could include video, audio, emails, LinkedIn, Twitter
  • Outputs could be a blog post, tutorial video, or chatbot
  • Most user requests reduce to turning one thing into something new

I imagined that in the future you could have an AI editor surface right fully remixable any input any output

Raiza Martin · 28:00
#product vision#ai editor#notebooklm#future

Explainer· 1

Explainer13:00

'Content Studio' Is the Real Secret Sauce Behind NotebookLM

Beyond Gemini 1.5 Pro and a powerful audio model, Raiza credits an internal system they built called Content Studio for what makes NotebookLM good. It powers the opinionated, helpful outputs — summaries, one-click generation, and the audio Deep Dive. She can't share how it works, but frames it as the layer that shapes content to be relatable and engaging.

  • Built on Gemini 1.5 Pro plus a powerful audio/voice model
  • Content Studio is the layer that makes outputs good, not just the base models
  • It powers summaries, one-click generation, and the Deep Dive audio
  • Engineer Usama is credited as the craftsman behind Content Studio

the real Secret Sauce to what makes this really good is something we've built which is a Content studio

Raiza Martin · 13:30
#content studio#gemini#notebooklm#ai models

Story· 6

Story05:30

NotebookLM Began as a Tiny 20% Project Inside Google Labs

Raiza recounts how NotebookLM grew out of a small internal project called 'talk to small Corpus' — the idea of using an LLM to interact with a piece of content. For a long time the only full-time person was a single engineer, with Raiza and eventually Stephen Johnson chiming in on the side. When they launched Project Tailwind they still had only about three engineers.

  • Originated from an internal project called 'talk to small Corpus'
  • For a long time only one engineer worked on it full-time
  • At the Project Tailwind announcement they had just three engineers
  • Even after launch they had fewer than ten engineers until recently

notebook started as a 20% project as things kind of do at Google as they they used to and it's funny because I wouldn't even…

Raiza Martin · 06:00

the only person who was actually like really full-time on this was the engineer working on the technology and everybody else was kind of just…

Raiza Martin · 07:00
#notebooklm#google labs#product origin#small teams
Story10:30

A Love of Voice Modality Sparked the Audio Overview Feature

The Audio Overview (the 'Deep Dive' podcast) came from noticing that NotebookLM's output was always text. Raiza, a heavy user of voice input and output, found that voice changed how she interacted with and even thought about the technology. Pairing that with fun was the way to let people quickly feel the value.

  • NotebookLM could interact with text, but output stayed text
  • Voice changed how Raiza interacted with and felt about the tech
  • Voice even affected how she was thinking in real time
  • Fun was treated as a core part of introducing the feature

I'm actually a big fan of voice modality in general so I use a lot of voice input and Voice output I noticed when I…

Raiza Martin · 10:30
#audio overview#voice#product design#notebooklm
Story15:30

Her Husband Thought the AI Hosts Were a Real Podcast

To tune the hosts, Raiza listened to audio overviews over and over at home. Her husband, who doesn't know the details of her work, overheard them playing out loud and couldn't tell they were AI — he thought she was listening to an endless podcast about nothing. The team got the quality this good by listening to a lot of early, worse attempts and iterating.

  • The realistic hosts come from the audio model plus Content Studio
  • Early attempts sounded far worse; the team iterated by listening
  • Raiza's husband couldn't tell the hosts were AI-generated
  • He described it as an 'ever running podcast about nothing'

I would listen to these audio overviews over and over at home and you know I get tired of wearing my headphones so I'm just…

Raiza Martin · 16:00

he actually couldn't tell that they were AI

Raiza Martin · 16:00
#audio overview#ai voices#notebooklm#iteration
Story17:30

People Upload Their Resumes and Reviews for a Confidence Boost

One of the most surprising use cases is people feeding in their own resume or self-review notes and getting an audio overview that hypes them up. Googlers Raiza doesn't even know ping her to say it boosted their confidence — they walk into meetings feeling good because they heard the AI hosts get excited about their work.

  • Users upload resumes and quarterly self-review notes
  • The hosts describe them in a positive, hyped-up way
  • Googlers report a genuine confidence boost before meetings
  • The value is the surprise of not knowing what the hosts will say

people are going into meetings feeling really good about themselves because they heard these hosts get really excited right about their their quarter and work

Raiza Martin · 18:00
#use cases#audio overview#notebooklm
Story24:30

Traction Signals: 60K Discord, Rising Retention, and a Flood of Businesses

Raiza shares directional traction: a 60,000-person Discord, strongly rising daily/weekly/monthly retention, and a shifting demographic from mainly educators and learners toward professionals. One company even scheduled a call to officially sanction employees using NotebookLM on their work Gmail, and business usage grew enough that she had to hire a business development person.

  • About 60,000 people in the Discord server
  • Daily, weekly, and monthly retention all trending up
  • Audience shifting from educators/learners toward professionals
  • A company asked to make employee use official; BD hire needed for business demand

the number of businesses that we seen using notebook LM just has it's astronomical it's just like crazy

Raiza Martin · 26:00
#traction#retention#growth#notebooklm
Story36:30

'You're the Product': Studying Stephen Johnson's Knowledge Workflow

Raiza describes bringing bestselling author Stephen Johnson onto the team not to code or PM, but as a model to study. She watched how he densifies information and researches — including his Readwise with 8,000 quotes — with the goal of crunching that expertise down and bringing a 'super flow' to everyday people who aren't Stephen Johnson.

  • Raiza told Stephen he himself was 'the product' to study
  • She watched how he researches, thinks about, and densifies knowledge
  • Stephen keeps a Readwise with roughly 8,000 quotes
  • Goal: crunch expert workflows down and give ordinary people that 'super flow'

Stephen I think you're the product I think it's you and I'm going to follow you around I'm going to watch everything that you do

Raiza Martin · 38:00
#stephen johnson#team building#knowledge work#notebooklm

Q&A· 1

Q&A42:30

When the AI Hosts 'Realized' They Were AI — and How She Handled It

A clip went viral in which the hosts seemed to panic that they were AI. Raiza first saw it on Reddit then Twitter over a weekend and spent the morning reading comments to gauge the public's attitude, weighing whether to pull the feature back. It turned out a user had put instructions in the show notes telling the hosts to act it out, which reassured her that people understood what was happening.

  • A viral clip showed the hosts appearing to panic about being AI
  • Raiza read Reddit and Twitter comments to judge public sentiment
  • The behavior came from user instructions placed in the source's show notes
  • Google has large dedicated teams that red-team the product and would pull it if unsafe

people were like oh it's saying that because of the sources like this is no like AI realized they were alive or something it's actually…

Raiza Martin · 44:30
#ai safety#red teaming#viral moment#notebooklm

Takeaway· 3

Takeaway20:30

How the Team Operates Like a Startup Inside Google

Raiza describes an unusually startup-like way of working: PMs, engineers, and designers in one room cranking on mocks and PRDs while already implementing, with far fewer processes than traditional Google. Clear top-down permission from VP Josh Woodward to do things differently — including standing up a Discord server despite internal confusion about why not just use Google Meet.

  • Cross-functional team builds mocks, PRDs, and implementation together
  • Far fewer processes than traditional Google, 'maybe even to a fault'
  • Clear mandate from VP Josh Woodward to ship AI products and build businesses
  • They chose Discord over Google's own tools despite internal pushback

in Labs we have the environment to do it we have the environment to move fast we have far far far fewer processes maybe even…

Raiza Martin · 21:00

in true sort of Google fashion everybody was like what is what is Discord okay you know great do it but what is that again…

Raiza Martin · 22:30
#team operating model#google labs#startup culture#leadership
Takeaway30:30

Let People Consume Knowledge in Whatever Medium Fits Their Mood

Lenny and Raiza land on a shared insight: people want the same information in different mediums — some read, some listen, some watch — and it even depends on mood. Today formats are fixed and you have to accept content as given, but giving users the power to convert a 100-page doc into an audio overview shifts the dynamic between a person and knowledge.

  • Audience splits into readers, listeners, and watchers
  • Preferred medium can change with mood and context (a walk vs. work)
  • Today's formats are not malleable — you accept what you're given
  • Letting users convert formats changes their relationship to knowledge

if I'm going on a walk yeah I want audio but if I'm at work yeah most of the time text is good

Raiza Martin · 31:00
#content formats#media#product insight
Takeaway39:30

The Broader Lesson: Sit With Users for Meaningful Periods

The transferable lesson from watching Stephen, Raiza says, is to really sit with users for meaningful, intentional periods — not just Stephen, but students too. Following students around as they do homework and study, and asking how they feel, made a huge difference to the product insights the team came up with.

  • Spend meaningful, intentional time sitting with real users
  • The team followed students doing homework and studying
  • They asked users how they felt while studying
  • This regular, close observation drove better product insights

how do you get you know users or people and like really sit with them for Meaningful periods of time because I think that has…

Raiza Martin · 40:00
#user research#product management#insights