LLenny's Podcast
← All episodes
Garrett Lord (Handshake CEO)24 August 2025

Inside the expert network training every frontier AI model

4Frameworks
14Insights

Episode overview

Lenny Rachitsky interviews Handshake co-founder and CEO Garrett Lord about the company's rapid expansion into expert-generated training data for frontier AI laboratories. Lord explains how post-training uses PhDs and other specialists to expose model failures, supply correct answers and reasoning, and create tool-use trajectories, rubrics, and multimodal data. They also examine Handshake's distribution advantage, the operational separation used to incubate its new AI business, and Lord's belief that AI-native workers will become more productive rather than simply disappear from the labor market.

Key ideas

  • As internet-scale pre-training gains flatten, frontier labs increasingly rely on post-training data to improve specific capabilities.
  • Models have advanced beyond many generalist labeling tasks, increasing demand for experts in STEM, medicine, law, finance, education, and other professional domains.
  • Experts break models, provide ground-truth answers and reasoning, create rubrics, and record complete tool-use trajectories that can improve model behavior.
  • Training-data providers must deliver quality, volume, and speed because research teams rapidly test hypotheses and scale only the datasets that show gains.
  • Handshake's network of students, alumni, PhDs, universities, and employers gives it low acquisition costs, strong trust, and fast access to specialized talent.
  • Handshake incubated the AI initiative with a separate, founder-led team, dedicated functions, distinct incentives, clear ownership, and a metrics-heavy operating cadence.
  • Lord argues that AI-native young workers can use new tools as an 'Iron Man suit,' increasing their productivity while shifting the skills jobs require.
  • The kinds of valuable human data will continue evolving toward multimodal content, scientific tools, CAD files, professional workflows, and other real-world trajectories.

Transcript available · source text is retained privately and is not published

Frameworks in this episode

Strategy3 steps

The Access-to-an-Audience Moat

In human data the only durable moat is a trusted, targetable audience — not recruiters or ads

Read →
Entrepreneurship5 steps

The Company-Inside-a-Company Incubation Playbook

Spin a 0-to-1 business inside a mature company by ring-fencing it completely from the core

Read →
Innovation4 steps

The Middleman-Signal Disintermediation Play

When middlemen and end-customers both come to you directly, that's the signal to build the business yourself

Read →
Strategy3 steps

The Quality-Volume-Speed Priority Stack

What high-stakes data buyers actually rank: quality first, then volume, then speed

Read →

People & resources mentioned

Attributed to the moment in the episode. Timestamps are approximate.

People · 5

  • Garrett LordMentionsGarrett is the co-founder and CEO of Handshake

    “Garrett is the co-founder and CEO of Handshake”

    ▶ Watch at ~1:00approx.
  • Lenny RachitskyMentionstoday we've got another very special compilation episode something I've been pulling on more and more with the podcast and the newsletter

    “Today my guest is Garrett Lord.”

    ▶ Watch at ~1:00approx.
  • Alex WangMentionsAlex now leads the super intelligence team at Meta.

    “Alex is running super intelligence and meta.”

    ▶ Watch at ~29:30approx.
  • Peter ThielCoinedyou had this debate with Peter Teal that I came across where you were debating whether technologies stop progressing

    “I'm a sucker for Peter Thiel's 0ero to1.”

    ▶ Watch at ~1:04:00approx.
  • Ben HorowitzCoinedBen is the Z in A16Z, the world's largest venture capital firm with over $46 billion in committed capital.

    “Hard things about hard things obviously”

    ▶ Watch at ~1:04:30approx.

Resources · 28

  • HandshakeCoinedcompany · Garrett Lord and co-founders

    “Garrett is the co-founder and CEO of Handshake”

    ▶ Watch at ~1:00approx.
  • LinkedInMentionswebsite · LinkedIn

    “They're essentially LinkedIn for college students.”

    ▶ Watch at ~1:00approx.
  • Lenny's NewsletterCoinednewsletter · Lenny Rachitsky

    “if you become an annual subscriber of my newsletter, you get a year free”

    ▶ Watch at ~2:30approx.
  • CodeRabbitMentionssoftware · CodeRabbit

    “This episode is brought to you by Code Rabbit, the AI code review platform”

    ▶ Watch at ~3:00approx.
  • ConductorMentionssoftware · Orkes

    “the company behind open-source Conductor, the orchestration platform powering modern enterprise apps and agentic workflows.”

    ▶ Watch at ~4:00approx.
  • Llama 4Mentionssoftware · Meta

    “when Llama 4 is released, you'll see like the benchmarks across various domains.”

    ▶ Watch at ~7:00approx.
  • GPQARecommendspaper

    “A great example is a public paper called like GPQA.”

    ▶ Watch at ~13:00approx.
  • ChatGPTUsessoftware · OpenAI

    “find flaws in what say Chad GBT is producing”

    ▶ Watch at ~15:30approx.
  • University of MiamiMentionsplace

    “She got her PhD from the University of Miami.”

    ▶ Watch at ~16:00approx.
  • Massachusetts Institute of TechnologyUsesplace

    “we just reach out to students at Stanford and Berkeley and MIT”

    ▶ Watch at ~20:30approx.
  • Stanford UniversityUsesplace

    “we just reach out to students at Stanford and Berkeley and MIT”

    ▶ Watch at ~20:30approx.
  • University of California, BerkeleyUsesplace

    “we just reach out to students at Stanford and Berkeley and MIT”

    ▶ Watch at ~20:30approx.
  • Scale AIMentionscompany · Alex Wang and Lucy Guo

    “Scale was just like acquired for I don't know 30 like sort of acquired for $30 billion.”

    ▶ Watch at ~28:00approx.
  • MetaMentionscompany

    “Alex is running super intelligence and meta.”

    ▶ Watch at ~29:30approx.
  • AnthropicMentionscompany

    “Today's episode is brought to you by Anthropic, the team behind Claude.”

    ▶ Watch at ~31:30approx.
  • ClaudeUsessoftware · Anthropic

    “I use Claude at least 10 times a day.”

    ▶ Watch at ~31:30approx.
  • AlphaSightsUsescompany

    “I hired like three expert firms, alpha in the alpha sites and like GLG”

    ▶ Watch at ~47:00approx.
  • GLGUsescompany

    “I hired like three expert firms, alpha in the alpha sites and like GLG”

    ▶ Watch at ~47:00approx.
  • PalantirUsescompany · Peter Thiel, Alex Karp, Stephen Cohen, Joe Lonsdale, and Nathan Gettings

    “I worked at Palunteer as an intern. It totally changed my life.”

    ▶ Watch at ~59:00approx.
  • ChatGPTUsessoftware · OpenAI

    “I use voice mode all the time. That's my default chat dbt experience”

    ▶ Watch at ~1:01:30approx.
  • How to Start a StartupUsescourse · Sam Altman and Stanford University

    “watched Peter Teal's like startup school class at Stanford.”

    ▶ Watch at ~1:04:00approx.
  • Shoe DogRecommendsbook · Phil Knight

    “love love Shoe Dog like think that you know it's epitome of like starting a company.”

    ▶ Watch at ~1:04:00approx.
  • Zero to OneRecommendsbook · Peter Thiel and Blake Masters

    “I'm a sucker for Peter Thiel's 0ero to1. I read it when I started the company”

    ▶ Watch at ~1:04:00approx.
  • The Hard Thing About Hard ThingsRecommendsbook · Ben Horowitz

    “Hard things about hard things obviously but these are these are all quite common books.”

    ▶ Watch at ~1:04:30approx.
  • Game of ThronesRecommendstv show · David Benioff and D. B. Weiss

    “I did start Game of Thrones with my wife and I”

    ▶ Watch at ~1:04:30approx.
  • SNOORecommendsproduct · Happiest Baby

    “Thene the baby automated. It's like has uh really helped us a lot.”

    ▶ Watch at ~1:05:00approx.
  • Princeton UniversityMentionsplace

    “I almost got arrested at Princeton”

    ▶ Watch at ~1:06:00approx.
  • Lenny's PodcastCoinedpodcast · Lenny Rachitsky

    “You can find all past episodes or learn more about the show at lennispodcast.com.”

    ▶ Watch at ~1:09:30approx.

Spot an error or want something removed? Request a correction or removal.

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Hot Take· 4

Hot Take24:00

GDP Growth Over UBI: The AI Jobs Tension

Asked about the tension between students training smarter models and those same models threatening entry-level jobs, Garrett lands firmly on the optimistic side. He believes AI accelerates every person's ability to create economic impact, and says the doom about young people having no jobs isn't what Handshake hears from the Fortune 500 employers on its platform.

  • Garrett is in the 'GDP growth over universal basic income' camp.
  • 100% of the Fortune 500 uses Handshake to hire young people.
  • Employers are not reporting that young people won't have jobs.
  • One AI-native person can now do the work that used to take a whole team.
  • Jobs will evolve; people will need to upskill and reskill.

“I'm brought in the camp of like GDP growth over like universal basic income.”

Garrett Lord · 24:30

“that's not what we're hearing from our employers.”

Garrett Lord · 25:00
#ai-jobs#economy#hot-take#future-of-work
Hot Take26:30

Why AI-Native Young People Have a Huge Advantage

Garrett argues that being AI-native is the new resume superpower, comparing it to when people once listed 'Google search' as a skill. Young people who grew up with these tools come in as high-output operators, and those training the models get an extra edge.

  • People used to list Google search as a skill on their resume.
  • Being AI-native is like wearing an 'Iron Man suit' at work.
  • Young people are at a huge advantage because they grew up with the tools.
  • Fellows bring insights from model training back into their classrooms and research.

“you used to put like Google search on as like a skill on your resume, right?”

Garrett Lord · 26:30

“I think being like AI native and having your Iron Man suit on and understanding how to leverage these tools is like uh young people…”

Garrett Lord · 27:00
#ai-native#future-of-work#young-talent#hot-take
Hot Take30:30

The Only Moat in Human Data Is Access to an Audience

Garrett states Handshake's core strategic thesis. Competitors spend millions on Instagram, TikTok, Google, and YouTube ads plus armies of LinkedIn recruiters to source experts they can't target well. Handshake already has a trusted, targetable audience of 18 million professionals, which lets it reach scale and quality faster than anyone.

  • The only moat in human data is access to an audience.
  • Rivals run TikTok, Instagram, Google, and YouTube ads plus 100+ recruiters on LinkedIn.
  • They can't target a physics PhD well on an Instagram feed and have no brand recognition.
  • Handshake built a decade of trust with 18 million people and has rich academic profile data.
  • That lets Handshake target effectively and scale high-quality data faster than competitors.

“the only the only moat in human data is access to an audience.”

Garrett Lord · 30:30
#strategy#moat#distribution#human-data
Hot Take56:00

Leave Nothing to Chance: A Once-in-a-Lifetime Demand Moment

Garrett describes the culture and mindset driving the new business. He drew the number of days left in the year on a whiteboard, convinced he'll never again see a market with this much unlimited demand. The motto is 'leave nothing to chance': get on the plane, make the late-night push, check the data six times.

  • The team's motto is 'leave nothing to chance.'
  • Garrett drew the number of days in the year on a whiteboard.
  • He believes he'll never again see a business with this much unlimited demand.
  • Success comes down to execution: get on the plane, make the late-night push, ship the extra feature.
  • A flat, celebratory culture calls out the people 'putting up points.'

“there will never be a time like this. I've never seen anything like it. I doubt I'll ever feel anything like this in business again…”

Garrett Lord · 56:30
#culture#execution#hot-take#leadership

Explainer· 3

Explainer06:00

Pre-Training vs Post-Training: Where Model Gains Now Come From

Garrett breaks down the two phases of training a frontier model. Pre-training was about ingesting the entire corpus of human knowledge on the internet, but gains from that have plateaued. Most of the improvement now comes from post-training, where labs collect high-quality data to improve specific capability areas like coding, math, and law.

  • Training has two primary phases: pre-training and post-training.
  • Pre-training ingested the whole internet, every YouTube video and book.
  • Gains from pre-training began to asymptote 18-24 months ago.
  • Post-training augments and improves data across each capability area a lab cares about.
  • Post-training data can be RL environments, trajectories, prompt-response pairs, or human preference ranking.

“There's a pre-training and a post-training process in training a model.”

Garrett Lord · 06:00

“And about 18 months ago, 24 months ago, we started to really see like an asmtoing of gains coming from because they had essentially like…”

Garrett Lord · 06:30
#ai-training#llms#post-training#data-labeling
Explainer12:00

How PhDs Actually 'Break' AI Models

Garrett explains what expert labelers do day-to-day: probe a model until it fails, then provide the ground-truth answer and the correct step-by-step reasoning. The average person can't reliably break a frontier model, but a domain PhD can find where its reasoning steps go wrong. He points to the public GPQA paper as a concrete example.

  • Experts break the model, provide the ground-truth answer, and supply the correct reasoning steps.
  • Models are non-deterministic, so a model may get an answer right once but not three of five times.
  • The focus is often on fixing the intermediate reasoning steps, not just the final answer.
  • The GPQA paper is a public reference for how this works.
  • The average person cannot break the models; a domain PhD can.

“I wouldn't say it's easy for them, but the average person cannot break the models.”

Garrett Lord · 12:30

“A great example is a public paper called like GPQA.”

Garrett Lord · 13:00
#ai-training#reasoning#gpqa#experts
Explainer17:30

What a 'Trajectory' Means in AI Training Data

Garrett defines a term he uses repeatedly: a trajectory. It's the full capture of a human solving a problem end-to-end, including their screen, mouse movements, and voiceover narration. The output of this work is typically JSON data.

  • A trajectory captures the entire environment of what you're doing.
  • It includes your screen, your mouse, and your voiceover.
  • The output of the work is JSON data.
  • Labs want to understand how humans actually think and problem-solve, including how they handle roadblocks.

“A trajectory is basically just like the entire environment that is collecting what you're doing.”

Garrett Lord · 17:30
#ai-training#trajectories#reinforcement-learning

Story· 3

Story16:00

Rachel: The Eighth-Grade Teacher Training AI on Education

To make expert labeling concrete, Garrett describes Rachel, a PhD from the University of Miami who taught eighth grade for two decades and was an adjunct professor in education. She interacts with state-of-the-art models on educational design, spotting where they get teaching wrong in a non-verifiable domain.

  • Education is a 'non-verifiable' domain with no single correct answer.
  • Rachel has a PhD in education plus 10+ years of hands-on eighth-grade teaching.
  • She helps models understand the forefront of educational design.
  • The example spans from non-verifiable domains all the way down to verifiable engineering problems.

“So there's like a PhD student uh Rachel on the network. She got her PhD from the University of Miami.”

Garrett Lord · 16:00
#data-labeling#education#experts#case-study
Story33:30

How Handshake Discovered a New Business Hiding in Its Network

Garrett tells how the AI data business emerged out of the existing $200M Handshake. Middleman companies started asking to recruit Handshake's PhDs, and Handshake noticed the experience for those experts was frustrating. When frontier labs began reaching out directly to cut out the middleman, Garrett saw the opening and went zero-to-fifty-million ARR in four months.

  • Middleman companies started asking to recruit Handshake's PhDs and master students.
  • Users reported a frustrating, transactional experience on other platforms.
  • Frontier labs began reaching out directly to cut out the middleman.
  • Garrett started flying around over Christmas and New Year's chasing leads.
  • The business hit $50M ARR in four months and is now working with seven frontier labs.

“we started to see all the what I would call like middleman companies reaching out to us saying, can we recruit your PhDs and master…”

Garrett Lord · 34:00

“zero to 50 is pretty good in four months, I think.”

Garrett Lord · 36:30
#startup-story#0-to-1#handshake#growth
Story1:06:00

Almost Arrested at Princeton for Showering in the Pool

In the lightning round, Garrett shares an early hustle story. To save money while pitching campuses, he and his co-founders slept in a Ford Focus in McDonald's parking lots and showered in university pools. Princeton campus security didn't appreciate it, but the incident actually made the sales meeting more exciting and showed the school his commitment.

  • The team slept in a Ford Focus in well-lit McDonald's parking lots with good Wi-Fi.
  • They freshened up by showering in university pools, which are open and always have showers.
  • Princeton campus security did not appreciate a non-student showering there.
  • Security called the career services director to ask who Garrett Lord was.
  • The stunt showed a level of commitment that excited the school he was selling to.

“we were sleeping out of our car. We had this like Ford Focus. We would put 20 30,000 miles on it. Sleep in the back…”

Garrett Lord · 1:06:00

“the Princeton campus security did not appreciate me showering as a non- studentent.”

Garrett Lord · 1:06:30
#startup-story#hustle#handshake#sales

Takeaway· 4

Takeaway10:30

The Labeling Market Shifted From Generalists to Experts

The old data-labeling market ran on low-cost international generalist labor doing basic tasks. As models got better, that work stopped adding value. Now labs need genuine experts across every economically valuable domain to push the frontier.

  • The old market used talented, low-cost international labor for basic generalist tasks.
  • Models got good enough that generalist labelers are no longer needed.
  • Labs now need experts across every capability area they care about.
  • Focus is on advanced STEM plus derivative fields like accounting, law, medicine, and finance.

“the models have gotten so good that the generalists are no longer needed. Like what they really need is experts. experts across every area that…”

Garrett Lord · 11:00
#data-labeling#experts#ai-labor
Takeaway20:00

The Three Things AI Labs Care About: Quality, Volume, Speed

Garrett zooms out to what model builders actually want from a data partner. Quality comes first because bad data is very hard to overcome, like teaching a student the wrong thing. Then volume in the most advanced domains, and speed, because researchers run many hypotheses at once and pour resources into whichever pipeline shows a gain.

  • Quality is first and foremost; wrong data is extremely hard to train out.
  • Volume matters: generating thousands of high-quality pieces in advanced domains.
  • Speed matters because labs test three or four bets at once and scale the winner.
  • Handshake reaches top-GPA physics students at Stanford, Berkeley, and MIT to hit volume fast.

“They care about three things. They care about like quality first and foremost. You have to have high quality data.”

Garrett Lord · 20:00

“And then the other thing I'd say model builders care about is speed because they have all these hypotheses and they're constantly testing different pipelines.”

Garrett Lord · 21:00
#data-labeling#ai-labs#quality#operations
Takeaway27:00

PhDs Make $150/Hour Breaking Models Instead of $25 as a TA

Garrett shares the economics for the expert fellows on the network. A PhD who might earn $25/hour as a teaching assistant can make $100-$200/hour in their field of expertise breaking frontier models. Beyond the money, fellows say the work helps advance their own research.

  • Fellows can make $100, $150, or $200 an hour in their field of expertise.
  • A teaching assistant might earn only $25 an hour by comparison.
  • Fellows bring insights back into the classroom to teach more effectively.
  • They learn to leverage the tools to advance their own research; it's getting paid to learn a skill.

“you can make like 25 bucks an hour being a teacher assistant or you can actually make $150 an hour breaking the latest models.”

Garrett Lord · 27:00

“it is quite cool to get kind of paid to learn a skill.”

Garrett Lord · 27:30
#experts#compensation#phds#data-labeling
Takeaway53:30

Separate Everything: Building 0-to-1 Inside a 10-Year-Old Company

Garrett explains what made incubating a new business inside a mature company work: radical separation. Separate engineering, design, finance, and operations teams; people with one job only; a different office, different comp tied to new-business hurdles, and a founder-mode CEO spending 80%+ of his time on it.

  • It's hard to incubate something new inside an established business.
  • Everything was separate: engineering, design, accounts, operations, and finance teams.
  • People had one job only, and it was making the new business successful.
  • The team sat in a separate part of the office, five days a week plus weekends.
  • Compensation was tied to hurdles in the new business so people felt like owners.
  • Garrett went founder-mode, spending 80%+ of his time and attention on it.

“it's also like hard to like incubate something new inside of a business.”

Garrett Lord · 39:30

“I just really believe it's separate and everything like separate engineering team separate design team separate accounts and operations team separate finance team”

Garrett Lord · 54:00
#0-to-1#founder-mode#org-design#leadership