LLenny's Podcast
← All episodes
Garrett Lord (Handshake CEO)24 August 2025

Inside the expert network training every frontier AI model

4Frameworks
14Insights

Frameworks in this episode

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Hot Take· 4

Hot Take24:00

GDP Growth Over UBI: The AI Jobs Tension

Asked about the tension between students training smarter models and those same models threatening entry-level jobs, Garrett lands firmly on the optimistic side. He believes AI accelerates every person's ability to create economic impact, and says the doom about young people having no jobs isn't what Handshake hears from the Fortune 500 employers on its platform.

  • Garrett is in the 'GDP growth over universal basic income' camp.
  • 100% of the Fortune 500 uses Handshake to hire young people.
  • Employers are not reporting that young people won't have jobs.
  • One AI-native person can now do the work that used to take a whole team.
  • Jobs will evolve; people will need to upskill and reskill.

I'm brought in the camp of like GDP growth over like universal basic income.

Garrett Lord · 24:30

that's not what we're hearing from our employers.

Garrett Lord · 25:00
#ai-jobs#economy#hot-take#future-of-work
Hot Take26:30

Why AI-Native Young People Have a Huge Advantage

Garrett argues that being AI-native is the new resume superpower, comparing it to when people once listed 'Google search' as a skill. Young people who grew up with these tools come in as high-output operators, and those training the models get an extra edge.

  • People used to list Google search as a skill on their resume.
  • Being AI-native is like wearing an 'Iron Man suit' at work.
  • Young people are at a huge advantage because they grew up with the tools.
  • Fellows bring insights from model training back into their classrooms and research.

you used to put like Google search on as like a skill on your resume, right?

Garrett Lord · 26:30

I think being like AI native and having your Iron Man suit on and understanding how to leverage these tools is like uh young people…

Garrett Lord · 27:00
#ai-native#future-of-work#young-talent#hot-take
Hot Take30:30

The Only Moat in Human Data Is Access to an Audience

Garrett states Handshake's core strategic thesis. Competitors spend millions on Instagram, TikTok, Google, and YouTube ads plus armies of LinkedIn recruiters to source experts they can't target well. Handshake already has a trusted, targetable audience of 18 million professionals, which lets it reach scale and quality faster than anyone.

  • The only moat in human data is access to an audience.
  • Rivals run TikTok, Instagram, Google, and YouTube ads plus 100+ recruiters on LinkedIn.
  • They can't target a physics PhD well on an Instagram feed and have no brand recognition.
  • Handshake built a decade of trust with 18 million people and has rich academic profile data.
  • That lets Handshake target effectively and scale high-quality data faster than competitors.

the only the only moat in human data is access to an audience.

Garrett Lord · 30:30
#strategy#moat#distribution#human-data
Hot Take56:00

Leave Nothing to Chance: A Once-in-a-Lifetime Demand Moment

Garrett describes the culture and mindset driving the new business. He drew the number of days left in the year on a whiteboard, convinced he'll never again see a market with this much unlimited demand. The motto is 'leave nothing to chance': get on the plane, make the late-night push, check the data six times.

  • The team's motto is 'leave nothing to chance.'
  • Garrett drew the number of days in the year on a whiteboard.
  • He believes he'll never again see a business with this much unlimited demand.
  • Success comes down to execution: get on the plane, make the late-night push, ship the extra feature.
  • A flat, celebratory culture calls out the people 'putting up points.'

there will never be a time like this. I've never seen anything like it. I doubt I'll ever feel anything like this in business again…

Garrett Lord · 56:30
#culture#execution#hot-take#leadership

Explainer· 3

Explainer06:00

Pre-Training vs Post-Training: Where Model Gains Now Come From

Garrett breaks down the two phases of training a frontier model. Pre-training was about ingesting the entire corpus of human knowledge on the internet, but gains from that have plateaued. Most of the improvement now comes from post-training, where labs collect high-quality data to improve specific capability areas like coding, math, and law.

  • Training has two primary phases: pre-training and post-training.
  • Pre-training ingested the whole internet, every YouTube video and book.
  • Gains from pre-training began to asymptote 18-24 months ago.
  • Post-training augments and improves data across each capability area a lab cares about.
  • Post-training data can be RL environments, trajectories, prompt-response pairs, or human preference ranking.

There's a pre-training and a post-training process in training a model.

Garrett Lord · 06:00

And about 18 months ago, 24 months ago, we started to really see like an asmtoing of gains coming from because they had essentially like…

Garrett Lord · 06:30
#ai-training#llms#post-training#data-labeling
Explainer12:00

How PhDs Actually 'Break' AI Models

Garrett explains what expert labelers do day-to-day: probe a model until it fails, then provide the ground-truth answer and the correct step-by-step reasoning. The average person can't reliably break a frontier model, but a domain PhD can find where its reasoning steps go wrong. He points to the public GPQA paper as a concrete example.

  • Experts break the model, provide the ground-truth answer, and supply the correct reasoning steps.
  • Models are non-deterministic, so a model may get an answer right once but not three of five times.
  • The focus is often on fixing the intermediate reasoning steps, not just the final answer.
  • The GPQA paper is a public reference for how this works.
  • The average person cannot break the models; a domain PhD can.

I wouldn't say it's easy for them, but the average person cannot break the models.

Garrett Lord · 12:30

A great example is a public paper called like GPQA.

Garrett Lord · 13:00
#ai-training#reasoning#gpqa#experts
Explainer17:30

What a 'Trajectory' Means in AI Training Data

Garrett defines a term he uses repeatedly: a trajectory. It's the full capture of a human solving a problem end-to-end, including their screen, mouse movements, and voiceover narration. The output of this work is typically JSON data.

  • A trajectory captures the entire environment of what you're doing.
  • It includes your screen, your mouse, and your voiceover.
  • The output of the work is JSON data.
  • Labs want to understand how humans actually think and problem-solve, including how they handle roadblocks.

A trajectory is basically just like the entire environment that is collecting what you're doing.

Garrett Lord · 17:30
#ai-training#trajectories#reinforcement-learning

Story· 3

Story16:00

Rachel: The Eighth-Grade Teacher Training AI on Education

To make expert labeling concrete, Garrett describes Rachel, a PhD from the University of Miami who taught eighth grade for two decades and was an adjunct professor in education. She interacts with state-of-the-art models on educational design, spotting where they get teaching wrong in a non-verifiable domain.

  • Education is a 'non-verifiable' domain with no single correct answer.
  • Rachel has a PhD in education plus 10+ years of hands-on eighth-grade teaching.
  • She helps models understand the forefront of educational design.
  • The example spans from non-verifiable domains all the way down to verifiable engineering problems.

So there's like a PhD student uh Rachel on the network. She got her PhD from the University of Miami.

Garrett Lord · 16:00
#data-labeling#education#experts#case-study
Story33:30

How Handshake Discovered a New Business Hiding in Its Network

Garrett tells how the AI data business emerged out of the existing $200M Handshake. Middleman companies started asking to recruit Handshake's PhDs, and Handshake noticed the experience for those experts was frustrating. When frontier labs began reaching out directly to cut out the middleman, Garrett saw the opening and went zero-to-fifty-million ARR in four months.

  • Middleman companies started asking to recruit Handshake's PhDs and master students.
  • Users reported a frustrating, transactional experience on other platforms.
  • Frontier labs began reaching out directly to cut out the middleman.
  • Garrett started flying around over Christmas and New Year's chasing leads.
  • The business hit $50M ARR in four months and is now working with seven frontier labs.

we started to see all the what I would call like middleman companies reaching out to us saying, can we recruit your PhDs and master…

Garrett Lord · 34:00

zero to 50 is pretty good in four months, I think.

Garrett Lord · 36:30
#startup-story#0-to-1#handshake#growth
Story1:06:00

Almost Arrested at Princeton for Showering in the Pool

In the lightning round, Garrett shares an early hustle story. To save money while pitching campuses, he and his co-founders slept in a Ford Focus in McDonald's parking lots and showered in university pools. Princeton campus security didn't appreciate it, but the incident actually made the sales meeting more exciting and showed the school his commitment.

  • The team slept in a Ford Focus in well-lit McDonald's parking lots with good Wi-Fi.
  • They freshened up by showering in university pools, which are open and always have showers.
  • Princeton campus security did not appreciate a non-student showering there.
  • Security called the career services director to ask who Garrett Lord was.
  • The stunt showed a level of commitment that excited the school he was selling to.

we were sleeping out of our car. We had this like Ford Focus. We would put 20 30,000 miles on it. Sleep in the back…

Garrett Lord · 1:06:00

the Princeton campus security did not appreciate me showering as a non- studentent.

Garrett Lord · 1:06:30
#startup-story#hustle#handshake#sales

Takeaway· 4

Takeaway10:30

The Labeling Market Shifted From Generalists to Experts

The old data-labeling market ran on low-cost international generalist labor doing basic tasks. As models got better, that work stopped adding value. Now labs need genuine experts across every economically valuable domain to push the frontier.

  • The old market used talented, low-cost international labor for basic generalist tasks.
  • Models got good enough that generalist labelers are no longer needed.
  • Labs now need experts across every capability area they care about.
  • Focus is on advanced STEM plus derivative fields like accounting, law, medicine, and finance.

the models have gotten so good that the generalists are no longer needed. Like what they really need is experts. experts across every area that…

Garrett Lord · 11:00
#data-labeling#experts#ai-labor
Takeaway20:00

The Three Things AI Labs Care About: Quality, Volume, Speed

Garrett zooms out to what model builders actually want from a data partner. Quality comes first because bad data is very hard to overcome, like teaching a student the wrong thing. Then volume in the most advanced domains, and speed, because researchers run many hypotheses at once and pour resources into whichever pipeline shows a gain.

  • Quality is first and foremost; wrong data is extremely hard to train out.
  • Volume matters: generating thousands of high-quality pieces in advanced domains.
  • Speed matters because labs test three or four bets at once and scale the winner.
  • Handshake reaches top-GPA physics students at Stanford, Berkeley, and MIT to hit volume fast.

They care about three things. They care about like quality first and foremost. You have to have high quality data.

Garrett Lord · 20:00

And then the other thing I'd say model builders care about is speed because they have all these hypotheses and they're constantly testing different pipelines.

Garrett Lord · 21:00
#data-labeling#ai-labs#quality#operations
Takeaway27:00

PhDs Make $150/Hour Breaking Models Instead of $25 as a TA

Garrett shares the economics for the expert fellows on the network. A PhD who might earn $25/hour as a teaching assistant can make $100-$200/hour in their field of expertise breaking frontier models. Beyond the money, fellows say the work helps advance their own research.

  • Fellows can make $100, $150, or $200 an hour in their field of expertise.
  • A teaching assistant might earn only $25 an hour by comparison.
  • Fellows bring insights back into the classroom to teach more effectively.
  • They learn to leverage the tools to advance their own research; it's getting paid to learn a skill.

you can make like 25 bucks an hour being a teacher assistant or you can actually make $150 an hour breaking the latest models.

Garrett Lord · 27:00

it is quite cool to get kind of paid to learn a skill.

Garrett Lord · 27:30
#experts#compensation#phds#data-labeling
Takeaway53:30

Separate Everything: Building 0-to-1 Inside a 10-Year-Old Company

Garrett explains what made incubating a new business inside a mature company work: radical separation. Separate engineering, design, finance, and operations teams; people with one job only; a different office, different comp tied to new-business hurdles, and a founder-mode CEO spending 80%+ of his time on it.

  • It's hard to incubate something new inside an established business.
  • Everything was separate: engineering, design, accounts, operations, and finance teams.
  • People had one job only, and it was making the new business successful.
  • The team sat in a separate part of the office, five days a week plus weekends.
  • Compensation was tied to hurdles in the new business so people felt like owners.
  • Garrett went founder-mode, spending 80%+ of his time and attention on it.

it's also like hard to like incubate something new inside of a business.

Garrett Lord · 39:30

I just really believe it's separate and everything like separate engineering team separate design team separate accounts and operations team separate finance team

Garrett Lord · 54:00
#0-to-1#founder-mode#org-design#leadership