LLenny's Podcast
← All episodes
Jessica Lachs (VP of Analytics and Data Science at DoorDash)14 July 2024

Building a world-class data org

8Frameworks
15Insights

Frameworks in this episode

Leadership5 steps

Central Org, Embedded Pods

Keep analytics reporting centrally, but mirror partner-team structure with pods that share their goals

Read →
Strategy5 steps

Fail-State Goaling

Name your worst-case experiences, set a team's goal on eradicating them, and hunt for the data you don't have

Read →
Leadership5 steps

Outcome Ownership Over Role Boundaries

Goal people on the outcome, not the job description, and let them cross any function to reach it

Read →
Innovation5 steps

Protected Deep-Dive Time

Goal the team on self-directed insight and use hackathons to stop exploratory work being eaten by inbound asks

Read →
Strategy5 steps

Proxy Metric Selection

Goal on simple short-term inputs you can prove drive the long-term output you actually want

Read →
Strategy5 steps

The Common Currency Model

Convert every lever in the business into one shared unit so cross-team tradeoffs become arithmetic

Read →
Leadership5 steps

The Curiosity Case Interview

Use a real historical company problem as the case, then test soft skills the candidate doesn't know you're testing

Read →
Communication5 steps

Tradeoff-Transparent Pushback

Never say no to a request — show the three things it displaces and let the asker decide

Read →

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Hot Take· 2

Hot Take05:00

Analytics Should Have a Seat at the Table, Not Be a Ticket Queue

Jessica Lachs argues analytics is a business-impact function, not a service org that answers Jira tickets and builds dashboards on request. Her team is expected to have a point of view on decisions and surface opportunities, not just explain the 'why' but recommend the 'so what do we do now'.

  • Analytics should sit alongside engineering, product, and operations with a real seat at the table
  • A pure service model, answering questions and building dashboards on demand, was never the team she wanted to build
  • The job is to find opportunities and bring a point of view, not just answer inbound questions
  • The team earns its seat by bringing opportunities everyone can go after

“for me analytics is a business impact driving function and not purely a service function”

Jessica Lachs · 05:00

“not just answering the why but answering the the so what so what do we do now that we know this”

Jessica Lachs · 05:30
#analytics#data-teams#leadership
Hot Take05:30

Why a Centralized Data Org Beats Embedding Analysts in Business Units

Lachs holds a contrarian view that a central 'Center of Excellence' model is superior to embedding analysts inside marketing, product, or ops. DoorDash divides its central team into pods that map to partner teams so analysts feel embedded day-to-day, but report up through a central org and share their partners' goals.

  • Central means reporting lines run through the analytics org, not that goals differ from partner teams
  • Marketing analytics is part of the broader analytics team rather than reporting into marketing
  • Pods map one-to-one to how product, engineering, ops, and marketing are structured
  • Shared goals with business partners keep incentives aligned so your success is their success
  • They experimented with the embedded alternative in the past and found it more problematic

“I strongly disagree I I believe a central Model A Center of Excellence is superior”

Jessica Lachs · 06:00
#org-design#data-teams#analytics

Explainer· 3

Explainer11:00

The Real Benefits of a Central Analytics Org

Lachs lays out why the centralized model wins: a consistent, high talent bar; more growth paths for analysts; shared methodologies and metric definitions; the ability to see problems across teams and get ahead of them; and a strong team culture and brand that helps recruiting and retention.

  • One consistent, higher talent bar and hiring rubric across the whole team
  • Growth opportunities across functions so a senior analyst who is bored in marketing can move to merchant analytics
  • Consistent methodologies and metrics, so 'sales' means one thing company-wide
  • Build one churn model with input from six teams instead of six separate ones
  • Seeing the same problems across teams helps you spot what to automate and see around corners
  • A proud team culture and brand that aids recruiting and gives analysts peers to learn from

“the third thing is just consistency of methodologies and metrics so you don't have sales that was as defined by one team and sales as…”

Jessica Lachs · 12:30
#org-design#hiring#data-teams#culture
Explainer25:30

How to Test for Curiosity and Judgment in Interviews

Lachs plants something not-quite-right inside a real DoorDash business case to see if candidates notice and investigate. She also watches how people react when told they're wrong, and pushes them to commit to a decision under incomplete information, because that's the real signal she's after even if it isn't what the literal question asks.

  • Embed something not quite right in the case and see whether candidates notice and where they go with it
  • Cases use real problems from DoorDash history to test structured problem-solving under ambiguity
  • Most candidates make a wrong assumption; how they respond to being corrected is a key signal
  • Push people to make a call under imperfect information because that's how real decisions work
  • Some questions assess a soft skill separate from the literal problem being solved

“have something that is not quite right within the case that you're presenting and see if people notice”

Jessica Lachs · 25:30

“I always push people to say if you had to make a call right now what would it be”

Jessica Lachs · 28:00
#hiring#interviewing#data-teams
Explainer46:30

Quantify Every Lever in a Common Currency

DoorDash converts every business lever, price, selection, quality, delivery time, into a common currency of gross order value and volume. Knowing what a dollar of price cut or a minute of faster delivery buys in volume lets teams like marketing and logistics make tradeoffs against each other and decide where to spend a dollar for the most return.

  • Translate disparate metrics into a shared unit so teams can compare across functions
  • Example: what does lowering price by a dollar, or delivery time by a minute, get you in volume?
  • A common currency lets marketing and logistics trade off against each other
  • Everything ultimately maps to gross order value (GOV) and volume
  • Having an inventory of levers and their payoffs speeds up and improves decisions

“if I were to lower price by a dollar what would I get in terms of we'll say volume”

Jessica Lachs · 46:30
#metrics#marketplace#decision-making

Story· 4

Story17:30

The Cupcake Fraud Deep Dive That Fixed DoorDash's Referral Channel

A hackathon deep dive found referrals looked below-average as an acquisition channel, but the team refused to just cut spend. They committed referral fraud themselves, ordering cupcakes over and over, and discovered the channel was bimodal: great customers referring great customers, versus discount-hunters posting codes online. The fix was fraud caps and better checks, and a lesson that averages hide bimodal distributions.

  • Referrals looked below-average on engagement and payback versus other channels
  • The team created new accounts and committed referral fraud, ordering cupcakes repeatedly, to see how the abuse worked
  • The channel was bimodal: strong customers referring strong customers, and discount-hunters posting codes online
  • Lax fraud rules and no caps let the bad group drag down the whole channel's efficiency
  • Outcome: better fraud checks and caps on referrals, plus a reminder that the average can be misleading

“we actually tried referring each other we tried committing referral fraud creating new accounts to get around rules and we uncovered a lot of fraudulent…”

Jessica Lachs · 18:00

“the average can be incredibly misleading and so looking at distributions”

Jessica Lachs · 20:00
#growth#fraud#referrals#data-analysis
Story28:30

The Head of Analytics With No Data Science Degree

Lachs jokes she has a job she'd never be hired for, because she has no traditional data science background, an art portfolio in school and a finance start at Lehman Brothers. She taught herself SQL and Python out of necessity in DoorDash's early days, and argues that finance-rooted pragmatism plus hiring people smarter than her keeps the team focused on business impact.

  • No formal data science training; background is art and investment banking
  • Self-taught SQL and Python because DoorDash needed someone to set and measure market goals ten years ago
  • Founder Tony Xu recognized the aptitude despite the lack of formal training
  • Her non-technical background lets her hire PhDs and ML experts while keeping them focused on business impact

“I sort of joke that I have a job I'd never be hired for”

Jessica Lachs · 29:00

“I became a data scientist out of necessity uh I completely self-taught uh in terms of SQL and Python”

Jessica Lachs · 29:30
#career#data-science#self-taught
Story33:30

Extreme Ownership: Garbage Duty, Cupcakes, and Dashing Pizza

Lachs traces DoorDash's culture of extreme ownership to founder Tony Xu, from taking out the garbage on Saturday nights to a sales rep handing out promo codes at 5am despite being comped only on signing merchants. During an early site outage the whole 20-person company jumped on customer support, and Lachs, too new to use the tools, went dashing to deliver pizza to the office herself.

  • Ownership means doing whatever the outcome needs regardless of your title
  • In 2014 Boston, a four-person team handed out promo cards attached to KIND bars outside the T at 5am
  • Sales rep Joey Graziano joined the promo handouts even though he was only comped on signing merchants
  • During a major early outage, the whole ~20-person company jumped on phones to do refunds and support
  • Lachs went dashing to deliver pizza so three Dashers wouldn't have to, one of the largest refund nights relative to their bank balance

“roll up your sleeves do something that's not your job I think back to you know early days of taking out the garbage on Saturday…”

Jessica Lachs · 34:00

“I was like I'm going to go out go out dashing go get everyone pizza so that we could kind of feed the masses”

Jessica Lachs · 37:30
#culture#ownership#startups#doordash
Story45:30

Why the Merchant Health Composite Score Failed

Data scientists love composite metrics with weighted coefficients, but nobody understands whether a 0.1 move is good. DoorDash's Merchant Health score, blending active hours, photos, and menu completeness into one weighted number, became meaningless. They replaced it with a few simple, understandable metrics the team could actually goal against, even if that meant three metrics instead of one.

  • Composite metrics with coefficients are hard to interpret; a 0.1 change tells you nothing intuitive
  • The Merchant Health score bundled active hours, images, and full accurate menus into one weighted number
  • A score of '35' was meaningless because nobody knew what it represented or how to move it
  • They switched to concrete inputs: measure new-merchant first-order rate and goal teams on photo coverage and accurate open hours
  • A simple metric people understand and can talk about beats a more 'perfect' composite nobody grasps
  • Getting 95% right with three clear metrics beats one opaque composite

“I always encourage folks just pick something simple even if it's not perfect”

Jessica Lachs · 46:00

“it was so hard to understand what it was and how to move it that it it became meaningless”

Jessica Lachs · 52:00
#metrics#simplicity#analytics#merchants

Tool· 1

Tool1:02:30

'Ask Data AI': Empowering Non-Technical Users to Self-Serve

DoorDash's analytics team has run weekly office hours for eight years to teach SQL and act as thought partners. Lachs is now excited about an internal AI tool, 'Ask Data AI', that lets non-technical users edit and adapt SQL queries themselves, freeing analyst bandwidth. It's named after the internal Slack channel, in keeping with DoorDash's habit of very literal naming.

  • Analytics office hours have run two hours weekly for eight years across time zones
  • The AI tool lets users adapt an existing query, e.g. to the grocery business, on their own
  • Goal is to empower non-technical users to self-serve rather than consume analyst bandwidth
  • It's named 'Ask Data AI' after the internal Slack Q&A channel, favoring clarity over cleverness

“it's not clever it's called ask data AI and that's named for our internal slack channel”

Jessica Lachs · 1:04:30
#ai#tools#self-serve#sql

Takeaway· 5

Takeaway15:30

Be Intentional About Protecting Time for Exploratory Work

Exploratory deep dives are the first thing to get cut when inbound requests pile up, because a known deliverable feels higher expected-value than an uncertain-ROI investigation. Lachs protects that time deliberately, setting team goals around self-directed insight work and running hackathons to carve out days for it.

  • Deep dives are always the first thing sacrificed when inbounds surge
  • Set explicit team goals around finding insights through self-directed work to stay accountable
  • Hackathons carve out dedicated days to chase interesting threads and find opportunities
  • Business partners support and even encourage this time because so many road-map-driving insights came from it

“you have to be very intentional to carve out time for exploratory work for deep Dives”

Jessica Lachs · 15:30

“we would do hackathons for our team to carve out days to just go and look into these really interesting things and find Opportunities”

Jessica Lachs · 16:30
#analytics#productivity#team-management
Takeaway20:30

How Data Teams Should Push Back: Share the Tradeoffs

Rather than flatly saying no or suffering in silence, Lachs recommends making tradeoffs explicit. Because her team shares goals with business partners, they can ask whether a new data pull is more important than the three things already planned, which often reveals the ask can wait.

  • Saying no is hard, especially for people-pleasers when the ask is quick and easy
  • Leadership should set the operating model so junior folks aren't forced to always say no
  • Shared goals let you frame requests as 'is this more important than these other three things?'
  • Making tradeoffs front and center often surfaces that the ask actually isn't important
  • Occasionally just knock out a quick request to build goodwill

“always share the tradeoffs don't kind of suffer in silence”

Jessica Lachs · 22:00
#prioritization#team-management#analytics
Takeaway24:30

The One Trait She Hires For: Curiosity You Can't Teach

Beyond a table-stakes technical bar, the differentiator among top analysts is curiosity and self-motivation, the instinct to pull on a thread when something looks off rather than stopping once the question is answered. Lachs says she hasn't found a way to teach it.

  • Technical skills are table stakes, screened separately
  • Curiosity and self-motivation are the standout traits in top talent
  • The valuable person digs into something that seems odd even after they've technically finished the task
  • Curiosity, in her experience, can't be taught

“the first thing is just curiosity you you can't teach curiosity or at least I I haven't found way to do it”

Jessica Lachs · 24:30
#hiring#data-teams#curiosity
Takeaway44:30

Goal on Short-Term Proxies, Not Retention

The core lesson Lachs draws from bad metrics: find a short-term metric you can move that drives a long-term output. Retention is a terrible thing to goal on because it's nearly impossible to move meaningfully in the short term, so you should identify and experiment on its input drivers instead.

  • You learn the most about metrics from picking the wrong ones
  • Find a short-term measurable metric that drives the long-term output you actually care about
  • Retention is a poor goal metric because it can't be moved meaningfully in the short term
  • Isolate the inputs that drive retention and test whether moving them moves the long-term output

“ultimately you want to find a shortterm metric you can measure that drives a long-term output”

Jessica Lachs · 44:30

“retention is a terrible thing to goal on because it's like it it it's almost impossible to to drive in a meaningful way”

Jessica Lachs · 45:00
#metrics#analytics#retention
Takeaway55:00

Set Goals Around Fail States, Not Just Averages

Rare disasters like a 'never delivered' order vanish inside average quality metrics, yet they drive churn and cost far more than their frequency suggests, refunding the order, repurchasing food, and sending another Dasher. Lachs makes eliminating these fail states an explicit goal, and notes some failures, like login errors, never even appear in the data because affected users drop out of the denominator.

  • Rare edge cases and fail states are invisible when you only track averages
  • 'Never delivered' orders are rare but drive churn and cost far more than their frequency implies
  • A dedicated team with product, engineering, and ops has the explicit goal to eradicate never-delivered
  • Churn costs are unobserved: you see one bad order, not all the future orders you lost
  • Login failures don't show up in the data because users who can't log in never enter the dataset
  • Data folks should ask what data they're missing, not just analyze what they have

“we have this concept of never delivered which is orders that are never IED were really great at naming things at door Dash”

Jessica Lachs · 55:30

“their goal is eradicate never delivered”

Jessica Lachs · 57:00
#metrics#quality#churn#analytics