LLenny's Podcast
← All episodes
Crystal Widjaja, Gojek and Kumu31 July 2022

How to scrappily hire for, measure, and unlock growth

7Frameworks
14Insights

Episode overview

Crystal Widjaja explains how Gojek used unusually scrappy, Wizard-of-Oz experiments to validate growth ideas before investing in full product builds. She presents a constraint-led growth model grounded in a company’s market, product, business model, and available channels, then shows how better instrumentation turns observations into actionable insights. The conversation also covers retention benchmarks, early growth-team hiring and structure, and Generation Girl’s work expanding access to STEM education in Indonesia.

Key ideas

  • Test the intended user experience as quickly as possible, even with manual operations, screenshots, surveys, or very small samples.
  • Model growth using the existing physics of the market, product, business model, and channels, then change one constrained lever at a time.
  • Look immediately before conversion or churn for trust gaps, friction, slow workflows, weak copy, and temporary problems that need temporary solutions.
  • Retention is too broad to diagnose directly; teams should identify the specific preceding behaviors and context that cause users to return.
  • Analytics events need rich contextual properties so teams can ask why behavior occurred and turn measurements into decisions.
  • Early growth hires should understand statistics, sampling, selection bias, experiment design, and the opportunity cost of slow tool adoption.
  • Growth teams work best when they begin with known, tractable gaps rather than being asked to model the entire business from scratch.
  • Generation Girl gives young women agency to explore STEM through free classes, teacher resources, and partnerships in Indonesia.

Transcript available · source text is retained privately and is not published

Frameworks in this episode

Strategy3 steps

Experiment at 30: Trend Over Precision

You can run meaningful experiments with as few as 30 users — more data buys precision, not different trends.

Read →
Marketing4 steps

Familiarity-Anchor Copy for New Concepts

When users land but don't act, the product isn't wrong — anchor unfamiliar concepts to something they already know.

Read →
Leadership4 steps

Growth-as-Cleanup-Crew Team Design

Point your first growth hire at known gaps, don't ask them to discover strategy from scratch.

Read →
Productivity4 steps

Measurements vs. Insights Instrumentation

A measurement is an observed fact; an insight answers 'why' and changes what you do — instrument for the latter.

Read →
Strategy4 steps

Physics-Then-Loops Growth Modeling

Map the fixed physics of your business first, then find the one underused lever inside them to change.

Read →
Strategy4 steps

Retention Benchmark Ladder

Concrete week-one cohort retention floors: ~80% friends/family, 60% free at scale, 20-30% paid.

Read →
Innovation4 steps

The Wizard of Oz Validation Method

Validate a feature's value and conversion rate before building anything by faking the backend manually.

Read →

People & resources mentioned

Attributed to the moment in the episode. Timestamps are approximate.

People · 5

  • Crystal WidjajaMentionswho's a long time Chief product officer at gojek she's currently Chief product officer at kumu

    “crystal wajaya has been leading product and growth teams at some of the largest consumer businesses in southeast asia”

    ▶ Watch at ~0:30approx.
  • Crystal WidjajaMentionswho's a long time Chief product officer at gojek she's currently Chief product officer at kumu

    “crystal thank you so much for being here”

    ▶ Watch at ~3:30approx.
  • Warren BuffettMentionsWarren Buffett who's outstanding arguable genius

    “so i think taking that warren buffett approach i knew that the product made sense the market made sense as well”

    ▶ Watch at ~10:30approx.
  • Darius ContractorMentionsmentioned

    “i'd like to see founders think about the user psych model that dorias contractor often talks about”

    ▶ Watch at ~21:30approx.
  • Lenny RachitskyMentionstoday we've got another very special compilation episode something I've been pulling on more and more with the podcast and the newsletter

    “thanks lenny this was a blast”

    ▶ Watch at ~1:02:30approx.

Resources · 29

  • GojekMentionscompany · Kevin Aluwi and co-founders

    “gojek where she built and led the growth team through the early years”

    ▶ Watch at ~0:30approx.
  • KumuMentionscompany

    “including kumu where she's currently the chief product officer”

    ▶ Watch at ~0:30approx.
  • FlatfileMentionssoftware · Flatfile

    “customers like square spotify and zora are able to grow their businesses on top of flat file”

    ▶ Watch at ~3:00approx.
  • ClubhouseUsessoftware · Alpha Exploration Co.

    “we may have like cross paths on clubhouse and the audio forums”

    ▶ Watch at ~3:30approx.
  • MySQLUsessoftware

    “all of the work needed to build a mysql database”

    ▶ Watch at ~6:00approx.
  • KumuUsescompany

    “with kumu it's more so of a i want to do clubhouse zoom google hangouts gather round all in one app”

    ▶ Watch at ~9:00approx.
  • WhatsAppUsessoftware · Meta

    “why don't we just add them to a whatsapp group”

    ▶ Watch at ~14:00approx.
  • TypeformUsescompany

    “we were sending out lots of fake features through things like type form surveys”

    ▶ Watch at ~15:30approx.
  • GoFoodUsesproduct · Gojek

    “for things like go food it would be things like when does a user try a new merchant”

    ▶ Watch at ~19:00approx.
  • Facebook ConnectUsessoftware · Facebook

    “we actually hacked it by connecting people's facebook connect login”

    ▶ Watch at ~19:00approx.
  • Data for PMsCoinedcourse · Crystal Widjaja and Reforge

    “until i worked with reforge to build my data for pms program”

    ▶ Watch at ~28:00approx.
  • ReforgeUsescompany · Brian Balfour

    “until i worked with reforge to build my data for pms program”

    ▶ Watch at ~28:00approx.
  • GoPayUsesproduct · Gojek

    “has this customer ever used our gopay product before did they have a digital balance”

    ▶ Watch at ~32:30approx.
  • AB InBevMentionscompany

    “for one of the companies i work with abn bev they run a lot of their d2c brands”

    ▶ Watch at ~38:00approx.
  • AmplitudeRecommendssoftware · Amplitude

    “amplitude actually has a pretty good long-winded documentation on this on how to do event tracking”

    ▶ Watch at ~46:30approx.
  • Google Data StudioRecommendssoftware · Google

    “usually i say like you can probably get by with google data studio it's free”

    ▶ Watch at ~47:00approx.
  • MetabaseRecommendssoftware

    “if not metabase has a great open source free tool if you have someone who can write sql”

    ▶ Watch at ~47:00approx.
  • CleverTapRecommendssoftware

    “if you need in-app mobile device event tracking i usually recommend clevertap”

    ▶ Watch at ~47:30approx.
  • MixpanelMentionssoftware · Mixpanel

    “because mixpanel has unfortunately failed me a lot”

    ▶ Watch at ~47:30approx.
  • SegmentRecommendssoftware · Twilio

    “then i would get into segment”

    ▶ Watch at ~47:30approx.
  • EppoRecommendssoftware · Eppo

    “i usually look at something like epo to just automate the decision-making flow”

    ▶ Watch at ~48:00approx.
  • Twilio APIUsessoftware · Twilio

    “connect a quick python script to the twilio api that we had access to”

    ▶ Watch at ~54:30approx.
  • Generation GirlCoinedcompany · Crystal Widjaja and co-founders

    “yes generation girl is very near and dear to my heart so i co-founded this with a couple of amazing women”

    ▶ Watch at ~57:00approx.
  • Legally BlondeRecommendsfilm

    “legally blonde is one of my favorite movies that represents you can take the powers that you have”

    ▶ Watch at ~57:30approx.
  • FigmaUsessoftware · Figma

    “how do you teach how to use figma and things like that”

    ▶ Watch at ~58:30approx.
  • MIT SolveMentionsevent · MIT

    “most recently we're part of the mit solve program with our new initiative class so plus”

    ▶ Watch at ~59:30approx.
  • Carnegie Mellon UniversityMentionscompany

    “we've seen studies at carnegie mellon that actually would create introductory computer science classes before the college class starts”

    ▶ Watch at ~1:01:30approx.
  • Generation GirlMentionscompany · Crystal Widjaja and co-founders

    “they can reach you on generationgirl.com generationgirl.org”

    ▶ Watch at ~1:02:00approx.
  • Lenny's PodcastCoinedpodcast · Lenny Rachitsky

    “you can find all past episodes or learn more about the show at lennyspodcast.com”

    ▶ Watch at ~1:03:00approx.

Spot an error or want something removed? Request a correction or removal.

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Hot Take· 1

Hot Take17:00

You Can Run Real Experiments With Just 30 People

Crystal pushes back on the idea that startups are too small to experiment. More data increases the precision of your trends, not the direction of them, so early-stage teams still learn the underlying signal. Her framing: what's better than 30 data points is 100, but what's better than zero is definitely 30 and cheap-to-run tests deliver disproportionately at that scale.

  • More data improves precision, not the underlying trend direction.
  • With a small sample you get the same trends, just less precisely.
  • Look for a large proportion (e.g. ~20 of 30) doing the target behavior.
  • Cheap non-scaling tests deliver more per experiment at 30 people than at 100.

“so what's better than having 30 data points certainly having 100 but what's better than having zero is definitely 30.”

Crystal Widjaja · 18:00

“even if you have a sample size of the data you get back generally does not change but its precision will”

Crystal Widjaja · 17:30
#experimentation#statistics#early-stage#growth

Explainer· 4

Explainer11:30

Why a Super App Hasn't Emerged in the US

Crystal explains why the super-app model dominates Southeast Asia but not the US. Consumers there grew up trusting integrated conglomerates, whereas Americans are wary of any one company knowing too much. She adds a hardware angle: many users leapfrogged straight to phones with limited storage, so when the phone fills up they delete apps rather than photos, forcing consolidation into one app.

  • Southeast Asia has cultural trust in conglomerates that own malls, housing, and schools.
  • Americans shy away from one company knowing too much (e.g. "does Google know too much about me").
  • Many Asian users leapfrogged to phones without owning a computer.
  • Limited device storage forces users to delete apps, so surviving apps must be super apps.

“there's a sense of trust in a conglomerate whereas in america we already kind of shy away from like does google know too much about…”

Crystal Widjaja · 11:30

“when your phone is full are you going to delete a photo of your kid or are you going to delete this app you're probably…”

Crystal Widjaja · 12:00
#super-app#strategy#southeast-asia#market
Explainer40:00

Treat Metrics as News, Not Entertainment

Crystal's core diagnosis of why analytics efforts fail: people track data to watch their OKRs go up or down, but never use it to find insights. She likens it to using Twitter for "news" when you're really using it for entertainment consuming interesting facts without acting on them. Real news is information that changes what you do in the real world; if nothing changes, you were just entertained.

  • Most teams track data to monitor OKRs, not to identify actionable insights.
  • Consuming metrics passively is entertainment, not news.
  • Real news is information that changes what you do in the real world.
  • If a data point doesn't change your behavior, it delivered no value.

“real news is information that changes what you do in the real world”

Crystal Widjaja · 41:00

“do not treat metric gathering as entertainment like it's not there for you to be like oh that's interesting how novel and then not act…”

Crystal Widjaja · 41:00
#analytics#metrics#instrumentation#insights
Explainer41:30

Measurements Aren't Insights: The Girlfriend's Cousin Analogy

Crystal distinguishes measurements from insights. A measurement is a raw observation ("power users do 4x more bookings") that your database obviously confirms but doesn't tell you why or what to do. Her analogy: seeing your girlfriend with an unknown man is an observed fact; the hypothesis is cheating, but the insight might be that he's her cousin and you're paranoid. Insight arrives once you answer the "why" and can act differently, like giving power users a free-shipping discount only on high-GMV baskets.

  • A measurement is an observation; an insight answers why and tells you how to act.
  • "Power users do 4x more bookings" is a fact, not an insight, without context.
  • The girlfriend-and-cousin analogy shows how the same observation yields different insights.
  • Real insight comes from instrumenting event properties to segment behavior, then testing a hypothesis.

“measurements do not equate to insights a measurement would be an observation it's a data point in your database”

Crystal Widjaja · 41:30

“your hypothesis could be that your girlfriend is cheating on you but the insight uh the actual fact might be that she's not cheating on…”

Crystal Widjaja · 42:00
#analytics#insights#instrumentation#segmentation
Explainer44:30

The Telltale Sign of Bad Event Instrumentation

Crystal can spot weak analytics just from the instrumentation spec: lots of events with only one property or none. Using a Gojek "map loaded" event, she shows the properties that matter number of drivers visible, pickup city, lat/long, surge pricing, minimum fare, voucher code. Those let you learn that a user seeing two drivers converts far less than one seeing five, then dig into where and why. Without that context, you can't explain non-conversion.

  • Bad tracking looks like many event rows each carrying one property or none.
  • A "map loaded" event should capture drivers visible, city, lat/long, surge, fare, voucher.
  • Rich properties reveal that seeing 2 drivers converts worse than seeing 5.
  • Missing context makes it impossible to ask or answer why a user didn't convert.

“the symptom of a bad data tracking approach is you have a ton of rows with a ton of events but every event has like…”

Crystal Widjaja · 44:30

“when a user only sees two drivers on the screen they're much less likely to convert than a user who sees five drivers on a…”

Crystal Widjaja · 45:30
#instrumentation#event-tracking#analytics#data

Story· 6

Story00:00

Renting a Stadium to Hire 60,000 Gojek Drivers in Weeks

Crystal recounts one of Gojek's most audacious early growth moves: renting an actual stadium, staffing long lines with boxes of pre-loaded phones and SIM cards, and onboarding tens of thousands of motorcycle-taxi drivers in a couple of weeks. She admits the risk of joining felt huge when she arrived to find the company operating out of a house, but the product was already growing fast at ~4,000 orders a day.

  • Gojek ran mass driver hiring by literally renting a stadium (a couple of football fields).
  • Roughly 60,000 drivers were onboarded in a couple of weeks.
  • Setup was long lines, boxes of phones and SIM cards handed to drivers.
  • Company was still tiny (in a house) but already growing at ~4,000 orders/day.

“we ended up renting a stadium to just hire like 60 000 drivers in a couple of weeks”

Crystal Widjaja · 00:00

“when i got there it was in a house and i realized i've probably made a huge mistake”

Crystal Widjaja · 00:00
#growth#hiring#gojek#scrappy#operations
Story13:00

Wizard-of-Oz Tests: Validating Features Without Building Them

Crystal describes Gojek's scrappy playbook for testing ideas before writing code. To test a subscription feature, they added 100 drivers to a WhatsApp group, had them sell the package on rides, and manually processed vouchers in the backend. For a new onboarding screen, a designer overlaid a mockup on a screenshot and shipped it as an in-app message. The point: manifest the desired user experience as fast as possible, then build only what works.

  • Tested a subscription feature via a WhatsApp group of 100 drivers selling manually, with interns processing vouchers in the backend.
  • Faked a new onboarding screen by overlaying a designer mockup on a screenshot and sending it as an in-app message.
  • Used Typeform surveys and quizzes as fake features to gauge demand.
  • Built in-app web deployment so web-testable features didn't wait on mobile app releases.

“this concept of doing things that are somewhat crazy but validate a point doing stuff that don't scale especially i think is really the bread…”

Crystal Widjaja · 13:30

“it's really this wizard of oz experience we don't have to build anything”

Crystal Widjaja · 14:30
#experimentation#scrappy#wizard-of-oz#product#gojek
Story19:00

Hacking Trust: Using Friends' Orders to Sell New Restaurants

For GoFood, Crystal's team realized users needed trust before ordering from an unfamiliar merchant. Using existing Facebook Connect permissions, they surfaced food that a user's friends had purchased and liked. Users shown a friend's purchase were twice as likely to order from a brand-new restaurant lifting GMV by solving for trust rather than optimizing conversion directly.

  • Diagnosed that trust, not conversion mechanics, was blocking orders from new merchants.
  • Leveraged existing Facebook Connect permissions to see friends' past purchases.
  • Showed users food their friends had bought and liked as social proof.
  • Users with the feature were 2x more likely to order from a brand-new restaurant, raising GMV.

“we actually looked at the food that their friends had purchased and used that as a data set of hey here's food that lenny purchased…”

Crystal Widjaja · 19:30

“you would be twice as likely to purchase from a brand new restaurant than users who did not have this feature”

Crystal Widjaja · 19:30
#growth#trust#social-proof#gofood#conversion
Story32:00

Turning Drivers Into GoPay Salespeople With a Captive Audience

To drive GoPay adoption, Crystal's team built a small service that detected when a driver was allocated a customer with no digital balance, then messaged the driver an incentive to get the customer to top up with cash. Trapped together in the car, drivers became remarkably effective salespeople and customers felt the benefit explained directly. It became ~60% of acquisition with no change to the product's physics, just a new use of an existing lever.

  • A backend service checked whether an allocated customer had ever used GoPay.
  • If not, it messaged the driver an incentive to convert them via cash top-up.
  • Drivers' captive attention in the car made them highly effective salespeople.
  • Drove ~60% of GoPay acquisition once released, with no change to product physics.

“you wouldn't believe how great of a salesperson someone can be when you were literally trapped in a car with them going somewhere”

Crystal Widjaja · 33:00

“it was huge it was like 60 of acquisition once we released that”

Crystal Widjaja · 33:30
#growth#gopay#incentives#distribution#gojek
Story38:00

Beat Subscription Churn With a Pause Button, Not a Cancel

Advising a beer D2C brand, Crystal found the top cancellation reason was "I still have too much beer." The app let users cancel or resume but not pause a permanent solution to a temporary problem. Adding a pause button alleviated churn that was otherwise hard and expensive to reacquire. Lenny notes the same pattern powered one of Airbnb's biggest wins: a snooze/pause feature for listings.

  • Top cancellation reason for a beer subscription was simply having too much beer.
  • App only allowed cancel or resume, forcing a permanent fix for a temporary problem.
  • Adding a pause button solved churn at the exact constraint where users dropped off.
  • Airbnb's listing snooze/pause was one of its biggest wins for the same reason.

“canceling the subscription is a permanent solution to having too much beer how do you make a temporary solution that solves the actual problem”

Crystal Widjaja · 38:30

“this is actually one of the biggest wins is adding a snooze feature to your listing”

Lenny Rachitsky · 39:30
#churn#retention#subscriptions#product#d2c
Story53:30

Hiring Growth: A Take-Home Experiment-Design Case Study

To hire growth people, Crystal screens for first-principles bias with a case study: given a claimed result, how do you know it's true, and how would you design an experiment to test it? She wants candidates who sample randomly and take a measured approach, not those who assume their feature will work. She gives a take-home (about four hours over five days), not a live test, and specifically values candidates who admit they had to Google it and read white papers.

  • Screens for first-principles bias: "how do you know this is true?" plus an experiment design.
  • Wants random sampling and a measured, deliberate approach over assuming a feature will work.
  • Uses a take-home (~4 hours over 5 days), not a live test, to see real quality.
  • Values candidates who admit they Googled it and read white papers to figure it out.

“i actually look for that first principle bias so i'll give people case studies of like here's what we see how do you know that…”

Crystal Widjaja · 54:00

“we like to hear people say that they literally had to google this and read a bunch of white papers”

Crystal Widjaja · 56:30
#hiring#growth-team#interviewing#experiment-design

Tool· 1

Tool47:00

Crystal's Recommended Analytics Stack by Company Stage

Crystal gives a stage-based tooling recommendation. Very early with a single data warehouse: Google Data Studio (free), or Metabase's open-source tool if you have SQL skills or multiple databases. For in-app mobile event tracking she recommends CleverTap over Mixpanel (which failed her) and Amplitude (which lacks the CRM components). Bigger needs point to Amplitude plus Segment, and for experimentation she shouts out Eppo to automate the decision-making flow.

  • Single warehouse / very early: Google Data Studio (free) or Metabase (open source, needs SQL).
  • In-app mobile event tracking: CleverTap preferred over Mixpanel and Amplitude.
  • At larger scale: Amplitude for analytics juice plus Segment to reduce ETL.
  • Experimentation: Eppo to automate the decision-making flow.

“if you need in-app mobile device event tracking i usually recommend clevertap because mixpanel has unfortunately failed me a lot and amplitude doesn't have the…”

Crystal Widjaja · 47:30
#tools#analytics#stack#experimentation

Takeaway· 2

Takeaway25:00

Concrete Retention Benchmarks: 60% Free, 20-30% Paid, 80% Friends

Crystal shares specific week-over-week retention targets. Free products should hold at least ~60% weekly; paid products more like 20-30%. At tiny scale, friends-and-family retention should be near 80% because if you can't convince people who care about you, the product likely doesn't solve the job. She thinks in cohorts: week-one retention should hit ~60% and then flatten within two to three periods.

  • Free product: at least ~60% week-over-week retention at scale.
  • Paid product: roughly 20-30% weekly retention.
  • Friends-and-family / tiny scale: near 80%, or the product isn't solving the job.
  • Thinks in cohorts: week-one ~60%, then flatten over 2-3 frequency periods; Gojek early days saw 60-70%.

“if it's a free product 60 percent right it has to be at least 60 if it's a free product week over a week if…”

Crystal Widjaja · 25:00

“that better be near close to 80 no matter what because if you can't even convince the people who care about you to use the…”

Crystal Widjaja · 25:30
#retention#benchmarks#metrics#cohorts
Takeaway48:30

Growth as the "Cleanup Crew" of Stats-Heavy People

Crystal describes how Gojek's growth team formed by filling obvious gaps the core product team left behind (phone-number masking, SMS/OTP delivery, first-time-user education) rather than inventing a mandate. The team called itself the cleanup crew, staffed heavily with statistics graduates who cared about numbers and odds. Her warning: the worst outcome is a growth person who thinks they're right but is measuring wrong and chasing the wrong areas.

  • Growth emerged by filling gaps the core product team couldn't cover, not from a grand plan.
  • Examples: phone-number masking, choosing SMS/OTP providers, educating first-time users.
  • The team was stats-heavy (statistics graduates) because scale makes it a numbers game.
  • Worst case is a growth hire who measures things wrong and focuses on the wrong areas.

“we really called ourselves like we're the cleanup crew we pick up the pieces that were left behind we connect the dots”

Crystal Widjaja · 52:00

“the worst possible thing is to have a growth person who thinks they are doing the right thing and is measuring things wrong and then…”

Crystal Widjaja · 53:00
#growth-team#hiring#org-design#statistics