What Actually Improves AI Apps
Stop chasing AI news and vector DBs; the real levers are users, data, and prompts.
- Difficulty
- Easy
- Time to result
- ~weeks to results
- Steps
- 3
- Confidence
- 92%
A prioritization filter for teams building AI products that separates high-visibility busywork from the activities that actually move quality. Use it whenever you catch yourself debating the newest model, framework, or database instead of doing the unglamorous work. The core claim: perceived-important work (keeping up with AI news, latest agentic framework, vector-DB choice, model comparisons, fine-tuning) mostly under-delivers versus talking to users, preparing better data, optimizing the end-to-end workflow, and writing better prompts.
Origin
Built from Chip Huyen's viral LinkedIn table contrasting 'what people think will improve AI apps' vs 'what actually improves AI apps.' Lenny reads the table aloud and Huyen explains why it hit a nerve: people keep asking how to stay current with AI news when talking to users would improve the app far more.
Core principles
- 01Effort should flow to the levers that change output quality, not the ones that feel current.
- 02If two options produce similar performance, the debate between them is not worth your time.
- 03User feedback, better data, and better prompts beat tool/model swapping in almost every case.
How to run it
- 1
Catch the busywork instinct
Notice when you are optimizing for staying up to date, adopting the newest framework, agonizing over which vector database, or constantly re-benchmarking which model is smartest.
Pro tip The tell is a question phrased as 'which is better, X or Y?' about a fast-moving tool.
- 2
Estimate the delta
Ask how much improvement the 'optimal' choice actually gives over the non-optimal one. If it's small, deprioritize the debate entirely.
Watch out Teams routinely burn weeks debating choices that move performance by a rounding error.
- 3
Redirect to the real levers
Spend the reclaimed time talking to users, building more reliable platforms, preparing better data, optimizing the end-to-end workflow, and writing better prompts.
Pro tip Talking to users to understand what they want and reading their feedback is the single highest-leverage input.
In the wild
People repeatedly asked Huyen whether to pick one protocol/technology over another. She reframed: how much improvement do you actually get from the optimal vs non-optimal solution? Often the honest answer was 'not much.'
→ The debate is revealed as low-value and abandoned in favor of user and data work.
Common mistakes
Treating currency as progress
Staying up to date with AI news feels productive but rarely improves the specific app in front of you.
Optimizing components that barely move the needle
Agonizing over vector databases when data preparation is the actual bottleneck.
Is it for you?
Best for
Product and engineering teams building LLM-powered apps who feel busy but aren't shipping quality gains.
Not ideal for
Frontier-lab researchers whose job literally is model and technique advancement.
From the transcript
“why do you want to spend so much time debating something that doesn't uh make that much difference to your performance”
“If you talk to the users and understand what they want, what they don't want, look into the feedback, then you can actually improve the…”
From the episode
Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix)