Hills-and-Valleys: Getting Value From Probabilistic AI Tools
Be patient and explicit, start small, and learn where the model is strong vs weak
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 4
- Confidence
- 85%
A usage framework for getting real leverage out of agentic AI coding tools (and similar probabilistic assistants) instead of being frustrated by them. The core insight: like autocomplete accepted only 30% of the time, the tool isn't annoying once you internalize its 'hills and valleys' — where it reliably helps and where it won't — and calibrate your prompts and scope accordingly.
Origin
Varun Mohan's onboarding advice for new Windsurf users, generalized from how experienced users relate to autocomplete.
Core principles
- 01Probabilistic tools aren't annoying once you know when NOT to rely on them
- 02Explicitness dramatically reduces irrelevant changes
- 03Small scopes limit blast radius when the model is wrong
- 04The tool's capability frontier shifts every ~3 months, so re-evaluate what it can do
How to run it
- 1
Be patient and as explicit as possible
Spell out exactly what you want. Vague asks let the agent make many irrelevant changes; explicit instructions are the single biggest lever against that.
- 2
Start with smaller changes
Begin with small, contained edits rather than asking the agent to refactor an entire large directory in one shot.
Watch out A wrong large refactor can destroy 20 files at once; small scopes cap the damage.
- 3
Learn the hills and valleys
Build a gut feel for where the model reliably succeeds versus where it won't — analogous to knowing autocomplete won't handle a very complex line before you even try it.
Pro tip Calibrate how specific vs abstract to be based on that gut feel; accepting only some suggestions is normal, not failure.
- 4
Re-evaluate the frontier every few months
Assume the capability boundary moves materially every ~3 months and periodically retest tasks that previously failed.
Watch out Sticking to a stale mental model means you'll keep avoiding tasks the tool can now handle.
In the wild
A tool suggesting completions accepted only 30% of the time sounds annoying, but experienced users aren't bothered — they've learned that 70% of the time they don't need it, and know how to extract value the other 30%, and can predict in advance when a complex command won't autocomplete well.
→ Users get consistent value from a probabilistic tool by calibrating expectations rather than expecting perfect output.
Common mistakes
Asking for huge refactors up front
Large-scope asks maximize the damage of a wrong output; starting small keeps errors cheap to recover from.
Being vague
Under-specified prompts invite many irrelevant changes; explicitness is the primary defense.
Is it for you?
Best for
New users of agentic AI coding tools (and non-developers building apps) who want reliable leverage quickly
Not ideal for
Tasks demanding guaranteed-correct output on the first try with no review, where a probabilistic tool is the wrong choice
From the transcript
“be really really explicit or as explicit as possible”
“in the beginning start by making smaller changes”
“because then if it's wrong, it's going to like basically destroy 20 files”
“understand what the hills and valleys of the product are the crazy thing is every 3 months that kind of gets changed and reevaluated”
From the episode
Building a magical AI code editor used by over 1 million developers in four months: The untold story of Windsurf
Varun Mohan (co-founder and CEO)