The Top-Three Weighted DevX Survey
Force a top-three, weight by frequency, and never ask four questions at once
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 5
- Confidence
- 90%
A survey-design method for getting a fast, quantified baseline of developer friction when you lack instrumentation. It forces respondents to pick only their top three problems, then weights each by how often it hits them (hourly to quarterly), producing a scoreable ranking. It also enforces single-signal question design so answers are interpretable. Use it as the fastest way to a baseline early in a DevX program.
Origin
Forsgren's recommended starting point 'if you have nothing at all,' with example surveys included in her book Frictionless. She stresses forcing a top-three because letting people pick everything makes the data 'super super messy.'
Core principles
- 01Let respondents pick only their top three barriers — picking everything makes the data unusable
- 02Weight each problem by frequency, because a daily annoyance and a quarterly ordeal differ in cost
- 03One survey question must ask exactly one thing
- 04If you can't answer a question with the data a metric gives, don't collect it
How to run it
- 1
Ask satisfaction and barriers
Ask how satisfied developers are and what the biggest barriers or challenges to getting work done are.
- 2
Force a top-three selection
From a set of tools or processes, have each respondent pick only their top three problems.
Pro tip Constraining to three is what keeps the resulting data clean enough to score.
Watch out Letting people pick everything makes the data super messy.
- 3
Weight by frequency
For each chosen item ask how often it affects them: hourly, daily, weekly, or quarterly — then compute a score or weighted score.
Pro tip A quarterly problem can still rank high if it's onerous enough at end of quarter.
- 4
Add an open-text catch-all
Finish with an open 'is there anything else we should know?' field to capture signal your structured questions missed.
- 5
Enforce single-signal questions
Review each question so it asks only one thing; split compound questions apart.
Pro tip Draft and pressure-test questions with Claude, Gemini, or ChatGPT over a couple of rounds, and consult someone familiar with survey design.
Watch out 'Were the build and test systems slow or complicated last week?' asks four questions at once — you can't tell which one a 'yes' refers to.
In the wild
A common survey mistake asks 'Were the build and test system slow or complicated in the last week?' If a developer answers yes, you can't tell whether it was the build or the test, or whether it was slow versus flaky or complicated.
→ Splitting into single-signal questions makes the resulting data actually interpretable.
Common mistakes
Collecting data you can't act on
If you can't answer a real question with the data a metric produces, don't collect it — including questionable metrics designed for another purpose.
Letting respondents flag everything
Unbounded selection produces messy, unrankable data; the forced top-three plus frequency weighting is what yields a usable score.
Is it for you?
Best for
DevX leads who lack instrumentation and need a fast subjective baseline of team friction
Not ideal for
Orgs that already have trustworthy, purpose-built telemetry across the pipeline
From the transcript
“let them pick three just three of those three how often does this affect you? Right? Is this hourly? Is this daily? Is this weekly?…”
“by making folks prioritize the top three things if you let them pick everything like it makes the data super super messy”
“Were the build and test system slow or complicated in the last week? You're asking four different questions there.”
“If you can't answer a question with data, don't get it.”
From the episode
How to measure AI developer productivity in 2025
Nicole Forsgren