The Confidence Meter
Score how much evidence actually backs an idea, from a shiny pitch deck (0.1) to a live A/B test (10)
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 3
- Confidence
- 95%
The Confidence Meter is a thermometer-like scale from 0 (guessing in the dark) to 10 (proven success) that ranks classes of evidence by how much confidence they should earn. Opinions and themes sit at the very bottom, group reviews and estimates slightly higher, anecdotal and market data in the middle, and only building-and-testing the idea reaches medium-to-high confidence. It exists to stop people from giving a gut-feel idea a high confidence score just because they feel strongly about it, and to tie the level of investment in an idea to the strength of evidence behind it.
Origin
Created by Itamar Gilad as the 'C' (confidence) component of ICE. He built it because he found people subvert ICE by going with gut instinct and then assigning themselves high confidence ('it's an eight and I'm pretty convinced'). The tool helps people 'realize when they have strong evidence and support over their guesses and when it's weak evidence.'
Core principles
- 01Behind every terrible idea, someone thought it was great — self-conviction earns only ~0.01 out of 10
- 02A polished pitch deck or six-page doc is slightly harder to make but still near-zero confidence
- 03Connecting an idea to a hot theme (blockchain, AI) or company strategy earns at most ~0.1 out of 10 combined — 'thousands of terrible ideas are being implemented right now based on these themes'
- 04Groups can reduce individual bias but add their own (politics, groupthink) and sometimes decide worse than individuals
- 05Only building and testing the idea earns medium-to-high (red-zone) confidence
How to run it
- 1
Locate your evidence on the meter
Identify which class of evidence you currently have for an idea — opinion/theme (blue, near 0), peer review or estimates (low), anecdotal or market data (middle), or an actual test (red, high) — and read off the honest confidence score.
Pro tip Use it as shared team language: 'we think impact is high but we're in the blue zone on confidence' makes the gap explicit and unemotional.
Watch out Don't let strong personal conviction inflate the score; self-confidence alone is worth 0.01 out of 10.
- 2
Match investment to confidence level
Do the cheap evidence-gathering first to raise confidence, then invest more only as confidence climbs. For a cheap, low-risk idea you can jump straight to a high-confidence test like an A/B experiment or early-adopter program.
Pro tip Some ideas need no testing at all — reordering settings nobody notices is low-risk, so expert opinion is enough; part of the skill is knowing when to stop climbing.
Watch out Assuming a leading competitor's feature validates your idea 'never works honestly — you should not assume that your competitor actually knows what they're doing any more than you do.'
- 3
Use it to say no
Deploy the meter to objectively and gently reject or defer weak ideas: acknowledge the idea, show it sits low on confidence and would cost months, and pivot the conversation to higher-scoring alternatives.
Pro tip A big part of a PM's job is to stop bad ideas from happening — the meter reframes 'no' as an objective evidence question rather than a personal veto.
In the wild
Gilad found people 'just go with God's Instinct and then give themselves a high confidence — they say it's an eight and I'm pretty convinced so it's eight for confident.' He found this disturbing because it subverts the whole ICE system, which motivated building the meter to distinguish strong from weak evidence.
→ The Confidence Meter gives conviction, decks, and themes near-zero scores, forcing teams to gather real evidence before claiming confidence.
Before writing a line of code, the Gmail team showed users a facade of HTML that looked like a working tabbed inbox — researchers manually moved the subject and sender of the top ~50 messages into the right tabs while an interviewer distracted the user.
→ Users said 'wow this is actually very cool,' giving the team enough evidence (moving up the meter) to justify building the real thing.
Common mistakes
Assigning high confidence to opinions and themes
A pitch deck, a strategy tie-in, or an 'it's about AI' framing feels persuasive but earns at most ~0.1 out of 10; treating these as validation ships thousands of terrible ideas.
Forcing every idea all the way up the meter
Over-testing low-risk, low-impact changes wastes time; knowing when to stop (expert opinion suffices for invisible, low-risk tweaks) is as important as knowing when to test.
Is it for you?
Best for
PMs and product teams who already run some experiments but want a shared, objective vocabulary to rank evidence strength and defensibly say no to weak ideas
Not ideal for
Trivial, low-risk, invisible changes where expert opinion is sufficient and any testing is wasted effort
From the transcript
“what happens is that people tend to just go with God's Instinct and then give themselves a high confidence they say it's an eight and…”
“behind every terrible idea that was ever someone thought it was great that gives you 0.01 out of 10”
“it's about AI that makes it a good idea absolutely not or the strategy of the company that's another thematic support thousands and thousands of…”
“you can also tie your investment into the idea based on the level of confidence you had found essentially so early on you want to…”
“people use this to kind of do an objective way to say no”
From the episode
Becoming evidence-guided
Itamar Gilad (Gmail, YouTube, Microsoft)