Multi-Prototype Comparison Testing
Show three distinct concepts instead of iterating one, so users can compare and you avoid over-commitment
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 3
- Confidence
- 90%
Rather than testing and polishing a single prototype, present three deliberately distinct 'recipes' of value proposition so customers can compare and contrast. Comparison unlocks richer feedback (people react differently when they have reference points) and keeps the team from getting over-attached to one idea. You aren't picking a winner — you're harvesting the best pieces to construct the ideal next version.
Origin
Rooted in Margolis' Walmart.com years, where he came to 'see the whole world as a shopping process' — people evaluate options comparatively, not in isolation. Formalized as the three-prototype leg of the Bullseye Customer Sprint.
Core principles
- 01People give shallow feedback on one option but sharp comparative feedback across two or three
- 02It's not the customer's job to imagine alternatives — the prototypes supply the reference points
- 03Multiple prototypes keep the team neutral and prevent over-commitment to a single idea
- 04You're collecting the best 'Lego pieces', not crowning a winner
- 05Keep prototypes flat and simple so nobody gets too wed to one
How to run it
- 1
Map the variables to spread across prototypes
Identify the key features and the distinct variables (e.g. who delivers, size of delivery window) and spread them across three concepts so each expresses a different value-prop recipe.
Pro tip Use a competitor's real product as one free prototype — studying how people respond to it is a shortcut too many teams skip.
- 2
Build three flat, visually distinct concepts
Create simple PDFs that look like a real homepage, each visually different (green/blue/red) so observers can track which one is being discussed.
Pro tip Naming them by color rather than A/B/C keeps the watch-party observers from losing track of which concept a participant means.
Watch out Proofread carefully — errors make people fixate and undermine the prototype's credibility.
- 3
Have participants compare and cherry-pick
Present each concept, let people say what they like and dislike, then have them compare across concepts to surface the best pieces of each.
Pro tip Parts they don't comment on at all are useful signal too — it means those elements don't register.
Watch out Don't push for a single winner; the value is in the mix-and-match of desirable attributes.
In the wild
Margolis illustrates that shown one couch you'll say 'I like the color, cushions are uncomfortable'; shown two or three you start reasoning comparatively — 'I like this one's down cushions but that one gets delivered' — producing far richer, decision-relevant feedback.
→ Comparison generates distinct value-proposition reference points that single-item testing can't.
For the specialty-med delivery company, prototypes varied who delivered it (pharmacist, courier, even a drone) and the delivery-window size (ASAP vs a 15-minute window vs whenever). Participants teased apart which elements mattered.
→ Revealed that a predictable narrow window was the high-value attribute, guiding the real product design.
Common mistakes
Iterating and polishing a single prototype
Testing one concept yields only opinions with no reference points and lets the team get over-committed and non-neutral about that idea.
Making prototypes too real or functional
Investing in polished, working prototypes wastes effort and deepens attachment; the point is a crisp written value prop, not production fidelity.
Is it for you?
Best for
Product teams choosing between concept directions or value propositions before committing engineering resources
Not ideal for
Late-stage optimization of a single validated product where the question is refinement, not direction
From the transcript
“as you start looking at two or three then you start having this different reference points”
“I'm not looking for a winner but this comparison”
“the other big benefit of having multiple prototypes is it helps teams avoid getting too wed to one specific idea”
“I want to like grab the best bits and pieces the best Lego pieces so I can then go construct the ideal version”
From the episode
Identify your bullseye customer in one day
Michael Margolis (UX Research Partner at Google Ventures)