When-It-Leaks Communications Pre-Mortem
At scale you can't test quietly — decide the message before you even know you want to launch
- Difficulty
- Moderate
- Time to result
- ~days to results
- Steps
- 3
- Confidence
- 86%
A communications discipline for any organization big enough that even a small experiment will be noticed and covered. Before running any potentially controversial test, assume it will leak and pre-decide the message and posture. Use when planning experiments, pricing changes, or design tests at a company with public scrutiny.
Origin
Mosseri explains that at Instagram's scale you can't launch to three billion people untested, but you also can't test at scale without people covering it — so the team prepares a communications posture in advance, framing it as 'not if it leaks, when it leaks.'
Core principles
- 01At scale, testing is not private — assume coverage
- 02Reframe from 'if it leaks' to 'when it leaks'
- 03Decide proactive-vs-reactive posture and the actual message before launch
How to run it
- 1
Assume the leak
For any design change or test that could be controversial, discuss it beforehand on the premise of when — not if — it leaks.
Watch out Pricing tests are especially dangerous — be extra careful with those.
- 2
Decide posture and message
Determine whether to talk about it proactively or reactively, and either way settle on the actual message in advance.
Pro tip You don't have to be proactive, but you must have a calm strategy ready.
- 3
Be ready before you commit to launch
Because you must test before launching but can't test invisibly, be ready to talk about it before you even know whether you want to launch it.
Watch out This makes the development cycle more complicated than it used to be — budget for it.
In the wild
Instagram ran an early test of a video-viewer feed to just 4% of iOS users, never intending to roll it out; it got conflated with reels, recommendations, and creator reach complaints, and big creators plus the press created a multiplier effect.
→ A never-to-launch test became a public 'the world protested' event — the case for pre-deciding the message.
Common mistakes
Treating a small test as private
One person spotting a 1% or 4% experiment can turn it into a Twitter firestorm; assuming invisibility leaves you scrambling.
Is it for you?
Best for
Product and comms leaders at high-scrutiny companies planning experiments or pricing tests
Not ideal for
Small companies whose experiments aren't newsworthy
From the transcript
“we talk about it beforehand and be like, "Okay, not if it leaks, when it leaks, what are we saying?"”
“you have to be ready to talk about it before you even know you want to launch it.”
“pricing particularly that one is a real you got to be real careful with that one.”
From the episode
Adam Mosseri: AI is a tailwind for authenticity