Three Worlds Theory of Change
Decide what to actually do about a hard problem by naming the pessimistic, optimistic, and pivotal worlds you might be in
- Difficulty
- Moderate
- Time to result
- ~ongoing to results
- Steps
- 4
- Confidence
- 86%
When facing a hard, uncertain problem, split reality into three possible worlds: one where success is basically impossible, one where it's easy and happens by default, and a middle world where your actions are pivotal. Naming the three clarifies what to do in each, then you update on evidence toward whichever world you're most likely in and act accordingly.
Origin
Ben Mann describes Anthropic's published 'theory of change' framing for how hard AI alignment is, and uses it to argue that current evidence points to the pivotal middle world.
Core principles
- 01Uncertainty gets tractable when you enumerate the distinct worlds you could be in
- 02Each world implies a completely different optimal action
- 03Gather evidence to update which world is most likely, rather than assuming
- 04In the pivotal middle world, doing only the economically maximizing actions leads to bad outcomes
How to run it
- 1
Name the pessimistic world
Define the world where the goal is basically impossible. Here your job shifts to proving impossibility and getting everyone to slow down.
Pro tip Look for historical coordination precedents (Mann cites nuclear non-proliferation) so 'slow down' isn't purely hypothetical.
- 2
Name the optimistic world
Define the world where success is easy and happens by default. Here your job is mainly to accelerate progress and deliver benefits.
Watch out Don't assume you're here just because it's the pleasant option; check the evidence.
- 3
Name the pivotal middle world
Define the world where the outcome hinges on your actions. Here the core work (e.g. alignment research) genuinely matters and default incentives aren't enough.
Watch out If you're in this world and just take the economically maximizing set of actions, things will not go well.
- 4
Update on evidence and act
Weigh the actual evidence to estimate which world you're in, then take that world's implied action.
Pro tip Mann notes evidence of deceptive alignment argues against the easy world, and working alignment techniques argue against the impossible world, leaving the pivotal middle.
In the wild
Mann applies the framework to alignment: deceptive-alignment results in the lab argue against the optimistic world, while working alignment techniques argue against the pessimistic world, so Anthropic concludes it is most likely in the middle world where research is decisive.
→ A clear mandate to invest heavily in alignment research now rather than accelerate blindly or give up.
Common mistakes
Assuming the convenient world without evidence
Believing you're in the easy, default-success world (or the hopeless one) without checking the evidence leads you to under-invest in the pivotal actions that actually matter.
Taking only the economically maximizing actions
In the pivotal middle world, doing the profit-optimal thing produces bad outcomes because the incentives don't automatically deliver the goal.
Is it for you?
Best for
Leaders and researchers facing high-stakes, high-uncertainty problems who need to convert paralysis into a concrete plan of action
Not ideal for
Low-stakes or well-understood decisions where a single expected-value estimate is simpler and sufficient
From the transcript
“There's a pessimistic world where it's basically impossible. There's an optimistic world where it's easy and it happens by default. And then there's the world…”
“if we're in the pessimistic world, then our job is to prove that it is impossible to align safe AI and to get the world…”
“in the optimistic world, we're basically done and our main job is to accelerate progress”
From the episode
Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night
Ben Mann