Abstract
The value of the future is (roughly) the product of “Surviving” — avoiding near-term existential catastrophe — and “Flourishing” — how good the future turns out to be, conditional on survival; since Survival probability already looks fairly high while Flourishing looks far below its ceiling, and Flourishing gets far less attention, the Better Futures essay series argues altruists should shift more focus from ensuring survival to improving the value of the future we get.
The two-factor model
- Announces the “Better Futures” essay series (by Forethought and William MacAskill), around eight years in the making, plus a companion video podcast episode.
- Central question: should future-oriented altruists focus on surviving, or on flourishing? In practice, most focus goes to survival (or avoiding permanent disempowerment by valueless AI).
- Frames the value of the future as the product of two factors: “Surviving” and, conditional on Surviving, “Flourishing.”
- Defines “not-Surviving” broadly: it includes extinction from a bio-catastrophe, but also a scenario where valueless superintelligence disempowers humanity without causing literal extinction — anything that locks in a near-0-value future in the near term. This is offered as roughly how “existential catastrophe” is used in practice.
Survival looks relatively likely
- Most people think humanity is more likely than not to survive this century.
- Cited estimates: Metaculus puts extinction risk at about 4%; a survey of superforecasters put it at 1%; Toby Ord put total existential risk this century at 16%.
- The suggestion is that we may be closer to the “ceiling” on Survival than to the “ceiling” on Flourishing.
Flourishing looks far from its ceiling
- In the two essays that follow in the series, Fin Moorhouse and MacAskill argue that the expected value of Flourishing — the fraction of a best feasible future’s value we should expect to achieve, conditional on survival — is low.
- Worked illustration: if the chance of Surviving this century is 80%, but the value of the future conditional on survival is only 10%, then the problem of non-Flourishing is 36x larger in scale than the problem of not-Surviving.
- Caveat: this scale argument doesn’t go through for someone with a very high “p(doom).”
- Stresses that it’s the fraction of possible value achieved that matters, not the absolute value: moving from a 50%-value future to a 60%-value future is just as important as moving from a 0%-value future to a 10%-value future.
- Cockaigne analogy: medieval myth described a utopia (“Cockaigne”) of endless food, endless youth, and endless sex. Rich countries today would likely look like Cockaigne to a medieval peasant — yet are still very far from a truly good society. A future that looks common-sensically utopian could still capture almost none of the value that’s actually possible.
Flourishing is more neglected
- The latent desire to survive is much stronger than the latent desire to flourish: most people don’t want to die or be disempowered in their lifetimes, so for existential risk to stay high there has to be some major failure of rationality (e.g. those controlling superintelligent AI being deluded about the risk, or willing to gamble their lives for more power).
- Example of latent concern for survival: the aggregate willingness to pay in the United States to avoid a 0.1 percentage-point chance of a catastrophe that kills everyone is estimated at over $1 trillion; a “warning shot” could unlock that latent concern and its attention.
- By contrast, there’s little latent desire to ensure people millennia from now haven’t made some subtle but important moral mistake — society could be clearly on track for a major moral error and simply not care.
- Even within the effective altruism community, most focus goes to Surviving. AI safety and biorisk reduction have gained much more attention and investment in recent years — but as they do, their comparative neglectedness declines.
Tractability is the open question
- The tractability of “better futures” work is the least clear part of the case — if the overall argument fails, it’s likely to fail here.
- Analogy: a decade ago, AI safety and biorisk mitigation looked highly intractable, but concerted effort made those areas tractable; the series argues for trying to do the same for flourishing-focused work.
- Candidate areas for future work: AI-enabled human coups; AI for better reasoning, decision-making, and coordination; what character and personality advanced AI should have; what legal rights AIs should have; governance of projects to build superintelligence; and deep space governance.
Caveats on the series
- The essays tend to use moral-realist language (e.g. a “correct” ethics), but most of the arguments are meant to port over to other metaethical views — readers can translate into language like “what I would think about ethics given ideal reflection.”
- The series only addresses one part of ethics — the long-term future, or “cosmic ethics” — and doesn’t cover more familiar reasons to prevent near-term catastrophe, such as not wanting oneself and one’s loved ones to die; those reasons are not being dismissed as unimportant.
- Thinking about making the future better can seem “creepily Utopian.” Some 20th-century Utopian movements were extraordinarily harmful, which is a reason to be wary of proposals for better futures based on some narrow, specific conception of an ideal future — especially given how much moral progress we should expect still needs to happen.
Viatopia, not utopia
- MacAskill favours aiming for “viatopia”: a state of the world where society can guide itself towards near-best outcomes, whatever those outcomes turn out to be, rather than towards any specific narrow end-state.
- Viatopia plausibly involves: very low existential risk, many different moral points of view able to flourish, many possible futures kept open, and major decisions made through thoughtful, reflective processes.
- States that, in his view, the key priority in the world today is getting closer to viatopia — not to any particular narrow end-state — and notes he hopes to write more about the concept in future.