The Possibility of an Ongoing Moral Catastrophe (Summary)

Core claim

Assuming moral objectivism (or a close approximation), we are probably unknowingly guilty of serious, large-scale wrongdoing — an “ongoing moral catastrophe.” The paper argues this both inductively (every past society got something catastrophically wrong) and disjunctively (there are so many independent ways to be wrong that getting everything right is near-impossible), and concludes that the response is aggressive moral research plus maximal material and social flexibility to change course once wrongs are identified.

Provenance

An outline of Evan G. Williams’ paper, originally made by Linch for a local discussion group and later published as a linkpost. Annotations marked LZ are the summariser’s own comments rather than the paper’s.

Definition: what is a moral catastrophe?

Three criteria:

1. Serious wrongdoing — closer to wrongful death or slavery than to mild insults or inconveniences.
2. Large-scale — not a single wrongful execution or a single man tortured.
3. Broad social responsibility — swathes of society are implicated through action or inaction; not unilateral, unavoidable acts by a single dictator.

Why we probably have unknown moral catastrophes

The inductive argument

• Assumption: it’s possible to commit great moral wrongs while acting in accordance with your own morals and those of your society. An honest, sincere Nazi still seems to be acting wrongly in important ways.
  • It doesn’t matter whether the wrongdoing stems from mistaken empirical beliefs or from wrong values.
• Given that assumption, pretty much every major society in history has acted catastrophically wrongly — conquistadores, crusaders, caliphates, Aztecs, all conquering in the name of gods they called good and just.
  • It’s implausible that all of these people merely professed such beliefs and were liars rather than true believers.
  • This is an existence proof: people can and do commit great evil without being aware of it.
• Our own case is not merely possible but probable:
  • We are not that different from past generations — hundreds of generations have believed they had figured out the One True Morality.
  • As recently as our parents’ generation, it was common to believe some people have more rights than others because of race or sexuality.
  • We live in a period of moral upheaval; our morality differs sharply from our grandparents’.
  • Even if some generation eventually gets all of morality right, that generation’s parents would be the ones who got almost everything right — and we have no reason to think we are them.

The disjunctive argument

• Activists are not exempt. Even if all your pet causes succeed, society isn’t thereby good, because unknown catastrophes remain.
• There are so many different ways a society could be badly wrong that getting literally everything right is almost impossible — and the errors could be a sizable fraction as bad as the Holocaust.
• Kinds of error:
  • Wrong about who has moral standing (e.g. fetuses, animals).
  • Empirically wrong about what harms those who morally matter (e.g. religious indoctrination of children).
  • Right about some obligations but not others — including wasting resources on false moral obligations, as the crusaders did.
  • Right about what’s wrong but wrong about how to prioritise fixes.
  • Right about what’s wrong but wrong about what is our responsibility to fix (e.g. poverty, borders).
  • Wrong about the far future (natalism, existential risk).
• Within each category there are multiple ways to go wrong, and some are mutually exclusive — e.g. either pro-lifers are right and abortion is a great sin, or fetuses don’t matter and restricting women’s freedom is itself gravely immoral. It’s unlikely we sit at the golden mean on all such trade-offs.
• The disjunction: even at 95% confidence on each of ~15 major issues, the probability of being right on all of them is ~0.95¹⁵ ≈ 46% (LZ notes this assumes independence). In practice 95% per issue is far too confident and 15 issues far too few.

What should we do about it?

Discarded option: hedging

• The “play it safe” strategy — e.g. going vegetarian just in case, even if you think farmed animals probably aren’t sentient.
• This doesn’t work well enough because it isn’t robust: too many things can go wrong, and some point in contradictory directions.

Recognition of wrongdoing

• Actively try to identify which catastrophic wrongs we are committing.
  • More research into practical fields where we could be critically wrong (e.g. animal consciousness).
  • More research into moral philosophy — it is dangerous to have increased technological knowledge without increased moral wisdom (imagine Genghis Khan with nuclear weapons).
  • These fields must interact. It isn’t enough for philosophers to say animals matter if conscious and for scientists to say dolphins are conscious but not know whether it matters; society must be able to integrate the two.
• Need a marketplace of ideas in which true ideas win out.
• Rapid intellectual progress is critical: if it’s worth fighting literal wars to defeat the Nazis or end slavery, it’s worth substantial material investment and societal loss to find out what we’re currently doing wrong.

Implementation of improved values

• Once wrongs are identified, we want to make reparations or at least stop the harm as quickly as possible.
• Maximise flexibility in material conditions:
  • Extremely poor or war-torn societies cannot make rapid moral changes.
  • LZ: complex systems built to specific designs are less resilient to shocks and harder to change (cf. Antifragile).
  • As we stockpile resources for war, we might save resources for future moral emergencies — to pay reparations or make changes quickly.
  • LZ: unclear how this works in practice — individuals save by investing and governments by buying others’ debt or investing in the private sector, but it’s unclear how the world “saves” as a whole.
• Maximise flexibility in social conditions:
  • Even where change is materially possible, inertia and conservatism bias can block it.
  • Entrenchment mechanisms such as constitutional amendments are suspect.

Counterconsiderations

• One: building a society capable of correcting moral catastrophes is not the same as actually correcting them.
• Two: some of the suggested preparations may themselves be evil — money spent on moral research could have gone to global poverty; maximal social flexibility might require draconian restrictions on present people’s rights.
• Nonetheless, the paper holds this is still worth doing in the short term.