The Catastrophic Harm Precautionary Principle

Abstract

When catastrophic outcomes are possible it makes sense to take precautions against worst-case scenarios — but the principle rests on three foundations rather than on blanket caution: people’s failure to appreciate the expected value of truly catastrophic losses, political incentives to delay when costs are immediate and benefits distant, and the distinction between risk and uncertainty. Regulators should not simply identify and eliminate the worst case; they should ask how much worse the worst case is than the next-worst, and what is lost by eliminating it.

Publication details

• Published in Issues in Legal Scholarship, symposium on Catastrophic Risks: Prevention, Compensation, and Recovery (2007), Article 3.
• Draws heavily on, and substantially revises, Sunstein’s earlier “Irreversible and Catastrophic,” 91 Cornell L. Rev. 841 (2006), with some conclusions altered.
• Illustrated throughout with climate change; other applications include avian flu, genetic modification of food, endangered species protection, and terrorism.

Starting point: existing formulations

The article opens with four international texts said to give the Precautionary Principle special force for serious risks:

• UN Economic Conference for Europe Ministerial Declaration (1990): where threats of serious or irreversible damage exist, lack of full scientific certainty “should not be used as a reason for postponing measures.”
• Rio Declaration (1992): the same, for cost-effective measures.
• UN Framework Convention on Climate Change: the same, with the added qualification that measures should be cost-effective “so as to ensure global benefits at the lowest possible cost.”
• First European “Seas At Risk” Conference Final Declaration (1994): if the worst-case scenario “is serious enough then even a small amount of doubt as to the safety of that activity is sufficient to stop it taking place.”

Sunstein extracts a refined principle: when risks have catastrophic worst cases, pay special attention to them even when probabilities cannot be reliably assessed. He immediately concedes the principle is “lamentably vague,” leaving undefined the triggering threshold, the role of costs, and how to incorporate what probability information exists.

Version 1 — Expected value

• Three stylised problems, each with an expected loss of 200 lives: (a) a one-in-a-million chance that 200 million die; (b) a 50% chance that 400 die; (c) a certainty that 200 die. If life is valued at $6 million, $1.2 billion should be spent on (a) — which is what the government’s own approach to risk reduction implies.
• Modest formulation: regulators should consider the expected value of catastrophic risks even when they are highly unlikely, choose cost-effective measures, and attempt to maximise net benefits. Catastrophes get no special treatment — merely no less attention than higher-probability harms of equivalent expected value.
• The justification is behavioural: people treat low-probability risks as if they were zero, especially when the harm is distant. Because probability judgments run on the availability heuristic, and low-probability risks by nature lack available real-world instances, catastrophic risks may go entirely ignored — “off-screen.”
• Own experiment (176 law students at Alabama and Chicago), comparing a one-in-a-million risk of 200 million deaths against a one-in-ten risk of 2,000 deaths: 41% ranked the second problem higher, 36% called them equal, and only 22% ranked the first higher. Far more people were risk-seeking than risk-averse about low-probability catastrophes. Sunstein expects the effect to be stronger in the general population than among law students trained to attend to expected value.

Version 2 — Social amplification

• The loss of 200 million people is plausibly more than 1,000 times worse than the loss of 2,000 — the nation’s private and public institutions would be damaged for a long time, perhaps permanently, and future generations would suffer.
• Evidence offered: the “Buffalo Creek Syndrome.” Nearly two years after a dam collapse that killed 120 and left 4,000 homeless, researchers still found loss of direction and energy, disabling character changes, and loss of communality — attributed to “the loss of traditional bonds of kinship and neighborliness.”
• The literature on the social amplification of risk documents secondary losses that outrun an event’s initial effect.
• Revised formulation: the same as version 1, but requiring regulators to consider catastrophe’s distinctive features, including social amplification, when assessing expected value.

Version 3 — Catastrophe aversion as regulatory insurance

• People buy insurance that expected value alone would not justify — John may sensibly pay $2,000 a year to insure a $150,000 house because losing it would shatter him, and because money has declining marginal value. Prospect theory predicts risk aversion in exactly these circumstances.
• By analogy, a “catastrophe aversion” premium — a margin of safety — can be built into regulation, with its extent set by the costs of indulging the aversion.
• Risk-risk and catastrophe-catastrophe tradeoffs must be included. The Iraq war is offered as an example: defensible ex ante as catastrophe avoidance if Saddam Hussein possessed WMD, but generating serious risks of its own; some believe it created catastrophic risks, making it a catastrophe-catastrophe tradeoff. Greenhouse gas controls can be analysed similarly.
• The principle grows longer with each addition: expected value, social amplification, cost-effective measures maximising net benefits, a cost-justified margin of safety, and attention to the risks created by precaution itself.
• Timing is a further ground: when costs are immediate and benefits decades away, people show extreme time preference. A cited survey found deferred benefits receive roughly 50–60% of their correct discounted value. Politically, officials face retribution for imposing immediate costs and little reward for long-term benefits; for climate change, “those who are most likely to benefit do not vote.”

Version 4 — Maximin, and why it fails as a general rule

• Risk vs. uncertainty vs. ignorance (following Knight): risk means outcomes and probabilities are known; uncertainty means outcomes are identifiable but probabilities are not; ignorance means neither the probability nor the nature of the harms is known.
• Crude maximin formulation: identify and attempt to eliminate the worst-case scenario. Sunstein rejects it.
• Numerical counterexample: a 99.9% chance of gaining $2,000 with a 0.1% chance of losing $6, versus a 50/50 chance of gaining or losing $5. Maximin prefers the second; only extraordinary risk aversion would.
• Harsanyi’s objection: taken seriously, maximin means never crossing the street, driving over a bridge, or marrying — “he would soon end up in a mental institution.”
• Sunstein’s correction to Harsanyi: the objection neglects that risks lie on all sides. Not crossing streets and not marrying have worst cases of their own. Implementing maximin requires identifying all relevant risks, not a subset — and a precautionary principle carelessly applied “might produce nightmarish scenarios too.”
• Nonetheless, under conditions of risk, maximin “seems to require infinite risk aversion” and should be rejected when the worst case is highly improbable and the alternative is both much better and much more likely.

Uncertainty: the harder case

The Rawls/Gardiner argument

• Rawls defends maximin where (1) outcomes are potentially catastrophic, (2) probabilities cannot be assigned, and (3) the loss from following maximin is a matter of relative indifference. Stephen Gardiner builds a “core” Precautionary Principle on this for environmental policy; Jon Elster makes an analogous case for restricting nuclear power. Gardiner sensibly adds a minimal plausibility threshold for the catastrophic threat.
• This yields a fifth formulation: eliminate the potentially catastrophic worst case where probabilities cannot be assigned and the cost of elimination is essentially zero.

Four objections

• Triviality. If elimination is costless, of course do it — but real policy disputes rarely take that form. For climate change, following maximin would mean higher energy prices and likely increases in unemployment and poverty, falling hardest on the poor; for genetically modified food, it might eliminate an inexpensive source of nutrition for people in extreme deprivation.
• “Maximin assumes infinite risk aversion.” Sunstein says this standard challenge is wrong: it depends on denying that uncertainty exists, stipulating that subjective choices reveal subjective probabilities.
• “Uncertainty does not exist.” Friedman, following Savage, treats people as if they assign numerical probabilities to every event. Sunstein’s replies: (i) Knight was concerned with objective probabilities, and subjective probabilities can be inferred from animals too — the behaviour of his Rhodesian Ridgeback would imply estimates of death by car accident, terrorism, another human, and thunderstorm at roughly 40%, 0%, 10%, and 99.9%; (ii) some questions (how likely is it that 100 million humans are alive in 10,000 years? an urn with an undisclosed colour ratio) admit no sensible assignment; (iii) subjective assignments are framing-dependent — “80% will suffer” and “20% will not suffer” produce different answers — and are corrupted by heuristics and biases including availability and unavailability bias. Keynes is quoted: about the prospect of a European war “there is no scientific basis on which to form any calculable probability whatever. We simply do not know.”
• “Uncertainty is too rare to matter.” Perhaps probabilities, or probabilities of probabilities, can usually be assigned; Posner argued no probabilities can be attached to catastrophic global-warming scenarios, while a 1994 expert survey produced estimated losses from climate change ranging from zero to a 20% decrease in gross world product. Sunstein concedes pure uncertainty may be rare but insists bounded uncertainty — unable to assign probabilities within specified bands — “is not rare at all.”

What people actually choose

• Ellsberg-style experiments show bounded uncertainty aversion: people prefer the urn with a known 50/50 split.
• Sunstein’s second experiment (71 Chicago law students) pitted a 90% chance of 600 deaths against an unquantifiable problem whose worst case was 700 deaths: 63% gave priority to the first, i.e. rejected maximin.
• His third (173 students at Chicago and Alabama) offered a range of 400–500 deaths versus a range of 10–600 deaths, probabilities unassignable: 85.5% chose the second, accepting the worse worst case.
• The interpretation is the Principle of Insufficient Reason — treating outcomes across a range as roughly equally likely — which pushes people toward the higher-expected-value option. Sunstein stresses that these findings describe behaviour and public demand for regulation, and do not settle the normative question, since the judgments are framing-sensitive and concern unfamiliar problems.

The final formulation

In deciding whether to eliminate the worst-case scenario under circumstances of uncertainty, regulators shall consider the losses imposed by eliminating that scenario, and the magnitude of the difference between the worst-case scenario and alternative scenarios.

• Presented as an orienting inquiry rather than a decision rule, and as a generalisation of the Rawls/Gardiner intuition beyond the trivial zero-cost case.
• It permits a rough cost-benefit analysis of maximin itself without reliable probabilities — provided outcomes can be ranked cardinally, not merely ordinally.
• Sunstein concedes the approach leans on some version of the Principle of Insufficient Reason, and that if probabilities (or probabilities of probabilities) genuinely cannot be assigned, the approach fails as a formal matter — but he argues it remains “more reasonable than any imaginable alternative.”
• Climate illustration: if catastrophic dangers could be eliminated by every nation contributing $10 million, the cost is plainly acceptable; if elimination required cutting world living standards by 50%, one should insist the risk clear a threshold — a demonstrable probability of, say, over 10%, or at least not below 1%.
• The portfolio objection to maximin: with ten, twenty, or a thousand uncertain catastrophic hazards and an equivalent set of quantifiable but non-catastrophic ones, spending heavily on even one uncertain hazard could exhaust resources for hazards with a 90% chance of occurring — leaving a society “condemned to real poverty and distress” by maximin alone.
• Sunstein declines to claim maximin is irrational, holding only that it makes most sense when one course of action’s worst case is much worse than the alternative’s and when choosing it does not impose extremely significant losses.

Catastrophe and irreversibility

• Knowledge grows over time, which argues for “wait and learn,” since immediate action is often taken in the dark.
• Two problems with waiting: by hypothesis we cannot exclude that catastrophic harm occurs while or because we wait; and inaction may be irreversible or reversible only at very high cost — greenhouse gases persist in the atmosphere, potentially saddling posterity with a risk they cannot eliminate.
• This supports buying an “option” through regulatory steps, as people routinely do by keeping courses of action reversible.

Concluding qualifications

• Regulators are rarely in conditions of pure uncertainty; rough probabilities, or rough probabilities of probabilities, are usually available.
• Maximin is most reasonable when the worst case is exceptionally bad and removing it inflicts no serious losses of its own.
• The principle must attend to the full range of social risks: taking steps to avert catastrophe that themselves create catastrophic risks renders the principle indeterminate — a preventive war that raises terrorism risk from another source is the example given.
• Distributional considerations matter: the principle should be applied so as to reduce extreme burdens on those least able to bear them.
• On the founding texts: the 1990 Ministerial Declaration and the 1992 Rio Declaration and Framework Convention can be read compatibly with this analysis, and their cost-effectiveness references are attuned to the risk of excessive expense. The 1994 “Seas At Risk” declaration comes closest to straightforward maximin and is therefore vulnerable to the objections raised.