Abstract
AI policy objectives depend on one’s strategic vision — a high-level view of how to navigate the transition to powerful AI. Mapping nine such visions against three severe risks (takeover, war, power concentration) shows that some policy objectives are robust across almost all visions while others are sharply contested, and that disagreement traces back to eight empirical cruxes about timelines, takeoff, leads, and institutional trustworthiness.
Purpose and structure
- The report aims to help policy entrepreneurs choose which strategic visions to embrace and identify policy objectives robust to uncertainty about which vision is best.
- Logical progression: views on cruxes → choice of strategic vision(s) → policy priorities.
- Section B outlines nine visions; Section C assesses six policy objectives against them; Section D collates the cruxes.
Scope and key concepts
- Focus is on the transition to artificial superintelligence (ASI): systems significantly outperforming humans across a wide range of cognitive domains.
- Three risks in scope:
- AI takeover — misaligned systems permanently disempower humanity
- War — geopolitical competition or misperception over AI escalates into great-power conflict
- Extreme power concentration — control over ASI centralises, foreclosing society’s ability to course-correct
- Misuse risk is deliberately excluded, on the grounds that mitigations (refusal training, withholding dual-use capabilities, defence-in-depth) are broadly similar across visions and don’t depend much on geopolitical structure.
- Key concepts: AI timelines, takeoff dynamics, intelligence recursion (automated AI R&D designing more capable successors), and the alignment tax — the extra resources required to build robustly aligned rather than merely capable models.
- Two organising axes: centralised vs distributed control, and private vs government control.
The nine visions
| # | Vision | Core logic |
|---|---|---|
| 1 | Competing private projects | Status quo; many companies compete under light-touch regulation |
| 2 | Single private project | One company builds a dominant lead and shapes ASI unilaterally |
| 3 | Global private project | One internationalised company develops frontier AI with globally sourced inputs |
| 4 | US leadership with domestic regulation | Private development plus meaningful US oversight; export controls limit foreign competition |
| 5 | US centralized government project | The US government takes direct control of frontier development |
| 6 | US + allies government project | As above, with allied governments participating |
| 7 | International competition and deterrence | Rival national projects in managed competition, constrained by mutual deterrence |
| 8 | Global centralized government project | Major powers jointly run a single project with shared control |
| 9 | Global coordinated regulator | An international body oversees frontier development in all key jurisdictions |
1. Competing private projects
- Treats AI as a normal technology; safety is continuous with regular product safety, driven by iteration and real-world deployment.
- Assumes alignment is achievable with modest investment, so market demand suffices; advances in commitment technology could let firms bind themselves without government enforcement.
- On war: AI does not become extremely geopolitically important — “perhaps less important than oil was in the 20th century.”
- Advantages: competitive ecosystem keeps prices down and innovation fast; companies may understand risks better than governments; no novel governance structures needed.
- Disadvantage: if alignment needs significant investment, individual firms underinvest because safety benefits are shared and costs are private.
2. Single private project
- Requires the leader to have a lead large enough to slow its automated AI R&D and pay the alignment tax.
- Explicitly rejects power concentration as something to avoid — the claim is that power must be concentrated in the right hands, with good values and governance.
- Advantages: sidesteps difficulties of regulation and international negotiation; minimises competitive deployment pressure.
- Disadvantages: severe legitimacy problems (decision-makers selected by market dynamics, not deliberate process); small groups may simply decide worse; pursuing a decisive lead could intensify racing industry-wide.
3. Global private project
- Arises from natural consolidation under long timelines and rising pretraining costs; governments involved via sovereign wealth funds and merger approvals, and optionally banning rival development.
- Proposed by Nick Bostrom (“Open Global Investment”).
- On war: major powers are financially invested in the project’s success, so are not incentivised to sabotage or intervene militarily; interdependency reduces war risk.
- Advantages: multinational corporations are a tried-and-tested legal structure; the project internalises a large share of benefits so is motivated to manage risk.
- Disadvantages: single point of failure; shortsighted investors may prioritise monetisation.
4. US leadership with domestic regulation
- The government requires compelling safety cases before frontier training runs and thorough third-party evaluations before public deployment.
- On war: because development remains private-led, adversaries may be less alarmed than under overt nationalisation or militarisation.
- Associated with Dario Amodei, emphasising both safety and US dominance to preserve democratic values.
5–6. US (and allies) centralized government projects
- Apollo-style consortium or single prime contractor; the lead funds alignment work and careful deployment without market-share pressure.
- Highest risk of triggering an arms race or hot war; the allied variant reduces power-concentration concerns but still risks provoking China.
- Expert forecasting gives a median 34% probability of government control of advanced AI.
7. International competition and deterrence
- Mutual deterrence (MAIM-style) constrains reckless racing; the future is shared between rival powers.
- Preserves autocratic influence over global affairs; deterrence equilibria may be unstable under differing understandings of what is unacceptable.
8. Global centralized government project
- Advantage: buy-in from all major powers minimises AI-related war risk and circumvents race dynamics, freeing capacity for alignment and control.
- Disadvantage: including autocracies preserves their influence over global affairs, which may be worse than a future where liberal democracies collectively control ASI.
9. Global coordinated regulator
- Multiple developers may still operate but are blocked from further capabilities research absent demonstrated alignment adequacy. Requires inspection powers, sanctions, or authority to physically intervene.
- On war: reassurance depends on more than accident-safety — countries may still fear “safe” systems being used against them deliberately. A possible fix is verifying that systems could not be used for aggression, though such mechanisms remain speculative.
- Disadvantage: subject to political capture — countries pressuring for fast approval of their own models and delay of rivals’.
Policy implications (six contested goals)
Increase AI developer cybersecurity
- Example policy: require security level 4/5 for eligibility for US government contracts.
- Broad agreement on at least level 3 (blocking terrorists and rogue actors). Visions relying on a durable lead benefit most from theft-resistance.
- Counterintuitive objection: level 5 could increase physical conflict risk in internationalised visions — a rival unable to level the playing field via cyber means may consider kinetic strikes on data centres. MAIM depends on all parties retaining some ability to disrupt reckless development.
Increase AI developer transparency
- Most valuable for the US domestic regulation vision; also helps make the government aware of risks and so enables stricter regimes later. Internationalised projects would be transparent by default.
- Against: if governments can’t interpret disclosures, requirements slow developers without reducing risk; in a US(+allies) project, secrecy limits what adversaries learn.
Sharpen US government focus on AGI/ASI
- Favoured by government-led and (indirectly) internationalised visions, since these require US buy-in.
- Against: risk of an “unhappy valley” where the government knows enough to think it can regulate wisely but not enough to actually do so.
Improve US-China coordination on AI risk
- Crucial for cooperative visions; dialogues form the basis for deeper cooperation.
- Against: it makes internationalised ASI governance more likely, undesirable if one prefers a US(+allies)-dominated future.
Increase the US lead over China
- The US currently leads by an estimated seven months, though estimates vary. Tools: export controls, immigration policy, domestic compute investment.
- Against: a large lead may encourage the US to pursue unilateral ASI, with associated misalignment, war and power-concentration risks.
Limit the pace of frontier capabilities advancement
- For: buys time to implement time-consuming visions, especially those requiring international negotiation.
- Against: the closer one’s preferred vision is to the status quo, the more valuable fast progress is before the strategic picture shifts. Unilateral pauses are hard to verify and may shift leadership to less safety-conscious actors.
Eight cruxes
Many cruxes are correlated — a short-timelines cluster typically also expects worse government involvement, weaker international coordination, and higher misalignment risk.
- AGI will come soon (roughly within a decade). For: company and expert forecasts of 2–10 years; extrapolation of recent progress. Against: LLMs still struggle with novel insight and on-the-job learning. Short timelines favour competing private projects and domestic regulation over international arrangements.
- ASI will follow shortly after AGI. For: automated AI R&D expanding the effective workforce; compute-efficiency breakthroughs; enormous revenues funding research. Against: first AGI systems may be expensive to run; automated R&D may plateau as ideas get harder to find. Rapid takeoff makes private-led development more likely, since governments act slowly; a longer lag allows using human-level systems for alignment and coordination work.
- One actor can build a large lead. For: natural-monopoly flywheel effects; persistent US compute and supply-chain advantage; US/allied dominance in algorithmic innovation. Against: loosened export controls and chip smuggling; potential loss of the US talent advantage; the difficulty of information security against state actors. Note that even with identical models, inference-compute differences can produce very different effective capabilities.
- ASI will confer a decisive strategic advantage.
- The US government will see AI as a top strategic priority.
- International cooperation is feasible and sustainable.
- Governments are more trustworthy AI developers.
- Alignment and control of superintelligence is profoundly difficult.