Abstract
There is no single best plan for navigating the transition to superintelligence. Nine distinct strategic visions each make different tradeoffs across three severe risks — AI takeover, great-power war, and extreme power concentration — and rest on different empirical assumptions about timelines, takeoff, and international cooperation. Rather than betting on one, actors should pursue interventions that are robust across visions and prioritise research on the cruxes that separate them.
Blog post summarising the IAPS report “Strategic Visions in AI Governance” by Oscar Delaney, Maria Kostylew, Oliver Guest, and Peter Wildeford.
Framing
- Three severe risks structure the analysis:
- Takeover by misaligned AI systems
- Wars between great powers competing for AI dominance
- Extreme concentration of power
- Visions are organised along two axes:
- Private vs government control — who decides when and how advanced AI is developed
- Distributed vs centralised power — control spread across many actors, or concentrated in one project
- Which vision one prefers depends on unresolved questions: whether AGI comes soon, whether international cooperation can work, and whether alignment is profoundly difficult.
Cluster 1 — Private company-led development
Competing private projects (status quo)
- Multiple companies compete under light regulation; AI treated as a normal technology where market competition drives both innovation and safety, with antitrust and taxation sufficient against power concentration.
- Advocated by: Silicon Valley techno-optimists.
- Problem: assumes alignment is relatively easy. If not, competitive dynamics create a race to the bottom — safety benefits are shared broadly while costs are borne privately.
- Feasibility: the default trajectory; the question is whether a crisis or government intervention disrupts it.
Single private project
- One company automates AI R&D first, triggers recursive improvement, and gains an unassailable lead; governments are too slow to intervene. The large lead lets it invest heavily in alignment and control.
- Advocated by: no one publicly.
- Problem: a small group of unelected individuals making civilisation-shaping decisions is illegitimate and unlikely to produce good outcomes.
- Feasibility: currently unlikely — multiple well-resourced actors sit near the frontier.
Global private project
- Rising costs drive consolidation into a single multinational company, with financing, talent, compute and energy from many countries. Governments invest and approve mergers, but the entity keeps standard corporate governance.
- Advocated by: Nick Bostrom.
- Problems: still a single point of failure; investors may prioritise monetisation over risk management; democratic oversight is poor since non-shareholders are unrepresented.
- Feasibility: requires some international cooperation, but less than formal treaties.
Cluster 2 — US government-controlled development
US leadership with domestic regulation
- Private development continues in the US with active government oversight: safety cases before training runs, mandatory third-party evaluations, incident reporting. Internationally, export controls and diplomacy maintain the US lead over China.
- Advocated by: Dario Amodei, Helen Toner, Cullen O’Keefe.
- Problems: the federal government currently lacks technical talent for hands-on inspection; without a crisis, oversight may be weak; racing could produce perfunctory compliance.
- Feasibility: needs political will but is less radical than nationalisation; the US is already committed to parts of it.
US centralized government project
- The government centralises development — an Apollo-style consortium or a single prime contractor. A large lead funds alignment and control work and careful deployment without market-share pressure.
- Advocated by: Leopold Aschenbrenner, the US-China Economic and Security Review Commission.
- Problems: most likely vision to trigger an arms race or hot war; China would likely launch a rival project. Massive expansion of executive power could threaten constitutional checks and balances.
- Feasibility: requires major expansion of federal authority. An IAPS expert forecasting study gave a median 34% probability of government control of advanced AI.
US + allies government project
- The US leads a coalition (EU, UK, Canada, Australia, Japan, South Korea, Taiwan) in a joint intergovernmental project. Collectively they dominate semiconductor manufacturing, talent and financing; joint decision-making reduces power-concentration concerns.
- Advocated by: Forethought; possibly Haydn Belfield.
- Problem: still risks an aggressive Chinese response — a rival project or sabotage.
Cluster 3 — International approaches
International competition and deterrence
- Rival powers each develop AI, but mutual deterrence prevents reckless racing: each threatens to sabotage projects that race irresponsibly toward ASI, akin to nuclear deterrence. The future is shared between rival powers rather than monopolised.
- Advocated by: Dan Hendrycks (with Schmidt and Wang).
- Problems: preserves significant control for non-democratic states; deterrence equilibria can be unstable, and miscalculation could trigger conflict that escalates to nuclear war.
- Feasibility: depends on a conjunction of premises each only somewhat more likely than not; requires radical divergence from current international relations.
Global centralized government project
- Major powers consolidate frontier development into a single jointly controlled project with a strong safety focus and a large lead over other developers.
- Advocated by: various civil society organisations (Baruch Plan analogies, GovAI, A Narrow Path).
- Problems: still a single point of failure; requires governance robust enough to prevent capture or an unwanted slide toward world government.
- Feasibility: the most ambitious vision; especially unlikely if ASI arrives soon.
Global coordinated regulator
- An “IAEA for AI” oversees all frontier projects, blocking further capabilities research where alignment and control have not been demonstrated. A more radical version bans development above a capability threshold until alignment progress is made.
- Advocated by: 2023-era OpenAI, MIRI.
- Problems: vulnerable to political capture (pressure to approve domestic models, delay rivals’); verification is hard without invasive datacentre access; enforcement against non-compliant states is difficult.
- Feasibility: harder than comprehensive US domestic regulation, but requires less trust than full consolidation.
Transitions between visions
- Status quo → domestic regulation: growing awareness and political will, without a forcing crisis.
- Domestic regulation → government project: a crisis or security incident triggers nationalisation, or costs become prohibitive for private actors.
- US project → US + allies: allied governments demand inclusion.
- Any unilateral approach → MAIM dynamics: adversaries fear being left behind and act to prevent a decisive advantage.
- MAIM → global cooperation: a tense standoff is formalised into binding agreements.
- Actions advancing one vision can inadvertently trigger a transition to a less desirable one — e.g. a US government project intended to ensure safety may be exactly what starts an arms race.
Policy implications
Transparency into private developers
- In favour: government-oversight visions need visibility to design and enforce guardrails; international visions benefit from reduced information asymmetries.
- Against: private-led visions are mixed — if governments cannot interpret disclosures well, requirements slow developers without reducing risk. Some voluntary transparency may forestall heavier regulation.
Accelerating US AI progress
- In favour: status quo visions want fast progress before the geopolitical landscape shifts.
- In between: government-led visions need progress fast enough that China doesn’t catch up, slow enough for the government to step in.
- Against: international cooperation visions need time to negotiate agreements and build institutions.
Key uncertainties (cruxes)
Technical
- Will AGI come soon? Short timelines favour the status quo or modest domestic regulation; long timelines make government-led or international approaches feasible.
- Will ASI follow quickly after AGI? Fast takeoff favours centralised visions; slow takeoff makes multipolar outcomes more likely and gives governments time to respond.
- Can one actor build a large lead? Depends on persistence of compute advantages, diffusion of algorithmic innovations, and cybersecurity against model theft.
- Will ASI confer a decisive strategic advantage? If so, unipolar outcomes and MAIM dynamics intensify. What key actors believe about DSA matters as much as the reality.
- How hard is alignment and control? Profound difficulty favours centralisation to eliminate racing; tractability favours pluralistic distributed approaches.
Institutional
- Will the US government prioritise AI? Greater focus makes regulation and government projects more likely.
- Is international cooperation feasible? Depends significantly on advances in verification between distrustful parties.
- Are governments more trustworthy than companies? Turns on whether one weights democratic legitimacy over technical competence.
Lessons
- No vision dominates. Each trades off the three risks differently and rests on different assumptions; prepare for multiple scenarios rather than assuming one pathway.
- Favour robust strategies valuable across nearly all visions:
- Improve AI developer cybersecurity to at least RAND security level 4
- Develop verification technologies for future agreements
- Make progress on alignment and control research
- Build government expertise on AI risks
- Resolve key uncertainties, particularly: whether power concentration or race-driven corner-cutting is more concerning; whether a large lead can be built and maintained; and which geopolitical scenarios are stable and desirable.