AI Governance at the Frontier: Unpacking Foundational Assumptions

Abstract

Policymakers can navigate the proliferation of AI governance proposals by identifying the assumptions each one presumes to be true. Assumptions shared across proposals are the enablers whose success multiple frameworks depend on; investing in them lets policymakers act now while preserving future decision-making flexibility, and surfacing them clarifies where stakeholder disagreement is real versus apparent.

The analytic approach

• An AI governance proposal is defined as any proposal seeking to harness benefits and mitigate risks by governing AI development (including inputs like data or compute) or deployment.
• Proposals are broken into four key components via guiding questions:

Question	Description	Examples
Why govern?	Objectives or reasons for governing	Promote innovation; protect humanity from catastrophic risks
What to govern?	Broad areas to be governed	Test and evaluation procedures; AI competition
Who governs?	Actors leveraging tools or mechanisms	Congress; AI developers
How to govern?	Tools or mechanisms	Enforce existing laws; map risk profiles to frontier models

• When to govern was excluded because proposals rarely discuss sequencing, though timing is acknowledged as important.
• Assumptions are derived from interactions between components, generating three analytic questions:
  1. Which risks are important to mitigate and who should have primary oversight? (from “why govern”)
  2. Who is delegated tasks and able to play a role? (from “who governs” × “how to govern”)
  3. Would the proposed mechanisms or tools actually achieve the objectives? (from “why” × “what” × “how”), broken into: which techniques are effective, which processes or frameworks are necessary, and which information or actions are useful.

Selection criteria and limitations

• Proposals were chosen because they govern frontier AI models, prescribe concrete policy actions, and are sourced from different stakeholder groups.
• Case studies are explicitly not representative of their sectors; disagreement exists within each group. The set also lacks international perspectives, since all authors are US-based.
• The paper was drafted before California SB-53 passed, so it analyses SB-1047, the earlier and more comprehensive related bill.

The five proposals

Industry — OpenAI’s Approach to Frontier Risk (Oct 2023)

• Published in response to the UK’s request for voluntary commitments at the AI Safety Summit.
• Why govern: protect humanity from frontier AI risks, including catastrophic risks.
• Covers six governance areas: frontier model development (Preparedness Framework, Deployment Safety Board with Microsoft), research into frontier risks (Superalignment and Preparedness teams), test and evaluation (external red-teamers, the Red Teaming Network, ARC/METR, system cards), post-deployment safety and security, data input filtering, and output measures (watermarking, classifiers, metadata).
• Uniquely focused on internal governance rather than external rules.

Civil society — Zero Trust AI Governance (Aug 2023)

• By Accountable Tech, the AI Now Institute, and EPIC. “Zero trust” borrowed from cybersecurity: constant verification rather than default trust.
• Why govern: the federal government should mitigate harms by changing the incentive structure of AI development.
• Mechanisms: enforce existing anti-discrimination, consumer protection and competition law; clarify Section 230 limits; provenance and disclosure standards; prohibit unacceptable use cases and most secondary uses of personal data; FTC structural interventions against “toxic competition”; require developers to affirmatively demonstrate compliance; grant third-party auditors full API and data access.
• Distinctive in its focus on consumer safety and competition policy, and in shifting the accountability burden from those harmed onto developers and deployers.

Academia — Frontier AI Regulation: Managing Emerging Risks to Public Safety (2023)

• Preprint by 24 scholars from think tanks, universities, technology companies and a law firm.
• Notable for its extensive scope and for explicitly stating preconditions (e.g. auditors needing adequate resources and time) and uncertainties (e.g. whether scaling trajectories continue).
• Mechanisms include capability and controllability assessments, repeated risk assessments with rollback, user-behaviour tracking, incident reporting, impact monitoring, and a licensing regime.

State government — California SB-1047

• Holds industry actors civilly liable for damages unless they implement safety and security protocols and comply with the law, while directing the state government to maintain infrastructure, update the “covered model” definition, and receive protocol reports.
• CSET judges primary oversight to rest with the state government, since it maintains public infrastructure, issues compliance guidance and enforces.

Federal government — Framework to Mitigate AI-Enabled Extreme Risks

• Bipartisan congressional framework (Romney, King and colleagues), later reflected in the Preserving American Dominance in AI Act.

Findings

Shared assumptions

• Risks and oversight: the majority of proposals assert frontier AI could present severe catastrophic risks (e.g. CBRN), and most assign primary oversight to government actors. OpenAI’s is the outlier, assuming industry actors should hold primary oversight.
• Delegated actors: all proposals delegate governance responsibilities to key actors, though scope and actor types vary. Most proposals view AI-enabling talent and AI processes and frameworks as important enablers.
• Techniques: proposals lack consensus on which techniques most effectively mitigate AI risks and harms.
• Information and actions: all five require some form of information sharing between government, developers and users. The two government proposals (SB-1047 and the federal framework) uniquely share assumptions about compute thresholds, customer screening for compute access, and reporting from compute providers — unsurprising, as compute is a lever governments can adjust as demands change.
• Two assumptions hold only conditionally: licensing frontier systems and expanding compute access work only if they balance risk mitigation against innovation — licensing must not lock out less established firms, and expanded compute access trades competition and equity gains against misuse risk.

Unique assumptions

• OpenAI assumes risks are best managed at the deployment stage, containing more deployment- than development-stage mitigations. Zero Trust and the academic proposal instead span the entire lifecycle.
• SB-1047 is the only proposal assuming “kill switch” mechanisms exist — that developers can fully shut down a model and compute providers can fully cut off a customer’s access. It also assigns compute providers the most responsibility in an AI-related emergency.

Two takeaways for policymakers

1. Use assumptions to understand disagreement precisely. Apparent disagreement may stem from differing assessments of empirically investigable assumptions — e.g. OpenAI, SB-1047 and the federal framework assume preventing model leakage and theft is technically feasible, while the civil society and academic proposals don’t address it, which could reflect either irrelevance or a judgment that it won’t be solved soon.
   • Conversely, apparent agreement can obscure complexity. Policymakers should be wary of grouping stakeholders by shared risk assessments: the academic proposal and SB-1047 both cite catastrophic risk, but only SB-1047 mandates compute buyer/seller reports, and only the academic proposal advocates licensing — mechanisms of very different difficulty placed on different stakeholders.
2. Act under uncertainty by addressing common assumptions. Shared prerequisites do not always exist and may need policymaker support: funding R&D of information-sharing channels, for example. Shared reliance on common actors (third-party auditors, non-AI-domain experts, compute providers) also risks overburdening them — suggesting the need to distribute responsibilities proportionately, foster partnerships, or incentivise new entrants.

Appendix — assumption interdependencies

• Assumptions group into categories that constrain one another:
  • Factual — about the state of the world
  • Capacity — about actors’ expertise and resources
  • Mechanism — about feasibility and effectiveness of techniques
  • Normative — about values and how the world should be
• Example: the normative assumption that government should hold primary oversight is contingent on the capacity assumption that government has adequate resources to provide it.
• Multi-dimensional analysis across these categories can anticipate points of failure, though the report treats it as out of scope.

Conclusion

• Future proposals will recommend different actions as technology and society change, but assumptions may have more staying power than proposal details — the need for actionable risk management frameworks and effective content provenance techniques remains current despite the proposals being a couple of years old.