Regulatory Markets: The Future of AI Governance

Abstract

AI regulation faces two competing deficits. A technical deficit: legislatures and regulators cannot rapidly translate command-and-control legal requirements into technical ones. A democratic deficit: relying on industry for technical standards leaves values-based decisions with politically unaccountable private actors. The proposed solution is regulatory markets — governments require regulated firms to purchase regulatory services from a government-licensed private regulator, so that policymakers set the goals while market forces and industry R&D pioneer the technical methods of achieving them.

Publication details

• Gillian K. Hadfield (Johns Hopkins University and Vector Institute) and Jack Clark (Anthropic).
• First posted to arXiv 11 April 2023; revised through v5, 3 February 2026. Published as 65 Jurimetrics J. 195–240 (2026).

The governance challenge

• Policy should focus less on computational technique and more on which AI systems create new governance challenges — chiefly, how far a system’s behaviour departs from what its designer contemplated and intended. Purely mechanical systems (an internal combustion engine) can be made safe through testing and formal verification; AI cannot be handled the same way.
• Two reasons the designer-behaviour gap grows. First, AI is valuable precisely because it solves problems humans cannot — “the goal in many cases is superhuman performance,” and a system whose intelligence surpasses its designers’ is hard to predict, check, or modify. This is why governance-oriented definitions like the OECD’s emphasise quasi-autonomy: even a system that merely recommends can change the reasoning and behaviour of humans who cannot match it. Second, intelligent systems are general: engineers building a facial recognition or language model “cannot possibly anticipate and test for all the different ways in which the models could be used.”
• AI is already a general-purpose technology — “a method of inventing inventions” — with a low barrier to entry, so developers ship tools they cannot validate across all uses, and deployers sit too far from design to foresee all failure modes.

The landscape and its two deficits

• The EU AI Act creates significant compliance obligations at risk of substantial penalties; the U.S. and U.K. rely primarily on voluntary and sectoral approaches. But the authors argue these “do not differ as much as might otherwise seem”: with the exception of the DMA and DSA, all current approaches are forms of management-based or risk-based regulation in which industry identifies and manages the risks of its own products. None yet constitutes a robust response.

Technical deficit

• A lack of technical detail telling developers and deployers what operational characteristics their systems must have. “It is one thing to say that an AI system must be ‘fair;’ it is quite another to say what ‘fair’ translates to in terms of system performance,” especially given the multiplicity of statistical fairness measures and the impossibility of satisfying all of them. The same holds for “explainable” and “robust.”
• Contrast with conventional product safety, where regulation supplies concrete benchmarks. Canada’s toy safety rules specify that a doll or plush toy fails “if samples of its outer fabric, held at an angle of 45 degrees, ignite within 1 second of contact with a flame and the flame travels a distance of 127 millimetres (5 inches) in 7 seconds or less.” U.S. aviation software demonstrates FAA compliance through the technically detailed private standard DO-178C.
• Risk-management systems, mandatory or voluntary, leave “all of the technical detail… to be decided by the business internally.”

Democratic deficit

• Relying on standards-setting organisations delegates “the fundamentally political tasks of reconciling important trade-offs in the design and deployment of AI systems to politically unaccountable private actors.” The concern is that governments are simply failing to make the hard choices.
• ISO is a nonprofit network of national standards bodies of varying independence from government (the British Standards Institute is independent; the Standards Council of Canada is a Crown corporation accountable to Parliament). Its standards are developed by technical committees of industry experts, mostly employed by the companies that will adopt the standards, and are proprietary — purchasable, not published for public review or comment. (A March 2024 CJEU decision may change this in Europe for standards referenced in EU law.)
• Even where civil society can participate, there is “substantial imbalance in the capacity for large corporate entities compared to small businesses and nonprofit public interest groups” to fund full-time participation.
• Crucially, the authors’ objection is not procedural: “We do not think the problem can be solved simply by making SSOs more open and democratic in their processes.” The concern is with the premise that AI standards are properly generated by actors not accountable to the public.

The regulatory markets model

Three principal actors:

• Targets — the companies building, deploying, or integrating AI.
• Private regulators — for-profit and nonprofit organisations developing and selling regulatory services in competition with one another.
• Governments — which require targets to purchase those services (entering a regulatory contract) and directly regulate the market for regulatory services to ensure it serves the public interest. Private regulators derive authority from the contract plus government authorisation to collect fines or impose requirements.

How competition is structured:

• Regulators must be licensed in each jurisdiction where they operate; multiple are licensed in any domain; targets must choose one but can compare and switch.
• Targets and regulators compete on cost and efficiency — not on the quality of regulatory outcomes. To obtain and keep a license, a regulator must demonstrate that its approach achieves outcomes mandated by government, set through public-sector processes. This is the mechanism that “makes legitimate” the delegation and overcomes the democratic deficit.
• Illustrative government-set outcomes: for self-driving cars, metric thresholds for accident or congestion rates, plus principles like maintaining public confidence in road safety; for banking, thresholds for consumer access to credit, with principles of transaction traceability and market stability; for facial recognition in drones, thresholds for the likelihood of malicious access, with principles of realistic consumer consent and comparable identification rates across demographic groups.
• The central innovation relative to existing new-governance models: government shifts to setting goals rather than methods (as in performance-based regulation), but the methods are developed by independent private regulators who are themselves regulated — rather than left to the regulated entities, as performance-based regulation does.
• Oversight combines upfront evaluation of a regulator’s capacity with ongoing auditing: tracking accident and congestion rates in autonomous vehicles, periodic audits of random transaction samples in banking, stress-testing procedures in drones.

Regulatory technology. Private regulators could use conventional means — rules, monitoring, penalties — but “the primary goal” is encouraging investment in technologies that directly shape target behaviour. Precedent is RegTech in financial regulation, which took off after 2008 as compliance requirements multiplied, met by private innovation in data scraping, big data analysis, and machine learning; the Bank of England’s Chief Economist articulated a 2014 “dream” of regulators with “a bank of monitors tracking the global flow of funds in close to real time.”

Worked example: red-teaming frontier models

• Red teaming already exists inside OpenAI and Anthropic, so demand exists and regulatory technology can plausibly emerge there. It has policy currency too: EO 14110 defines red-teaming, directs NIST to establish guidelines, and requires dual-use foundation model developers to report red-team results to the Secretary of Commerce; the EU AI Act also references adversarial testing.
• But these efforts suffer the democratic deficit — “the criteria for evaluating the value and efficacy of these technologies are being set under corporate governance within private companies.” Developers are already moving from wholly internal teams to nonprofits like METR and Apollo Research and consultancies like Gryphon Scientific.
• Sketch of implementation, using the objective of reducing bioweapons misuse risk: governments announce a timeframe for licensing independent red-teaming and evaluation companies and a date by which frontier developers (open- or closed-source) must contract with one. Given how little is understood, initial licensing would rest on an expert group assessing companies against a general principle — e.g. that a method “warrants high confidence that a model is not vulnerable to state-of-the-art adversarial efforts to increase baseline capabilities among non-state actors to produce category A, B or C bioterror toxins as classified by the U.S. CDC.” Judgments about “high confidence,” “not vulnerable,” and “state-of-the-art” are left to the expert group on an ongoing basis.

Opportunities

• Correcting the investment imbalance: society is “investing billions in building evermore powerful AI capabilities but very little in building comparably powerful AI governance tools.” Because AI’s speed and complexity will require other AI technologies to govern it, private-sector incentives must be harnessed to attract capital and expertise. Recruiting technical experts into government is not enough — “governments are not designed to take the risks, financial and otherwise, needed to innovate and deliver core technologies.” The model creates opportunities for people currently building safety and oversight tools inside AI companies to found their own businesses, and for venture capital to place multiple bets on which approaches succeed.
• Globalization: AI is deployed across jurisdictional boundaries and trained on global data flows, raising pressure for harmonised standards — but harmonisation is “often a pipe dream,” as jurisdictions are too varied, dynamic, and protective of sovereignty to cede rulemaking to supranational bodies.
• Regulatory disruption: because AI is a general-purpose technology, it threatens to disrupt almost all existing regulatory goals across sectors.

Limitations and risks

• Competition failure. Insufficient scale (if only two or three firms develop a given type of AI, a competitive market of regulators cannot be sustained), excessive market share concentration, high switching costs, or collusion. Partial fixes: antitrust and competition law, plus possible market-share limits and rules reducing switching costs.
• Independence and capture. Capture already afflicts government regulation through corruption and subtler channels — campaign finance, lobbying, overlapping professional networks, dependence on industry-supplied information. Regulatory markets add a layer, creating the risk that private regulators “will collaborate with those companies to cheat on government goals.” Mitigations require governments to monitor outcomes and hold credible threats to condition, suspend, or revoke licenses.
• But the model may also make regulating easier: multiple regulators mean multiple sources of data and expertise; market participants have incentives to expose competitors “cheating” on outcomes to win share; and instead of regulating 1,000 companies, government regulates perhaps five or ten regulators. “But regulatory markets only work if governments are willing to regulate private regulators. The model cannot fix a lack of political will.”
• Failure to invest in oversight — the core risk, illustrated by two cautionary tales: credit rating agencies in the 2008 financial crisis, which were deliberately shielded from liability for rating errors with no formal government oversight; and FAA oversight of the Boeing 737-MAX, which was “grossly underfunded and inadequate, as repeated reports from government inspectors before the crashes made clear.” Both point to the need for a sustainable funding model reflecting the true cost of regulation — some of which the market would price directly rather than leaving it wholly to taxation and budgets.

The urgency argument

• The proposal is offered “not merely as an addition to the literature on regulation” but out of concern about “the impending and potentially widespread failure of our regulatory models.”
• The authors acknowledge the feasibility objection: implementation would require novel licensing systems, possibly new agencies, new metrics and methods aimed at intermediaries rather than targets, a nascent sector of regulatory intermediaries attracting investment, legislative agreement on incentives to purchase regulatory services, and new collaboration between regulators and targets on liability. Above all, “a shift to regulatory markets requires countries to bet on a new approach to regulation at a time when the stakes seem incredibly high: Will the United States, for example, risk regulatory missteps that could diminish its lead over China in AI development?”
• They also stress continuity with existing trends — private vendor compliance technologies, greater regulator reliance on digital tools in finance, outcome-metric rather than prescriptive models, and a growing role for third-party intermediaries.
• The closing position: the theoretical and practical challenges are real, “But we also think that the AI governance challenge cannot be met without regulatory innovation,” since AI technologies “are galloping past the regulatory frameworks put in place to manage the twentieth-century world.”