AGI, Governments, and Free Societies

Abstract

AGI threatens the delicate equilibrium between state capacity and individual liberty that sustains free societies, risking a slide toward either a “despotic Leviathan” of unprecedented surveillance and control or an “absent Leviathan” in which AGI-empowered non-state actors erode state legitimacy. Preserving the narrow corridor requires robust technical safeguards, hybrid institutional designs retaining meaningful human oversight, and adaptive regulatory mechanisms.

Three channels of transformation

The paper analyses AGI’s effects on liberal institutions through three channels, at micro (individual bureaucratic decisions), meso (organisational structure) and macro (democratic processes) levels:

1. Automation of discretionary decision-making within agencies
2. Evolution of bureaucratic structures toward system-level architectures
3. Transformation of democratic feedback mechanisms

How AGI could reshape government

1. Deep integration within government decision-making — adopting AGI in daily operations requires tailoring products to civil servants’ workflows and fostering a culture that embraces experimentation. Governments will increasingly be responsible for maintaining complex networks of infrastructure and sensors, plus their governance, oversight and regulation.
2. Restructuring the machinery of government — some departments may expand while others become obsolete. AGI’s capacity to synthesise enormous amounts of data while accounting for multifaceted preferences and constraints opens possibilities for more responsive, equitable and effective governance mechanisms.
3. Reinforcing democratic feedback loops — the model in which a single MP acts as agent for constituents’ views could be improved, addressing voter apathy, information asymmetry, and the disconnect between officials and constituents.

Whether the net effect is positive depends on who uses the technology and how. AGI can significantly empower both authoritarian and democratic states; part of the challenge is ensuring the latter stays ahead.

Historical pathways to free societies

• Liberal democracy is a relatively recent phenomenon. The printing press and a growing mercantile class enabled administrative record-keeping amid demands for property and contract enforcement, laying groundwork for centralised governance (Dittmar, 2011).
• Paradoxically, in the aftermath of the European wars of religion, this centralisation created the conditions for constitutional constraints on power and the modern Westphalian nation-state. The Glorious Revolution (1688) established parliamentary sovereignty, limited monarchical authority, and enforced Acts of Toleration.
• The Enlightenment catalysed liberal thought, expressed in the American and French Revolutions. Through the 19th century constitutional monarchies emerged and suffrage expanded — but industrialisation exposed the limits of laissez-faire, driving the transition from classical to modern liberalism and, in the early 20th century, the welfare state and modern administrative bureaucracies (Heath, 2020).
• The parallel: just as the printing press and industrial revolution necessitated reformulations of the social contract and the structure of government, the AI revolution is likely to require similar adaptations.

Current institutional features and their payoffs

Free societies are characterised by rule of law and an independent judiciary; separation of powers and checks and balances; protection of individual rights and civil liberties; competitive markets with secure property rights; and freedom of expression and a free press.

These generally outperform alternatives on:

1. Economic prosperity — property rights, contract enforcement and open markets favour investment, innovation and entrepreneurship.
2. Innovation — economic freedom plus rights protection lets new ideas flourish.
3. Social progress — free advocacy plus democratic translation of public will allows self-correction; democratic accountability means poor performance can be sanctioned at the ballot box (Schedler et al. 1999).
4. Resilience — distributed power and self-correction make them more adaptable to shocks.
5. Human development — higher scores on education, health outcomes and life satisfaction.

Why the corridor is narrow

• Acemoglu & Robinson’s narrow corridor is the space where state capacity and societal power are in equilibrium. On one side lies the absent Leviathan (too weak to provide services, maintain order or protect rights); on the other the despotic Leviathan (an overly powerful state suppressing society). Liberty thrives only in between.
• Few societies have entered and stayed in the corridor, maintained through coevolution between state institutions and societal organisations — ongoing negotiation in which societal forces push back against overreach while demanding effective governance.
• Reasons it is narrow: the inherent tension between needing a strong state and limiting it; the fact that both state and societal actors continually seek to expand power; the context-dependence of the balance point, making institutional models hard to import; and exogenous technological shifts in the state-society balance requiring constant adaptation. Geopolitical competition and global economic forces amplify these.
• Free-society institutions are not self-enforcing — they are a meta-stable social equilibrium dependent on contingent historical, social and technological factors.
• Evidence for technological drivers: Van Noort (2004) argued a larger manufacturing employment share makes mass mobilisation more likely and more costly to suppress, increasing mass power against autocratic elites. Early 20th-century democracies adapted through social security reform and formalised collective bargaining. Analogously, the Arab Spring and the contemporary rise of Western populism have been attributed to the internet’s capacity to potentiate mass mobilisation — suggesting 21st-century democratic legitimacy crises reflect a rebalancing driven by the digital revolution (Miller & Zissimos, 2022).

Seeing like an AGI state

New coordination mechanisms

• AGI could enable “AI-native” organisations leveraging thousands of agents to coordinate networks of human and artificial agents, potentially outcompeting traditional hierarchies.
• Sophisticated commitment devices and smart contracts could allow more direct and participatory democracy, with citizens making binding, dynamically adapting policy commitments.
• Agents could lower barriers to Coasean bargains by autonomously handling information gathering, negotiation and enforcement — overcoming transaction costs, information asymmetries and commitment problems, and enabling nuanced solutions where only coarse regulatory instruments currently exist.
• Risks: malicious actors could orchestrate large-scale coordination of unwitting participants toward harmful ends, including AI-assisted coups d’état. Who controls these capabilities, and under what conditions, becomes paramount — potentially warranting regulated access protocols analogous to advanced biotechnology.

Legibility

• Drawing on James C. Scott’s Seeing Like a State: governments can only regulate what they can measure, so rendering society legible is a major activity of modern government.
• AGI dramatically enhances the state’s capacity for legibility — real-time analysis of vast data streams enabling unprecedented monitoring and prediction of behaviours and threats.
• But legibility is not one-way. Democratised AI tools let individuals and non-state actors complexify behaviour and obfuscate activity: intricate financial structures, novel communication methods, synthetic identities. Legibility may also erode through the sheer volume of activity from billions of agents.
• This creates a “legibility arms race” — though paradoxically AGI itself could help navigate the resulting complexity, turning regulatory intricacy into a manageable feature.
• Digitalisation illustrates both sides: the shift to electronic payments greatly increased economic legibility and suppressed grey and black markets in countries like India, while quasi-anonymous cryptocurrencies expanded money laundering options despite the blockchain’s perfect transaction legibility.
• The key is balance: some legibility is necessary for governance and public goods provision, but excessive legibility leads to oppressive control.

Democratic input and “digital twins”

• Research cited suggests LLMs can be “a win-win: fairer voting outcomes for humans with fairer AI representation” — for simple ballots, consistency becomes significant under fair aggregation methods promoting proportional representation.
• Three takeaways:
  1. A sufficiently high-fidelity simulation would represent at least indirect democratic input. If a digital twin accurately represents your interests 99% of the time, it seems reasonable to make it your proxy when you lack time or contextual knowledge — increasing direct citizen feedback so much that the role of elected representative may shift toward oversight of the AGI system rather than representation of constituents.
  2. An AGI system could set the agenda for policy deliberation — generating proposals, ranking them, and soliciting feedback from digital twins and humans — further obfuscating the need for elected representatives.
  3. High-volume experimentation, ideally local and alongside current systems, would help identify the best socio-technical arrangement.
• Three concerns beyond the bias, transparency and accountability issues raised by Aoki et al. and Majumdar et al. (who ask who determines training data, how voter privacy and autonomy are protected, and whether citizens retain control over AI representatives):
  • Alignment — enmeshing unaligned AGI with democratic input makes the process manipulable
  • Accuracy — are all relevant factors available, measurable and measured?
  • Dignity — does delegating democratic input to a proxy throw away the chance to engage meaningfully in a joint democratic endeavour?

Securing the narrow corridor

The two-directional risk

• Toward the despotic Leviathan: AGI could make governments radically more efficient, responsive and data-driven, and help solve coordination failures and principal-agent problems — but the same enhancement enables unprecedented surveillance and control, stifling dissent and entrenching existing power. Automated administrative decision-making could erode human agency and democratic accountability as bureaucracies move toward centralised “system-level” architectures.
• Toward the absent Leviathan: if AGI diffuses faster among individuals and civil society than governments, it weakens state legitimacy and capacity relative to non-state actors — a hollowing out of governability and social cohesion, with malicious actors exploiting accessible AGI to undermine elections, manipulate opinion or coordinate insurgencies.

Administrative values at risk

• AGI systems acting as artificial bureaucrats could automate complex, high-discretion tasks traditionally requiring human judgment, promising efficiency, effectiveness and scalability gains.
• But equity is uncertain (systems may replicate or exacerbate biases in training data), and transparency and accountability could suffer from opaque decision processes. Mechanistic interpretability and explainable AI provide some tools, but alignment with democratic principles remains an ongoing task.
• Ethical implications: delegating value-laden decisions risks loss of moral accountability. AGI agents may excel at optimising for efficiency while lacking the ethical nuance to address competing societal values — a disconnect between computational optimisation and human morality that risks eroding public trust.

Proposed safeguards

• Technological: privacy-enhancing technologies (PETs) to counterbalance surveillance capabilities and preserve individual autonomy; investment in explainable AI and mechanistic interpretability to keep systems transparent and accountable.
• Institutional: hybrid AI-human governance structures combining computational power with human judgment and accountability; bureaucratic models that stay flexible and adaptable while retaining oversight; adapted regulatory and legal frameworks for non-state AGI use, including modernised antitrust paradigms addressing market concentration.
• Democratic: large-scale deliberative platforms, real-time citizen feedback systems and representative digital twins — with robust safeguards to prevent misuse — plus clear frameworks for democratic oversight ensuring artificial bureaucrats remain accountable to elected officials and the public.
• Epistemic: most importantly, a shift toward anticipatory governance — proactively imagining and stress-testing institutional paradigms rather than passively reacting. Deploy scenario planning, threat modelling and forecasting; run governance experimentation across scales from local sandboxes to international norm-setting.

Conclusion

• Achieving this requires a far more dynamic and adaptive approach to policymaking than most governments are accustomed to, greater collaboration between AI researchers, social scientists and policymakers, and honest reckoning with the ways existing institutions — from the administrative state to representative democracy itself — may be inadequate.
• “The great political question of the 21st century may well be whether liberal democracy can reform itself in time to reap the rewards and manage the risks of artificial general intelligence.”