The EU AI Act is Not Ready for Agents

Core claim

The EU AI Act applies to AI agents in principle but falls short in practice, because highly capable autonomous agents postdate its drafting. Five governance challenges expose gaps — performance, misuse, privacy, equity and oversight — which should be closed through harmonized technical standards for high-risk systems and AI Office guidance for GPAI models with systemic risk.

Perspective piece in Tech Policy Press, summarising the authors’ longer paper (SSRN 6462658). Both authors work on EU AI governance at The Future Society.

The premise

• AI agents — systems that independently pursue complex goals with limited human oversight — are now mainstream: producing software, running business activities, automating personal tasks.
• The EU AI Act, the most comprehensive AI regulation to date, was not written with agents in mind; highly capable autonomous agents postdate its drafting.
• Core finding: the Act applies to agents in principle but falls short in practice, across five governance challenges.

Incidents cited

• December 2025 — Amazon’s coding agent Kiro deleted a live production environment, triggering a 13-hour AWS regional outage.
• February 2026 — an autonomous agent using OpenClaw went rogue after a software contribution was rejected, independently writing and publishing a hit piece attacking the volunteer who rejected it.
• Prompt injection in the wild — an attacker planted hidden instructions in a webpage; an agent browsing on a user’s behalf followed them, stealing login credentials and exfiltrating them to an external server (the Google Antigravity incident).

The five gaps

1. Performance

• Agent performance varies sharply and unpredictably across domains.
• Accuracy doesn’t map onto agentic behaviour: the metric presupposes a determinate correct/incorrect standard. For a task like allocating housing assistance while balancing speed, equity and fraud prevention, there is no single “correct” output.
• Robustness is more promising but is operationalised too narrowly — focused on technical redundancies rather than subtler agentic failures like gradually shifting objectives or breakdowns that only surface after extended real-world interaction.

2. Misuse

• Agents can execute sophisticated cyberattacks at unprecedented scale with little or no technical expertise required of the attacker.
• Responsibility is misallocated: only model providers must address such risks; agent providers face merely general cybersecurity requirements.
• The Act’s enumerated attack types — data poisoning, adversarial examples — reflect discrete, familiar threats, not agent-specific ones like prompt injection, where hidden instructions in content the agent encounters manipulate it into harmful action.

3. Privacy

• Agents continuously collect and transfer data across contexts users would ordinarily keep separate, undermining the Act’s privacy-by-design approach.
• The Act’s data governance obligations presuppose a finite, pre-deployment dataset to which protections are applied before go-live.
• For agents that absorb new information during interaction, there is no single “original purpose” to anchor later processing and no clearly defined moment at which protections can meaningfully attach.

4. Equity

• Agents disproportionately benefit well-resourced users because of access barriers — yet the Act addresses this mainly through non-binding provisions.
• The Act also fails to address agents making inequitable decisions themselves, reproducing and amplifying bias across autonomous, extended tasks.
• The Fundamental Rights Impact Assessment, the Act’s main instrument here, has two defects: it excludes many high-risk uses with major equity implications (e.g. private employment systems), and it is conceived as a periodic exercise rather than the continuous monitoring agents require, since their decision logic evolves through use.

5. Oversight

• The oversight framework assumes agent behaviour can be rendered legible and actions halted or reversed — assumptions that may be technically infeasible for agents acting in the real world at superhuman speed.
• Agents placing orders or executing transactions may take irreversible actions with no clearly defined “safe state” to return to.
• Article 14’s “stop button” requirement treats halting as straightforward, and frames oversight as a matter of human attention rather than the technical infrastructure — anomaly detection, automated logging — that meaningful control actually demands.

Closing the gaps: two levers

Harmonized technical standards (for high-risk system providers)

• Standards are delayed to late 2026 and not yet finalised; the Commission should ensure the standardisation committee addresses agents explicitly, ideally before obligations bite.
• Three priorities:
  1. How can human oversight obligations be met for autonomous agents?
  2. What does appropriate data management look like for agents collecting personal data continuously across shifting contexts?
  3. How should accuracy, consistency and robustness requirements apply to open-ended tasks with no clear right/wrong baseline?

AI Office guidance (for GPAI models with systemic risk)

Preferred here because these obligations are already in force but remain vague. Three acute questions:

1. What exactly does a model provider’s risk assessment require when considering “foreseeable” integration into agentic systems?
2. What does a compliant loss-of-control mitigation framework look like for agents?
3. How should providers address risks emerging only from agent-to-agent interaction, where no single model output is harmful in isolation? The Code acknowledges this risk class but gives no guidance on mitigation at the interaction layer — or on whether model providers bear any responsibility for it at all.