AI agents promise to move artificial intelligence beyond answering questions and generating content. By connecting models to tools and business systems, organisations can give AI the ability to take actions on their behalf. But that capability also changes the security equation. A chatbot that produces an inaccurate answer creates one type of risk, where its human operator acts as a line of defence. An AI agent that can send emails, modify files, execute commands or issue customer refunds can turn a bad decision into an immediate real-world consequence.
The challenge is that large language models (LLMs) cannot operate with complete reliability. Their probabilistic nature means unexpected outputs remain possible. Prompt injection creates another problem: because an LLM does not maintain a reliable security boundary between instructions and the data it processes, malicious content can influence its behaviour. Taking steps to improve reliability and reduce prompt injection are prudent but will never achieve perfection. Security teams therefore need to think beyond just trying to prevent an AI system from ever making a bad decision. They also need controls that limit what happens when it does.
Excessive agency changes the risk equation
This is the problem described by “excessive agency”: giving an AI system more ability to act than it requires, expanding the blast radius of any erroneous or manipulated operation. The concept can be reduced to three principles: minimise functionality, minimise permissions and minimise autonomy.
Minimising functionality means giving an agent only the tools necessary for its task. Consider an agent designed to read an inbox and create a daily priority list. It needs access to email, but it does not need access to every file on the user’s computer.
The same principle applies to the capabilities of individual tools. If an agent needs to read a file, give it a function that only reads files. Do not provide arbitrary command execution simply because one of the commands it could execute happens to read a file.
The difference is essentially a nutcracker versus a sledgehammer. Both might crack a nut, but one creates greater scope for damage.
Apply least privilege to AI identities
Minimising permissions extends familiar least-privilege principles to agents. The inbox assistant, for example, needs permission to read messages. It does not need permission to send them. A malicious email containing a prompt injection could attempt to persuade the agent to forward sensitive information elsewhere, but any successful exploitation is neutered if an agent in unable to send emails.
OAuth scopes can help constrain an agent operating on behalf of a user so that it receives only the subset of permissions required. Where agents operate under their own identities, those identities should similarly receive only the privileges necessary for their defined functions.
None of this is conceptually new. The difficulty lies in implementing these controls consistently and without friction as agentic systems become more complex and interact with other agents and services.
Human oversight needs to be proportionate
The third principle, minimising autonomy, requires more nuance. Requiring human approval for every AI action sounds safe, but quickly becomes impractical. Not only will a human become quickly overloaded, but if a person approves the same harmless action 100 times, there is a strong chance they will approve the 101st without examining it carefully.
A better approach distinguishes between low-risk reversible actions, and higher-risk or irreversible decisions. Consider a customer-service agent authorised to issue refunds. Allowing refunds below $20 without individual approval might be reasonable. A refund above that value could require human confirmation.
Volume matters too. Twenty small refunds during a day could represent normal business activity; 10,000 should trigger scrutiny. Controls should also consider cumulative activity so an attacker cannot simply make repeated requests for $19.99 to remain below a per-transaction threshold. This creates an opportunity for graduated controls.
A “yellow line” can identify unusual activity without immediately stopping operations. Crossing it triggers human-on-the-loop oversight: the process continues, but a person is alerted to supervise it.
A “red line” represents a higher-risk threshold. Crossing it stops autonomous activity and introduces human-in-the-loop approval, meaning the process cannot continue until a person explicitly authorises it.
Humans do not need to sit in the middle of every AI interaction, but neither should organisations deploy autonomous agents and assume that initial permissions and instructions will keep them within safe boundaries.
Design for failure, not perfect behaviour
AI security cannot depend on the assumption that models will always behave as intended. Prompt injection, hallucinations and unexpected interactions between agents make that unrealistic. Controlling excessive agency is less about preventing every undesirable outcome and more about limiting the damage when something inevitably goes wrong.
Organisations deploying agents should ask three questions: Does this AI need every function it has? Does it need every permission it has? And does it need to make every decision autonomously?
AI agents derive their value from being able to act. Securing them does not mean removing that capability. It means establishing sensible boundaries around it and knowing when a human needs to be in the loop, on the loop, or able to pull the emergency brake.
As organisations move from experimenting with AI agents to embedding them into everyday business processes, getting those boundaries right will become increasingly important. The organisations that use agentic AI successfully will strike the right balance: giving systems enough autonomy to deliver value without allowing that autonomy to introduce unnecessary risk. For end-user AI products, that balance should be configurable by the user, so they can achieve their own risk appetite and adjust it as their faith in a system evolves.
AI agents do not need unlimited freedom to be useful. They need enough freedom to do their jobs, backed by controls that stop an unexpected decision from becoming an avoidable security incident.
Discussion about this post