Skip to content

Pattern

How to make AI agents safe, from permissions to prompt injection

You make an AI agent safe by limiting what it can do, not by trusting what it says. Give it the narrowest tools and permissions, run risky work in a sandbox, require a person to approve consequential actions, and treat every email, document and web page it reads as possibly hostile.

StudiedLast reviewed 4 min read

Explained from public sources. Mai has not built this yet.

On this page

In plain English

An AI agent can be talked into things. It reads emails, documents and web pages, and it cannot always tell the difference between information and instructions, so a cleverly worded message can steer it. The answer is not to hope it resists, but to build the system so that even a confused agent cannot do much harm: it only has the keys it needs, it works inside a fenced-off area, and anything that matters waits for a person to say yes.

How it works

Safety comes from layers outside the model, each of which still holds if the model gets it wrong.

Input: Your request, Emails and files. Then Agent. Then Controls: Allow-listed tools, Scoped permissions, Sandbox. Then You approve. Then Audit log.

Untrusted input meets a confusable model, so the controls sit outside it

Least privilege. OWASP’s LLM Top 10 lists “excessive agency” as a core risk and breaks it into three causes: too many tools, too many permissions, and too much autonomy. The remedies follow directly. Offer only the tools the job needs, and prefer narrow ones (“send a reply to this thread”) over open-ended ones (“run any shell command”, “fetch any URL”). Use read-only scopes where reading is enough. Where possible, act under the identity of the person the agent is working for, so it cannot reach anything they could not.

Enforcement outside the model. The permission check belongs in the tool layer and the downstream system, not in the prompt. OWASP calls this complete mediation: the system that holds the data decides whether the request is allowed, whatever the model asked for. An instruction in the system prompt saying “never delete records” is a hint; a database role without delete rights is a control.

Allow-lists. Tools, recipients, domains and file locations work best as explicit lists of what is allowed, with everything else refused. A list of banned phrases, by contrast, is easy to get around, and the NCSC specifically warns against relying on deny-lists of known attack text.

Sandboxing. Where an agent runs code or shell commands, the operating system should enforce the boundary. Claude Code’s sandbox is a good public example: commands can write only inside the working folders, and network traffic goes through a proxy that allows only listed domains. The point is that the limit holds even if the agent is persuaded to try something else.

Approvals. Consequential actions, such as payments, external emails and deletions, wait for a person. See human approval gates for how to choose which actions need one.

Handling untrusted content. Indirect prompt injection is the agent-specific threat. Researchers showed in 2023 that instructions planted in content an application later retrieves can steer it remotely, without the attacker ever using the application. Practical defences: mark external content clearly as data rather than instructions, keep it apart from the system prompt, check output formats with ordinary code, and, most importantly, reduce what an agent can do while it is handling untrusted material. The NCSC’s advice is blunt: if a system processes content from an untrusted party, its privileges should drop to that party’s level.

Where it goes wrong

  • Believing it can be fixed. The NCSC argues that prompt injection is unlike SQL injection, which parameterised queries solve, because language models have no built-in separation of instructions from data. Treat it as a residual risk to manage, and be wary of any product claiming to stop it outright.
  • One powerful service account. Connecting the agent to email, files and finance with one admin-level identity means any successful injection inherits all of it.
  • Rules in the prompt only. Instructions to the model are useful for behaviour, but they are not a security boundary.
  • Approval fatigue. If everything needs approval, people click yes without reading. Approvals should be reserved for actions that matter, and show clearly what will happen.
  • Leaks through side doors. An agent that can read private data and also fetch any URL can be tricked into sending that data out, for example inside the address it is told to fetch. Limiting outbound destinations closes this.
  • No monitoring. Repeated failed tool calls can mean someone is probing. The NCSC recommends logging inputs, outputs and tool calls and watching for exactly this.

How Mai AI uses it

In a Mai AI system the model is never the security boundary. I design each agent with an allow-list of narrow tools, permissions scoped to its area, checks enforced in the tool layer and the systems behind it, and a person approving anything consequential. Content arriving from outside the business, such as inbound email or uploaded files, is handled as untrusted, and an agent working on it does so with reduced access. Every tool call is logged so unusual patterns can be spotted, as described in traces and audit logs.

The AI chief of staff built for a UK property group shows the approach in practice: every action an agent takes is checked against what it is allowed to do, anything that matters needs a signed approval token, the trail goes to an audit log, and the whole system runs inside the client’s own Azure tenant. Safety cases I find along the way become tests, as described in testing agents.

If you want to see how these controls would apply to your tools and data, the custom AI operations systems page is the place to start.

Questions and answers

How do you stop prompt injection in AI agents?

You cannot fully stop it today, so you limit the damage. Keep untrusted content separate from instructions, give the agent only the permissions the task needs, enforce those permissions in code outside the model, and require human approval for anything consequential.

What is indirect prompt injection?

It is when instructions hidden in content the agent reads, such as an email, a document or a web page, change what the agent does. The attacker never talks to the agent directly; they plant text where the agent will find it.

What permissions should an AI agent have?

The minimum for its job. Prefer narrow, purpose-built tools over general ones like a shell or open web access, use read-only access where reading is enough, and act under the identity of the person it works for rather than a powerful shared account.

Do AI agents need a sandbox?

Any agent that runs code or shell commands should, so that it can only write to permitted folders and reach approved network addresses. Agents that only call a fixed set of well-defined tools need tight tool permissions more than a sandbox.

Sources

  1. OWASP GenAI Security Project, LLM01:2025 Prompt injectiongenai.owasp.org
  2. OWASP GenAI Security Project, LLM06:2025 Excessive agencygenai.owasp.org
  3. NCSC, Prompt injection is not SQL injection (it may be worse)ncsc.gov.uk
  4. Greshake et al., Compromising real-world LLM-integrated applications with indirect prompt injection (arXiv)arxiv.org
  5. Claude Code documentation, Sandboxingcode.claude.com