Pattern
How to make AI agents safe, from permissions to prompt injection
You make an AI agent safe by limiting what it can do, not by trusting what it says. Give it the narrowest tools and permissions, run risky work in a sandbox, require a person to approve consequential actions, and treat every email, document and web page it reads as possibly hostile.
Explained from public sources. Mai has not built this yet.
On this page
In plain English
An AI agent can be talked into things. It reads emails, documents and web pages, and it cannot always tell the difference between information and instructions, so a cleverly worded message can steer it. The answer is not to hope it resists, but to build the system so that even a confused agent cannot do much harm: it only has the keys it needs, it works inside a fenced-off area, and anything that matters waits for a person to say yes.
How it works
Safety comes from layers outside the model, each of which still holds if the model gets it wrong.
Input: Your request, Emails and files. Then Agent. Then Controls: Allow-listed tools, Scoped permissions, Sandbox. Then You approve. Then Audit log.
Least privilege. OWASP’s LLM Top 10 lists “excessive agency” as a core risk and breaks it into three causes: too many tools, too many permissions, and too much autonomy. The remedies follow directly. Offer only the tools the job needs, and prefer narrow ones (“send a reply to this thread”) over open-ended ones (“run any shell command”, “fetch any URL”). Use read-only scopes where reading is enough. Where possible, act under the identity of the person the agent is working for, so it cannot reach anything they could not.
Enforcement outside the model. The permission check belongs in the tool layer and the downstream system, not in the prompt. OWASP calls this complete mediation: the system that holds the data decides whether the request is allowed, whatever the model asked for. An instruction in the system prompt saying “never delete records” is a hint; a database role without delete rights is a control.
Allow-lists. Tools, recipients, domains and file locations work best as explicit lists of what is allowed, with everything else refused. A list of banned phrases, by contrast, is easy to get around, and the NCSC specifically warns against relying on deny-lists of known attack text.
Sandboxing. Where an agent runs code or shell commands, the operating system should enforce the boundary. Claude Code’s sandbox is a good public example: commands can write only inside the working folders, and network traffic goes through a proxy that allows only listed domains. The point is that the limit holds even if the agent is persuaded to try something else.
Approvals. Consequential actions, such as payments, external emails and deletions, wait for a person. See human approval gates for how to choose which actions need one.
Handling untrusted content. Indirect prompt injection is the agent-specific threat. Researchers showed in 2023 that instructions planted in content an application later retrieves can steer it remotely, without the attacker ever using the application. Practical defences: mark external content clearly as data rather than instructions, keep it apart from the system prompt, check output formats with ordinary code, and, most importantly, reduce what an agent can do while it is handling untrusted material. The NCSC’s advice is blunt: if a system processes content from an untrusted party, its privileges should drop to that party’s level.
Where it goes wrong
- Believing it can be fixed. The NCSC argues that prompt injection is unlike SQL injection, which parameterised queries solve, because language models have no built-in separation of instructions from data. Treat it as a residual risk to manage, and be wary of any product claiming to stop it outright.
- One powerful service account. Connecting the agent to email, files and finance with one admin-level identity means any successful injection inherits all of it.
- Rules in the prompt only. Instructions to the model are useful for behaviour, but they are not a security boundary.
- Approval fatigue. If everything needs approval, people click yes without reading. Approvals should be reserved for actions that matter, and show clearly what will happen.
- Leaks through side doors. An agent that can read private data and also fetch any URL can be tricked into sending that data out, for example inside the address it is told to fetch. Limiting outbound destinations closes this.
- No monitoring. Repeated failed tool calls can mean someone is probing. The NCSC recommends logging inputs, outputs and tool calls and watching for exactly this.
How Mai AI uses it
In a Mai AI system the model is never the security boundary. I design each agent with an allow-list of narrow tools, permissions scoped to its area, checks enforced in the tool layer and the systems behind it, and a person approving anything consequential. Content arriving from outside the business, such as inbound email or uploaded files, is handled as untrusted, and an agent working on it does so with reduced access. Every tool call is logged so unusual patterns can be spotted, as described in traces and audit logs.
The AI chief of staff built for a UK property group shows the approach in practice: every action an agent takes is checked against what it is allowed to do, anything that matters needs a signed approval token, the trail goes to an audit log, and the whole system runs inside the client’s own Azure tenant. Safety cases I find along the way become tests, as described in testing agents.
If you want to see how these controls would apply to your tools and data, the custom AI operations systems page is the place to start.
Questions and answers
How do you stop prompt injection in AI agents?
You cannot fully stop it today, so you limit the damage. Keep untrusted content separate from instructions, give the agent only the permissions the task needs, enforce those permissions in code outside the model, and require human approval for anything consequential.
What is indirect prompt injection?
It is when instructions hidden in content the agent reads, such as an email, a document or a web page, change what the agent does. The attacker never talks to the agent directly; they plant text where the agent will find it.
What permissions should an AI agent have?
The minimum for its job. Prefer narrow, purpose-built tools over general ones like a shell or open web access, use read-only access where reading is enough, and act under the identity of the person it works for rather than a powerful shared account.
Do AI agents need a sandbox?
Any agent that runs code or shell commands should, so that it can only write to permitted folders and reach approved network addresses. Agents that only call a fixed set of well-defined tools need tight tool permissions more than a sandbox.
Harnesses known for this
- Harness profileClaude Code and Agent SDK
Claude Code is Anthropic's agentic coding tool that reads a codebase, edits files and runs commands from the terminal, IDE, desktop app or browser. The Claude Agent SDK packages the same agent loop and tools as a Python and TypeScript library, for building your own agents on Claude.
- Harness profileOpenClaw
OpenClaw is an open-source, self-hosted AI assistant that runs on your own machine and answers you in the chat apps you already use, such as WhatsApp, Slack, Teams and Telegram. It is best for one person or a small trusted team who want an always-on assistant with scheduled jobs, skills and tools under their own control.
- Harness profileHermes Agent
Hermes Agent is an open-source AI agent from Nous Research that runs on a server or your own machine, talks to you through the terminal or chat apps, and writes reusable skills from the work it does. It suits people who want a long-lived agent that remembers, runs scheduled jobs and improves with use, on any model provider.
- Harness profileOpenAI Agents SDK
The OpenAI Agents SDK is an open-source library, in Python and TypeScript, for building agents that use tools, hand work to each other and pause for approval. It suits teams that want a light agent loop with tracing built in, rather than a full orchestration platform.
Related entries
- PatternHuman approval gates
Human approval gates sort every action an AI agent can take into three levels: draft only, needs a person to approve, or allowed to run automatically. The agent prepares, a person signs off anything that matters, and the system refuses any action that does not match what was approved.
- PatternThe tool layer and MCP
AI agents reach real systems through tools: named functions such as "find invoice" that the model asks to run, while ordinary code does the actual work. MCP, the Model Context Protocol, is an open standard for offering those tools and data to any AI application in the same way.
- PatternTraces and audit logs
You see what an AI agent did and why by recording a trace of every run, each model call, tool call and decision as a timed step, plus a separate audit log of every action that changed something and who approved it. Traces are for debugging; the audit log is the permanent record.
- PatternTesting agents
You test AI agents with evals, a set of realistic tasks with clear pass or fail checks that you run every time the agent, its instructions or its model changes. The best tasks come from real failures, and each one runs several times because agents do not behave the same way twice.
Sources
- OWASP GenAI Security Project, LLM01:2025 Prompt injectiongenai.owasp.org
- OWASP GenAI Security Project, LLM06:2025 Excessive agencygenai.owasp.org
- NCSC, Prompt injection is not SQL injection (it may be worse)ncsc.gov.uk
- Greshake et al., Compromising real-world LLM-integrated applications with indirect prompt injection (arXiv)arxiv.org
- Claude Code documentation, Sandboxingcode.claude.com