Skip to content

Pattern

How routing and specialist AI agents work

Routing means one agent reads each request and hands it to the specialist best placed to handle it, such as finance, operations or sales. Each specialist has its own instructions, tools and permissions, which keeps answers sharper and access tighter than one agent trying to do everything.

StudiedLast reviewed 4 min read

Explained from public sources. Mai has not built this yet.

On this page

In plain English

Think of a good office manager. They do not answer every question themselves; they know who in the building handles invoices, who handles maintenance and who handles new clients, and they pass each request to the right person with a short note. A routed agent system works the same way. One agent, the router, reads what has come in and decides who should deal with it. Specialist agents then do the work in their own area, using only the tools and information that area needs.

How it works

The router. The router’s job is classification, not problem solving. It reads the request (an email, a Teams message, a form) along with a little context, such as who sent it, and picks a destination. It can be a language model, simple rules, or both: rules catch the obvious cases cheaply and the model handles the ambiguous ones. Anthropic’s guide to building agents describes routing as classifying an input and sending it to a specialised follow-up, so that each category can have its own prompt and tools, and notes that the same idea can send easy requests to smaller models and hard ones to more capable models.

The specialists. Each specialist agent has:

  • its own instructions, written for its area (how this business handles arrears, say, rather than generic advice);
  • its own tools, limited to the systems its area uses;
  • its own permissions, so the sales agent cannot read payroll;
  • its own evaluation cases, so it can be tested on its own.

The handoff. When the router passes work on, it sends a structured note: what was asked, who asked, why this specialist, and any facts already gathered. In the OpenAI Agents SDK, for example, each possible handoff appears to the model as a tool named after its destination, and the developer can filter how much of the earlier conversation the receiving agent sees. Deciding what to pass is a design choice: the full history is simplest, a filtered summary is cheaper and leaks less.

In: Email, Teams, Forms. Then Router. Then Finance, Operations, Sales, People. Then Own tools and data. Then You approve.

One router hands each request to the right specialist

Some requests span areas: “can we afford to take on this contract?” needs finance and operations. The router then either sends it to a coordinating agent that consults both, or asks each specialist for its part and combines the answers. That is where routing meets sub-agents and parallel work.

Routing also helps security. OWASP’s guidance on excessive agency recommends giving each part of a system only the functions and permissions it needs. Splitting work by area is a natural way to do that: the blast radius of any one agent is its own area.

Where it goes wrong

  • Misrouting. The router sends a complaint to sales or a supplier invoice to the people team. Every route needs a fallback (“not sure, ask a person”) and a log of routing decisions, so mistakes are visible and can become test cases.
  • Too many specialists. Ten agents with overlapping remits confuse the router and multiply maintenance. Start with a few areas that match how the business is actually organised.
  • Lost context. If the handoff note is thin, the specialist asks the customer the same questions again. If it is too full, it carries data the specialist should not see.
  • Ping-pong. Specialists handing work back and forth without anyone owning it. One agent must own each task until it is done or escalated.
  • Hidden cost. Every hop is another model call. Cheap routing (rules first, small model second) keeps this in check; see agent cost control.

How Mai AI uses it

Routing is the backbone of most systems I design. The AI chief of staff is the clearest example: a chief of staff takes each request and hands it to one of six director agents, each responsible for its own area of the business. Every action is checked against what that agent is allowed to do, and anything that matters needs a signed approval token.

In a Mai AI system, I draw the specialists from the client’s real organisation chart rather than from a template, so each agent maps to a person or team who can own its instructions and review its work. Each specialist reaches business systems only through the tool layer, with permissions scoped to its area, and every routing decision is logged.

I also keep the router modest. It classifies, adds context and hands over; it does not take actions of its own. If you want to see what the specialists would be in your business, you can sketch it in the designer for a custom AI operations system.

Questions and answers

What is a router agent?

A router agent reads an incoming request, decides what kind of work it is, and passes it to the agent or process that handles that kind of work. It usually does little of the work itself.

Why use multiple specialist AI agents instead of one?

Each specialist gets focused instructions and only the tools and data its area needs. That makes behaviour easier to test and limits what any single agent can reach if something goes wrong.

What is an agent handoff?

A handoff is when one agent passes control of a conversation or task to another. In frameworks such as the OpenAI Agents SDK, the model sees each possible handoff as a tool it can call.

Can routing send simple questions to cheaper AI models?

Yes. Routing can choose a model as well as an agent, sending easy requests to a smaller, cheaper model and harder ones to a more capable one.

Sources

  1. Building effective agents (Anthropic engineering)anthropic.com
  2. Handoffs (OpenAI Agents SDK documentation)openai.github.io
  3. LLM06:2025 Excessive Agency (OWASP GenAI Security Project)genai.owasp.org