BlogSecurity
Containing prompt injection when an agent reads outside content
Assume that an instruction hidden in an email, document or tool result will eventually be followed, and design so that following it cannot reach anything that matters. That means mapping which untrusted inputs can reach which actions, removing the paths you do not need, and enforcing the remaining limits in code the model cannot talk its way around. Filters and system prompts lower the odds of an injection working; they do not set a boundary.
Plan for an injection that works
OWASP's 2025 entry on prompt injection describes indirect injection as content from external sources, such as websites or files, that changes the model's behaviour, and notes that the instruction need not be visible to a person. It also states that it is unclear whether fool-proof prevention exists. Its mitigations include least privilege, human approval for high-risk actions, separating external content and adversarial testing. Our planning assumption follows from that: the useful question is what the agent can do once it has been fooled.
Our coding agent security guide covers agents working in repositories. This article is about business agents that read mail, documents, tickets, web pages and tool results.
Map untrusted inputs against actions
Take a hypothetical accounts payable agent. It reads supplier emails with PDF invoices, looks up purchase orders, and as first proposed could create payment proposals, reply to suppliers and update supplier bank details. Its map looks like this:
- Email bodies and attachments: untrusted, written by anyone who can send mail to the inbox
- Purchase orders: trusted for amounts, but free-text fields may hold text the supplier provided
- Supplier replies: an outbound channel, so anything in the agent's context can leave through it
- Bank detail changes: a high-impact write that an injected message about a changed account would aim for
The dangerous combination is plain to see: untrusted text in context, a write that moves money and a way to send data out. The decision is which edge to cut. Here we would remove bank detail changes from the agent entirely, restrict replies to fixed templates with no free text, and let the agent only propose payments that a person releases in the ERP system.
Design patterns that remove the path
A 2025 paper by Beurer-Kellner and colleagues sets out design patterns built on one principle: once an agent has read untrusted input, it must be constrained so that this input cannot trigger consequential actions. Four of the patterns that apply to business workflows:
- Action selector: the model picks one of a fixed set of actions and never sees their results, so no tool output can steer it
- Plan then execute: the model fixes its tool calls before reading untrusted data, so injected text can change arguments but cannot add calls
- Map-reduce: each untrusted document is processed in isolation, so one malicious file cannot change how the others are handled
- Dual LLM: a privileged model with tools never reads untrusted text, while a quarantined model without tools reads it and returns values by reference
The CaMeL system, described by Debenedetti and colleagues, extracts the control and data flow from the trusted request and enforces capability policies when tools are called. Its guarantees hold within the authors' threat model, which treats the user's request as trusted, and depend on the security policies defined for each tool. They report solving 77% of AgentDojo benchmark tasks with provable security, against 84% for an undefended system. That gap is a measured utility cost on their benchmark, not a forecast for your workflow.
Controls the model cannot argue with
- Tool permissions scoped to the task and enforced by the server or business system
- Approvals bound to the exact action and payload and verified by trusted code, for anything that moves money, changes access or sends data out
- Egress limits on the agent's runtime, and output rendering that does not load images or links automatically, since OWASP describes exfiltration through an inserted image URL
- Deterministic checks of outputs against expected values, such as amounts matching the purchase order and recipients on an allowlist
- Tool results treated as untrusted input, which the MCP tools specification reflects when it asks clients to validate results before passing them to the model
Marking external content clearly in the prompt is still worth doing. It reduces how often an injection works, but the model can still be talked past it.
Checks before and after go-live
Build an adversarial test set from your own input channels: white text in PDFs, instructions in email signatures, text in other languages or encodings, and instructions inside tool results. For each case, assert the outcome at the boundary, not the model's wording: no unapproved write, nothing outbound beyond the templates, an alert raised. Re-run the set whenever the model, prompt or tools change. After launch, log each tool call with the IDs and sources of the inputs read before it and the decision taken. Capture raw content only through an approved, restricted and redacted capture with its own retention. Review refused actions, and treat a blocked attempt as a sign that someone is probing.
What the boundaries cost
Each pattern removes flexibility. An action selector cannot handle a request outside its list, plan then execute cannot adapt to what it finds, and approvals add a person's time to every case they cover. Match the strength of the boundary to the worst action the agent can reach. An agent that summarises documents for the person who uploaded them may need light controls. One that can pay, send or grant access should start with those paths cut, with capabilities added back one at a time after testing.
A practical next step: draw the input and action map for one agent and mark every path from untrusted text to a write or an outbound channel.
Questions
Will a better model or an injection classifier solve this?
They can lower the rate at which injections succeed. OWASP notes it is unclear whether fool-proof prevention exists, so treat them as one layer and keep the boundaries that do not depend on the model.
Do we need to treat our own documents as untrusted?
Treat anything that others can edit as untrusted, including shared drives, wiki pages, CRM notes and ticket comments. Trust follows who can write the content, not where it is stored.
Does following these steps make the agent safe?
No set of controls makes an agent immune. These steps shrink what a successful injection can reach and make attempts visible. The remaining risk should be stated and accepted by the owner of the process the agent works in.
Related services
AI engineering and architecture
Assistants, voice interfaces and agents in the products you sell, with the serving and retrieval underneath.
MCP server development
MCP servers that connect approved internal APIs to AI assistants, with narrow permissions, approval for changes and tests.
AI agents and process automation
Agents that take on the routine steps of complex cases and pass decisions to people, with a cost per handled case.
More from the blog
Coding agent security
Practical security and governance for coding agents: prompt injection, secrets, permissions, sandboxing, unattended runs and ownership.
MCP server permissions
How to connect an assistant to internal systems without giving every MCP tool broad access: scopes, checks inside each tool and separate downstream identities.
AI trace data retention
What to log to debug agents without keeping every sensitive prompt: three tiers of trace data, content capture off by default, the limits of redaction and retention set by your policy owners.
Talk to an engineer
Bring one piece of work you would like to improve. The call takes 30 minutes and needs no access to your systems or your data.
Booking runs on Google Calendar.
Prefer to write first? Leave your email and a few lines, and we reply with times.
Look for the draft in your email program
Send it from there, and an engineer replies in person with times for a 30-minute call. If no draft has opened, write to us at miki@cloudsail.com.