Agentic Security Starts With the Blast Radius

TL;DR

• Prompt injection becomes dangerous when untrusted content reaches a model that can access sensitive data or take consequential action.
• Model instructions and content filters can reduce some attacks, but they are not security boundaries. Constrain data flow, tools, credentials, execution, and irreversible actions outside the model.
• Design as though the model can be manipulated. Limit the blast radius, observe every action, and rehearse containment and recovery.

---

An agent can interpret text, retrieve data, call tools, write code, and act across systems. That flexibility is the product value and the security problem.

Language models process instructions and data through the same medium. A document, email, webpage, tool result, or memory entry can contain text that the model interprets as a new instruction. When the agent also has authority, an attacker may redirect the system towards data disclosure or an unintended action.

Do not make model obedience the security boundary. Build a system that limits what a manipulated model can reach and do.

Threat-model the complete job

Start with the workflow, not a generic list of model risks.

Map five surfaces:

Risk grows when untrusted input, sensitive context, and action authority meet inside one model loop.

The AI governance framework classifies the broader use case. This playbook turns the authority boundary into a security design.

Distinguish the attack paths

Three related problems require different tests.

Jailbreaking attempts to bypass a model's behavioural restrictions. The attacker interacts directly with the model and tries to elicit prohibited output.

Direct prompt injection places malicious instructions in the user's input to an application. The attack tries to override the developer's intended task or policy.

Indirect prompt injection hides instructions in content the agent retrieves or observes, such as a webpage, email, document, code comment, image, or tool result. The user may not know the instruction entered the context.

Indirect injection is particularly important for agents because the model may read untrusted content while holding tools or private context. The attacker does not need direct access to the agent session.

Assume the model can be manipulated

Prompt wording, delimiters, safety instructions, input filters, output filters, and model-based classifiers may increase attack cost or catch known patterns. They should remain defence layers, not the final control.

The UK's National Cyber Security Centre advises secure system design rather than treating prompt injection like a conventional bug that can be patched once. OWASP's prompt-injection guidance similarly places filters beside least privilege, validation, monitoring, and approval for destructive actions.

Use a hostile-model assumption for architecture reviews:

If the model chose the worst valid tool call available to it, what could happen?

The answer reveals the blast radius more reliably than another instruction telling the model to be careful.

Grant authority per task

An agent identity with permanent access to every tool creates unnecessary authority.

Derive permissions from the user's current request and the workflow state. A summarisation task may need read access to selected records and no ability to send, edit, or delete. A draft-email task may need a recipient and content editor but no inbox history. A payment investigation may need read access while a refund requires a separate bounded action.

Use these controls:

• Issue short-lived, task-scoped credentials where the platform supports them
• Prefer read-only tools and accounts by default
• Restrict objects, tenants, fields, destinations, amounts, and time windows
• Validate tool arguments against the authenticated user and current task
• Separate data retrieval from consequential action
• Require a new authorisation when scope expands
• Prevent one agent from granting itself or another agent more authority

Research systems such as CaMeL explore explicit control and data flow with capability-based enforcement. The useful product principle is durable even when the implementation differs: untrusted content should not be able to expand program control or data authority.

Separate reading from acting

An agent that reads untrusted content and can immediately send, modify, or execute creates a direct path from injection to consequence.

Break the path where the job permits it:

1. A constrained component reads or extracts information from untrusted content.
2. The system converts the result into a typed, validated structure.
3. A separate policy layer compares the proposed action with the original user intent and permissions.
4. A privileged component executes only an approved action.

Do not pass an untrusted free-text summary into a privileged model and assume the risk disappeared. Preserve provenance and validate the fields that drive action.

Some jobs genuinely combine reading and writing, such as reviewing an inbox and forwarding selected requests. These need tighter scopes, stronger action checks, destination controls, sampling, and often human approval. Architecture cannot make every broad agent safe without reducing what it can do.

Sandbox generated execution

Generated code, commands, queries, and files should run in an environment designed for failure.

Constrain:

• Filesystem and network access
• Available secrets and environment variables
• Process, memory, runtime, and spend
• Package installation and executable formats
• Internal service and cloud metadata access
• Output types and paths
• Persistence after the job ends

Treat the sandbox output as untrusted until deterministic checks and policy validation pass. Do not execute generated code on the same server, account, or network plane that holds the product's production authority.

Keep consequential decisions outside the model

Use deterministic controls wherever the rule can be expressed directly.

Examples include:

• Schema and type validation
• Customer and tenant authorisation
• Transaction, rate, and destination limits
• Allowlists for tools and domains
• Required approvals
• Separation of duties
• Idempotency and duplicate prevention
• Policy rules for prohibited actions

The model can propose. The system decides whether the proposal is valid under the current identity, task, state, and policy.

Human approval is appropriate when judgement is required and the reviewer can see the relevant evidence. Test whether reviewers detect seeded failures, understand what will happen, and can stop or reverse the action. A confirmation button with no useful context is not a security control.

Secure context, memory, and tools

Agent security extends beyond the immediate prompt.

Context can be poisoned. Retrieved documents, shared instructions, generated summaries, and long-term memory may preserve malicious or stale content. Record source, trust level, owner, and expiry. Keep untrusted content visibly distinct.

Tools can mislead the model. Tool descriptions, schemas, dependencies, and remote servers become part of the control surface. Approve tool sources, pin versions where appropriate, verify changes, and prevent overlapping tools from silently shadowing trusted behaviour.

Credentials outlive experiments. Remove tokens, schedules, webhooks, memory, and integrations when an agent is retired. An unused agent with valid authority remains an attack path.

The agent ownership contract assigns these maintenance responsibilities.

Test adaptively

A static set of attack strings proves only that the system handles those strings under that configuration.

Build adversarial tests around the threat model:

• Direct attempts to override the task
• Malicious instructions inside each untrusted source
• Encoded, fragmented, multilingual, and multimodal inputs
• Attempts to combine individually permitted actions into a prohibited outcome
• Cross-tenant and cross-user data access
• Tool argument manipulation and privilege escalation
• Memory poisoning and persistence across sessions
• Exfiltration through links, messages, files, images, or tool parameters
• Failure of approval, timeout, rollback, and shutdown paths

Use adaptive human testing for consequential systems. The tester should observe defences and change the attack, rather than replaying a frozen benchmark.

Add successful attacks and near misses to the regression suite, but keep searching for new paths. The agentic safety inspection covers trace grading, circuit breakers, budgets, and drift drills.

Observe actions and rehearse containment

Log enough to reconstruct the job:

• Authenticated user and initiating request
• Context and source provenance
• Model, configuration, and relevant policy version
• Proposed and executed tool calls
• Credentials and permission scope used
• Approval, denial, override, and escalation
• External side effects
• Final outcome and recovery action

Protect the logs as sensitive data. A trace can contain prompts, customer records, secrets, and generated content.

Define incident controls before release:

• Pause new jobs
• Revoke credentials and tool access
• Stop in-flight execution
• Isolate affected tenants or data sources
• Reverse or compensate actions where possible
• Preserve evidence
• Notify accountable security, product, legal, operational, and customer owners
• Add the failure to evaluation and governance records

Run drills. A shutdown path that has never been exercised is an assumption.

The agent security review

Before expanding tools, data, volume, or autonomy, ask:

1. Which untrusted content can enter the context?
2. What sensitive data and actions are reachable from that context?
3. Can permissions be narrowed for this task?
4. Which controls operate outside the model?
5. Where are reading, deciding, and acting separated?
6. Can generated execution escape its sandbox?
7. Have adaptive attacks tested the actual production path?
8. Can the team contain, investigate, and recover from misuse?

NIST's Generative AI Profile and the NCSC secure AI development guidance provide broader risk and lifecycle references. Apply them with current security specialists and the organisation's existing security programme.

Anti-pattern: the safety prompt

The agent has broad credentials and can read external content. The team adds a long system prompt, an input filter, and a second model that labels suspicious responses. A red-team script passes, so the feature ships.

One indirect instruction reaches the agent through a trusted-looking document. The model makes a valid tool call with valid credentials. Every component behaves as designed. The design was unsafe.

Agentic security starts by assuming manipulation is possible and making the resulting action small, observable, reversible, and containable.