Designing bounded-action supervisors for internal AI agents

An agent that can do everything is an agent you cannot reason about. The interesting engineering problem is not capability — it is the boundary: what the agent is allowed to touch, who enforces that, and what record survives the run.

The failure mode is a familiar one

Every access-control bug I have written up has the same shape: the rule lived in the client. The SplashIn assessment (CVE-2025-45156 and CVE-2025-45157) is the clearest example — a free-tier app politely waited ten minutes between location updates, and the backend simply did not care what interval the caller chose. The enforcement was a property of the program asking, not of the system answering.

Agent supervisors fail the same way when the boundary is a prompt. "Only read the staging database" is a sentence, not a control. A model that has been given a credential will use it in whatever way the current context suggests, and the context is attacker-influenced: a log line, a fetched page, a filename, a tool result. Prompt-level restrictions are advisory text inside the same channel as the attacker's text.

So the boundary has to live where the side effects happen.

Capability-scoped tools, not a shell

The useful unit of authority is a tool with a narrow verb. Compare two designs for an internal agent that triages alerts:

# Design A — one tool, maximum reach
run_shell(command: string)

# Design B — several tools, each with one verb and a schema
list_open_alerts(service: string, since: string) -> Alert[]
read_alert(alert_id: string) -> Alert
annotate_alert(alert_id: string, note: string) -> Ack
open_ticket(alert_id: string, queue: enum) -> TicketRef

Design B is not merely tidier. Each tool is a place to enforce something:

• Input shape. since is parsed as a timestamp or rejected. There is no path for ; rm -rf.
• Scope. service is checked against the set of services this agent is responsible for.
• Verb separation. Reading and annotating are different authorities. An agent that only needs to triage does not get the ticket queue.
• Observability. Every call is a typed event with a name, not an opaque command string.

The same reasoning shows up in a portfolio MCP server: the read-only surface exposes one tool that takes no arguments and returns public data. Nothing to validate, nothing to escalate, no write path to protect. Constraints are cheapest when the interface makes the dangerous operation unrepresentable.

Enforce in the server, not in the caller

Whatever the agent's instructions say, the process that holds the credential must re-derive the decision. Three rules that hold up:

1. Derive authority from the request, not from the prompt. If the tool needs to know which tenant it is acting for, that value comes from an authenticated session, never from a model-supplied field.
2. Deny by default and name the allow-list. An agent that gets a new integration should require a deliberate edit somewhere a human reviews, not a new sentence in a system prompt.
3. Make the boundary observable. A rejected action is an event worth recording. Silent denials teach nobody anything; the useful artefact is the record of what was asked and why it was refused.

This is the same ordering that fixes the class of bug in the SplashIn report: verify identity, read the entitlement, check the rate or interval policy, then touch the data. The query does not change. The policy in front of it does, and the order is the security property.

Provenance is not logging

Logs tell you what happened. Provenance tells you why the current state is what it is, and it has to survive the run that produced it.

A shape that works for internal agents:

node      one decision, one action, one state transition
edge      causal link: produced_by, derived_from, supersedes
actor     human, agent run id, or service identity
input     the exact payload that was read (digest, not a summary)
output    the exact payload that was written (digest)
verdict   allowed / refused / deferred, with the rule that decided

The two properties worth paying for:

• Immutable revisions. A state transition appends; it never rewrites. When an agent and a human disagree about what happened, the graph settles it.
• Digests over summaries. A summary is a claim about data; a digest is a claim about bytes. Summaries are what the model reads, digests are what the audit compares.

Graph state in SQLite is a good fit for this: a single file, transactional writes, queryable causality, and it is trivially copyable for review. The important part is not the storage engine — it is that provenance is produced by the same code path that performs the action. If the audit trail is assembled by a second system, it will eventually describe a world that never existed.

Retries are part of the design

Agents retry. Networks fail, providers time out, supervisors restart. Any write an agent can perform needs an idempotency key: a client-supplied identifier the server records, so a repeated request returns the original result instead of performing the action twice.

This is not exotic — it is the same discipline that keeps a contact form from sending the same message twice, or a billing call from charging twice. The failure mode it prevents is quiet: a duplicated record looks exactly like a legitimate one.

Keep the model in the loop, out of the keys

The point of an agent is judgment at the edges: reading ambiguous input, choosing the next query, writing the summary a human will read. The point of the supervisor is that judgment never becomes authority.

Concretely, the split I aim for:

• The model chooses which tool to call and what to put in the schema-validated fields.
• The server decides whether the call is permitted, what it may read, and what gets recorded.
• The human decides anything irreversible, or anything that leaves the organisation.

An agent operating under that split is not less capable. It is the only kind that can be left running.

What to build first

If you are adding a supervisor to an existing system, in this order:

1. Write down the boundary as data — a table of tool, verb, scope, and who may call it.
2. Move enforcement into the server that holds the credential, in front of the existing code.
3. Give every write an idempotency key before the first agent uses it.
4. Record provenance in the same transaction as the action.
5. Only then widen what the agent is allowed to attempt.

Steps 2 and 4 are the ones that get skipped, and they are the two that decide whether an incident is a paragraph in a postmortem or a week of archaeology.