Building Reliable Agentic AI Systems

The false permission, and why it beats the wrong answer

Picture an agent tasked from a document: a work order, a ticket, an email thread. It reads the source, extracts what it thinks it was asked to do, and acts. Now let the reading be slightly wrong. A value parsed as the wrong type. A scope read broader than written. A validity window recorded as starting earlier than it did. A candidate action that was never confirmed, slipping through as if it had been.

Nothing in any of those moments looks like a failure. There is no exception to catch. The agent now acts with authority it was never granted, and every downstream step inherits the error with compounding confidence. A wrong answer is a bad output; a false permission is a wrong question, executed. That asymmetry drives every design decision that follows.

Capability is not authority

The distinction the whole discipline rests on: capability is what the agent’s tools can do; authority is what the agent is allowed to do, for this purpose, on these objects, under these conditions, now. An agent with database credentials has the capability to delete records. Whether it has the authority to delete this record, in service of this instruction, today, is a different fact, held in a different place.

Most agent frameworks conflate the two by construction: whatever tools are wired in are, in practice, permitted, and the only thing standing between capability and action is the model’s judgment about what the task requires. That is the false permission waiting to happen, because tool availability silently becomes permission. The design rule is blunt: the tool roster defines what is possible; authority must be granted separately, per mandate, and checked at the moment of action. A capable agent with narrow authority is a useful system. A capable agent whose authority is implicit is an incident report with a timestamp not yet filled in.

The authority lifecycle

Authority is not a configuration flag; it has a lifecycle, and each stage is a place where engineering either holds or leaks. The shape I build toward:

source instruction        the document or request, kept verbatim
  -> interpreted mandate  structured proposal: intended actions, each tied
                          to the span of source that produced it, each
                          marked confident / ambiguous / unconfirmed
  -> reviewed scope       a person confirms, corrects, rejects; ambiguous
                          items don't pass silently
  -> authority boundary   identity (on whose behalf), objects, allowed
                          actions, purpose, validity window; frozen at
                          approval
  -> execution            every action checked against the boundary at
                          runtime, not at planning time
  -> durable record       proposal, review decisions, boundary, effects;
                          attributed and timestamped
  -> expiry / revocation  authority ends by time or explicit withdrawal,
                          and ending is enforced, not remembered

Three stages deserve their footnotes. The interpreted mandate is a structured proposal, not prose, because prose has to be re-interpreted downstream and a second reading can quietly differ from the first. The machine proposes; it never certifies that its own reading was good enough. Automated cross-checks near the review are advisory (a second model may flag “this looks wrong” to the reviewer; it may not silently block or approve), a constraint I’ve written about more generally in Designing Evidence Gates.

The boundary freezes at approval: what the reviewer confirmed becomes read-only, and what executes is what was reviewed, not a later edit shifting under the signature. Delegation, where it exists at all, only narrows: an agent may hand a sub-task to another agent with a subset of its boundary, never a broadened one, and the delegation is itself recorded.

And expiry is enforced at action time, because it is the one limit that fails silently. Nobody does anything wrong on the day a window closes; unless the runtime actively refuses, attention simply continues. The same applies to revocation: permission that cannot be withdrawn after issuance is not permission, it is a launch. Stale authority (a mandate reviewed against last month’s state of the world, still executing today) belongs on the same list, and long-running mandates deserve re-confirmation points for exactly that reason.

Ceremony scales with reversibility

None of this means every action gets a confirmation dialog. Demand a confirmation for everything and people stop reading the confirmations; the ceremony survives while the safety dies. The dividing line is reversibility, and the approval budget is spent accordingly.

Reading, drafting, staging, computing inside the boundary: reversible, so the agent moves freely and the record absorbs the risk. Sending, deleting, deploying, paying, publishing: one-way doors, where explicit approval is cheap against the cost of error. Between the two sit distinctions worth engineering explicitly: read versus write, propose versus execute, a bounded batch (“these forty records”) versus an open-ended mandate (“keep them tidy”), current authority versus authority granted against a world that has since changed. Each step toward the irreversible, the open-ended, or the stale is a step that deserves a stronger check.

For the one-way crossings, three properties are worth copying from any well-run change process: only eligible actions cross (reviewed, in scope, in window); borderline crossings carry a recorded reason a later reviewer can read; and the step is treated as genuinely one-way, so the weight sits on the review before it, not on the hope of undoing it after. My own development toolchain applies this in miniature: agents explore, edit, and verify at speed, and the path to anything irreversible runs through a person, by construction rather than convention.

The record is the product

Every serious question about an agentic system arrives after the fact. Why did it do that? Who approved this? What did it believe the source said? Was it still authorised when it acted? A system that cannot answer those questions from its own records is indefensible even when it behaved correctly, and in regulated settings, indefensible and wrong converge on the same outcome.

So the trail is a first-class output, not a debug artifact: what the source said, what was proposed, what the reviewer changed and confirmed, what boundary resulted, which version of that boundary was in force at each action, what executed inside it, when authority ended. A decision trace of what the system did. Asking the model to explain itself afterwards does not substitute, because a language model’s account of its own past output is a fresh generation, not a memory. The test worth holding the design to: any action the agent took can be reconstructed and challenged by someone who wasn’t there, from records alone.

The one-sentence version: give an agent authority that is explicit, narrow, time-bounded, enforced at action time, and recorded, and reliability follows from the structure. Leave authority implicit in the tool roster, and no amount of model quality will save you, because the model was never the thing that needed to be trusted.