Designing Evidence Gates for Production LLM Systems
Every production LLM system I trust is built around one asymmetry: a model may interpret, propose, and draft; it may not certify that its own output is correct. Certification belongs to gates. A gate is a check that sits outside the model, runs in deterministic code or under independent constraint, and has the authority to stop an output from taking effect. This post is a field guide to the gates I build, the order they run in, and the failure modes each one exists to close.
Why gates and not better prompts
The instinct when a model misbehaves is to strengthen the prompt. Prompts are policy, and policy drifts: models update, context grows, an instruction that held for a thousand outputs quietly stops holding on the thousand-and-first, and nothing tells you. A gate is ordinary code, so it fails the ways code fails (a bug, a stale configuration, a schema that no longer matches upstream), and you defend it the way you defend code, with tests and review. What it does not do is drift with the generator’s behaviour. That is the actual advantage: the check is decoupled from the thing being checked.
The deeper reason is epistemic. A language model’s fluency and its correctness are different properties, and in the systems I’ve built and measured, the generator’s own signals have not been a basis for certification: I don’t treat self-reported confidence as evidence that a claim is correct, and I don’t treat a model agreeing with itself across samples as verification, because repeated sampling happily reproduces a stable error with conviction. If the generator can’t certify its output and the generator’s signals can’t either, certification has to come from somewhere structurally outside. That somewhere is the gate.
Gate one: structure, parsed not interpreted
The first gate is the cheapest, and it catches a large class of failures for almost nothing: model output must conform to a declared schema, and non-conformance is rejection, not repair. Structured output, parsed against a frozen vocabulary of types, fields, and operations the system actually supports.
The rule with teeth is the second half: never coerce. When a model produces a value that is almost right (a synonym for a field name, a plausible operation that doesn’t exist, a date format nobody declared), mapping it to the nearest valid thing feels helpful and is actually the system guessing at intent. A guess that changes the output silently is worse than a rejection, because the rejection is visible and the guess is not. Out-of-vocabulary means out. The model can try again.
The compiler analogy holds precisely. A type checker doesn’t know whether a program is correct and doesn’t try; it knows the program isn’t incoherent, and it rejects incoherence before any human reviews the logic. Nobody replaces code review with type checking, and nobody skips type checking because review exists. Cheap mechanical gates run first because they are cheap and mechanical.
Gate two: executability, checked by the code that executes
A well-formed output can still describe something the system cannot do. A model drawing on a broad idea of what systems like yours usually offer will occasionally propose a capability yours doesn’t have. The output parses, every field is legal, and it is still a plan that fails at execution.
The strongest version of this gate has one design rule: validate the proposal with the same capability implementation that would execute it, not with a separate checker written to approximate that code. A parallel validator is a second implementation of the truth, and second implementations drift apart from the first. When the executor is the validator, the validator cannot approve an operation the system does not support, and that holds without anyone anticipating the specific ways a model might overreach.
Worth being precise about what this does and doesn’t buy. It is a structural guarantee about the proposal: everything that passes maps onto real, supported operations. It is not a guarantee the action will succeed at runtime, where a dependency can still be down, a permission revoked, a record gone. The gate eliminates the category of confidently impossible plans; runtime failure handling stays exactly as necessary as it always was.
Gate three: sufficiency, before any answer exists
The gates so far check form. The next one checks ground: before an answer is composed at all, a sufficiency gate decides whether the retrieved evidence covers what the question needs, and refuses with the gap named when it doesn’t. The contract this gate enforces (cite or refuse, and why refusal is a success state) has its own article; here the architectural point is narrower and about placement.
Sufficiency has to run before generation, not after. Once a fluent answer exists, everything downstream is biased toward keeping it, including the humans; a refusal is much easier to issue when there is nothing yet to be attached to. “I don’t know” must be a computable outcome, decided by coverage arithmetic in code, before the model has said anything persuasive.
Gate four: faithfulness, independently constrained
After generation, each claim is checked against the records it cites, under a narrow support criterion: does this evidence, read on its own, support this statement? The design property that makes this gate real rather than ceremonial is independent constraint. The verifier sees the claim and the cited records and nothing else. Not the generator’s framing, not its confidence, not the rest of the answer. A citation is a hypothesis the verifier tests, never a fact it inherits.
Break that constraint and the gate quietly dies. A verifier that reads the generator’s full output inherits its framing and starts agreeing with rhetoric instead of checking evidence; a verifier prompted alongside the generator in one context is the generator wearing a different hat. Independence is the same principle as blind review: the checker must not know what answer it is supposed to reach. And the honest limit: independence reduces correlated failure, the verifier falling for the same framing that produced the error. It does not make the verifier infallible. It is one more layer designed to block unsupported claims before display, sized for the failures it can actually see.
Gate five: the human, advisory checks kept in their place
The last gate is a person, and the first design decision is where to put them, because human review is expensive and attention is finite. Not every output needs it: cheap, reversible, low-blast-radius effects can flow on the strength of the mechanical gates alone. The human gate belongs where effects are material or hard to undo, and a system that demands sign-off on everything trains its reviewers to stop reading. Where the gate does sit, two rules keep it honest.
First, automated checks near the human gate are advisory, not approvals. A second model can flag “this looks wrong” to the reviewer; its suspicion must not silently block, and its blessing must not silently approve. The moment a model’s opinion is treated as the sign-off, the human gate has been inverted while its ceremony remains.
Second, what was reviewed is what takes effect. Once a person approves a proposal, it freezes. Read-only, then executed. Without that freeze, the artifact can shift between review and effect, and the signature attaches to something nobody actually read. Review-then-freeze-then-act sounds bureaucratic and is three lines of state machine.
And the honest caveat: the human gate is load-bearing and can be wrong. A reviewer can approve the wrong thing with the whole machine’s confidence now behind a human mistake. The answer isn’t removing the person; it’s recording what they decided and why, so judgment can be re-examined instead of reconstructed from memory.
Composition: order is most of the design
The gates compose in a fixed order: structure, executability, sufficiency, generation, faithfulness, human. Cheap and mechanical first, expensive and judgmental last, and each stage only ever sees material that survived the previous one. Running a model-based check on something a parser would have rejected isn’t just wasted cost; an invalid candidate present at a judgment stage contaminates the judgment.
Two failure modes recur when teams adopt gates, and both are worth naming. Gate theatre: checks that log warnings instead of blocking. A gate without the authority to stop an output is telemetry, and under deadline pressure telemetry gets ignored. Gate erosion: every gate accumulates pressure to add exceptions, because gates block things and blocked things annoy people. The moment an exception path exists that skips a gate silently, the gate’s guarantee is gone and its reputation remains, which is strictly worse than having no gate.
The one-sentence version: models propose, gates certify, and the two must never be the same component. Certify means certified against the system’s declared contract (its schema, its capabilities, its evidence requirements), which is the only kind of correctness a gate can check and the kind that matters operationally. Get that separation right and you can use surprisingly modest models in surprisingly serious places.