describe the-careful-machine
The Careful Machine
AI for high-stakes work: every claim traces to a record, and weak evidence ends in a refusal, not a guess.
cite_or_refuse = enforced
Described at an architectural level. The underlying work sits in a sensitive, private-domain environment, so this page covers how the system is built, not what it touches.
Problem
Some environments can't tolerate a confident guess.
When an answer will inform a real, high-stakes decision, “probably right” isn't good enough: every claim has to trace back to evidence, and the system has to know when it doesn't know. That rules out a single model answering from memory. The design problem is an AI that reasons over retrieved facts, shows its work, and declines rather than fabricates.
Architecture
Ground before you reason
Retrieve relevant evidence first, then let the model reason over those facts, never free-associating from parametric memory.
Tier the models
Route cheap, common work to small local models and escalate to heavier reasoning only when the task earns it, keeping cost and data exposure low.
Separate extraction from judgment
Deterministic components structure the inputs; the reasoning layer works over that structure rather than raw noise.
Citation is first-class
Every surfaced claim carries its source. An answer without defensible evidence is downgraded or withheld.
Tradeoffs
Defensibility over latency
Retrieval and verification passes cost time, acceptable when a fast but wrong answer is worse than useless.
Restraint over recall
Tuned to refuse under uncertainty, accepting some "can't support that" answers to eliminate confident fabrication.
Local-first over raw capability
Small local models trade peak capability for control and privacy; tiering buys it back only where justified.
Verification
Cite or refuse
Enforced as a gate, not a suggestion.
Faithfulness checks
Generated claims are compared against their cited sources, automatically.
Human in the loop
The system proposes, a person decides, with the full reasoning trace available for audit and replay.
Failure modes
- Weak retrieval
- Thin or contradictory sources trigger a refusal.
- Unsupported claims
- Anything the model asserts without a backing record is blocked before display.
- Silent drift
- Every answer is replayable, so a bad output can be traced to its inputs rather than argued about.
- Over-trust
- A person stays in the decision path for anything consequential.
Deliberately not
- Present output as fact when the evidence is not there. Refusal is a feature.
- Send data to external services or general-purpose cloud APIs.
- Make final calls on its own. Output is a draft until a person accepts it.
My role
End-to-end owner.
- retrieval + grounding
- routing logic
- attribution + refusal paths
- replay logging
- locked-down deployment
I set where the boundaries sit and what “not confident enough to answer” means in practice.
Outcome
A reasoning system trusted to sit in front of decisions that matter: answers that hold up under audit, refusals when they wouldn't.
Stack
- RAG
- ReAct reasoning
- ONNX classification
- NER
- citation verification
- local-first model tiering