Offline AI agent policy: why the engine can't be a model

Most "AI guardrails" products are models. A classifier scores the prompt, a judge model reviews the output, a safety model decides whether the tool call looks dangerous. They run in someone's cloud, they need a network connection, and they give a slightly different answer every time. For a chatbot that is fine. For deciding whether an agent may send money, send email, or run a command under a person's name, it fails three tests at once.

It fails on managed devices

On a government furnished laptop, a bank's locked down desktop, or any air gapped network, the cloud guardrail is not there. Workers in those environments either go without agents or route around policy. A policy engine that only works online is a policy engine for the people who needed it least.

It fails on reproducibility

An assessor asks: why was this allowed? The honest answer from a model based guardrail is "the model scored it 0.31 and the threshold was 0.4." Run it again, get 0.43. There is no rule to point at, no version to cite, and no way to show the same inputs would give the same answer tomorrow.

It fails on authority

A model can be talked into things. Prompt injection against a judge model is a published attack class. A policy written as data, evaluated by ordinary code, cannot be persuaded. Deny beats ask beats allow, explicit beats default, the default is deny, and nothing in the request text changes that ordering.

What works instead

- id: r_send_email_ask
  subject: "*"
  action: send
  resource: email
  effect: ask
  ask_ttl_minutes: 60
  obligations: [log, notify]

A small vocabulary: subject, action (read, write, send, spend, execute, share), resource, effect, conditions (time window, amount ceiling, counterparty list, confidence floor), obligations (log, notify, redact). An evaluator that is a pure function. A harness that wraps the agent's tools so the check is not optional. And a hash chained record of every decision.

The model still has a place. It can rephrase a question, summarize facts, pull structured facts out of a document, or turn "never let anything email my accountant without asking" into the YAML above. Everything it produces is tagged as proposed until the owner confirms. It helps. It never decides. The system has to pass a parity test: a few hundred recorded requests produce byte identical decisions with the model attached and with it unplugged. If that test fails, the model has authority it should not have.

Bring your own model

Once the model is a helper rather than the engine, which model stops mattering. Claude through the API, an OpenAI compatible endpoint, or a GGUF file running locally through llama.cpp all plug into the same four method adapter. On a managed device, the local one is the only option, and it works because the thing that matters never needed a model at all.