Operator OSruns offline
Audit

What makes an AI agent audit log verifiable, and why most of them aren't

Every agent platform has logs. Almost none of them are verifiable, and the distinction matters the moment someone outside the team asks what the agent did. A log is verifiable when a person who does not trust you, does not have an account on your system, and cannot see your code can still confirm that the record they were handed is complete and unaltered.

Three properties

Tamper evidence. Each line commits to the previous one: hash = sha256(prev_hash + canonical_json(line)). Change one byte on line 4,212 and every hash from there forward is wrong. Delete a line and the chain breaks at the gap. Append a fake line and its prev_hash won't match. The log can still be destroyed, but it cannot be quietly edited.

Reproducible decisions. The log records that an agent asked to spend $80 and was denied. For that to mean anything the assessor has to be able to take the same policy file and the same request and get the same denial. That rules out a model as the decider. The evaluator must be a pure function of the policy, the request, the profile, and the clock.

Independent verification. If checking the log requires logging into the vendor's console, the vendor is still the trusted party. The verifier has to be a separate artifact, small enough to read, with no dependencies, that anyone can run on the exported file.

The verifier

$ python3 verify_chain.py audit.jsonl --expect-head 9f3c41ae…
INTACT  14212 events  head 9f3c41ae6b2d…

$ python3 verify_chain.py audit.jsonl
ALTERED at line 4212: hash does not match content
(4211 events verified before the break)

The one we publish is about 120 lines of Python, standard library only. It takes field names as flags so it works on any log with this shape, not just ours. The --expect-head flag closes the last gap: the exporter publishes the final hash out of band (in an email, a ticket, a signed PDF), and the assessor checks that the file they received ends where the exporter said it did. Truncating the tail of a chain is otherwise undetectable.

What a line holds

Agent identity (a key, not a name), action, resource, the context that drove conditions (amount, counterparty, time), the effect, the rule ids that matched, the obligations that attached, and the timestamp. Not the content of the email the agent wanted to send. The log proves what was permitted and what was asked; it is not a copy of the data. That separation is what lets it be exported to an auditor without becoming a second PII store.

The verifier described here, verify_chain.py, ships with Operator OS, the business OS it was built for.
Implementation reference: the person layer in Operator OS, lib/person/audit.py and verify_chain.py. The hash chain construction follows the pattern used in append only ledgers generally; nothing here is novel cryptography, which is the point.