How to Check Whether Your AI Agent Is Production-Safe (10-Minute Governance Audit)

You've built an AI agent. It works in the demo. Before it touches anything real — a customer, a database, a payment — there's one question worth ten minutes: what is the worst thing this agent can do, and what stops it? This is a practical, framework-neutral audit you can run on any agent, whether it's built on Claude, LangGraph, OpenAI, or n8n. At the end there's a free scanner that does the first pass for you.

The reason this matters: most agents that fail in production don't fail from a clever jailbreak. They fail because a consequential tool — one that spends money, sends a message, or changes a record — was wired to run without a human, and one day the model was confused, or wrong, or fed bad input, and it acted. The audit below is built to catch exactly that, before your users do.

1. Name the worst-case action

In one sentence, write down the most harmful irreversible thing your agent could do. Not "give a bad answer" — the specific, damaging, hard-to-undo action: issues a refund to the wrong account, emails a customer the wrong contract, deletes a production record, publishes an incorrect status update. If you can't name it, you don't yet understand your agent's blast radius — and neither will whoever reviews it. Everything else in the audit is measured against this sentence.

2. Check that the tool behind it is gated

Now find the tool that could cause that worst case. Ask a single question: can the agent call it without a human approving first? If yes, that's your highest-priority finding. The fix is a human-approval gate — the tool requires an approval token the model cannot mint itself. This is the one control that holds even when the model is jailbroken, because the boundary lives outside the model. A prompt can ask the model to behave; only a gate can stop it.

A useful test: separate your tools into three buckets — read-only (safe to auto-run), reversible writes (usually safe), and irreversible or consequential actions (gate these). Anything that spends, sends, deletes, or provisions belongs in the third bucket.

3. Verify the bounds

An agent that loops forever, or spends without a ceiling, is a slow-motion incident. Confirm three bounds exist: a cost ceiling (max spend per action, and per run), a loop bound (max iterations, with an escape hatch so it can't spin), and an output boundary (it can't emit things it shouldn't — PII, secrets, unbounded content). These are cheap to add and they're the difference between "the agent misbehaved" and "the agent misbehaved 4,000 times before anyone noticed."

4. Confirm an audit trail

When something goes wrong, can you reconstruct what the agent did and why? Every consequential decision should be logged — the input, the tool called, the reasoning, the outcome. Without this, your postmortem is guesswork, and you can't prove to a security or compliance reviewer that the gates held. A tamper-evident log (each entry signed over the previous one) is better still, because it detects after-the-fact edits.

5. Assign a Trust Level

Finally, place the agent on a simple ladder by how far it can run before a human must step in. This is the AgentAz Trust Level: A0 has no tools; A1–A2 are advisory (they can't change anything); A3 can act but only behind a gate; A4 acts autonomously within declared bounds; A5 acts autonomously without bounds. Most agents that should ship land at A3 — able to do real work, but with every irreversible action held at a human gate. If your audit puts you at A4 or A5, be sure that's a deliberate decision, not an accident of wiring.

Let the scanner do the first pass

Running that audit by hand is the right instinct, but you don't have to start from a blank page. The AgentKits Compliance Scanner reads your system prompt (or an agentaz.json spec), scores it against published agent-governance guidance, and hands you the same findings above — which gates are missing, which failure scenarios you're exposed to, and a concrete fix list. It's deterministic (same input, same result), runs in about ten seconds, and never stores your prompt. Treat its output as a fast first pass, then apply your own judgment on the worst-case action — that part is yours to own.

If you want to track an agent over time rather than scan it once, Agent Watch remembers your agent's Trust Level and warns you when a change quietly makes it riskier — a tool slipping out from behind its gate is the exact regression that causes incidents.

From audit to build

An audit tells you where the gaps are; a good blueprint shows you what "done right" looks like. Every kit in the registry is built with a documented worst-case action, gated tools, and a Trust Level — so they double as worked examples of this audit passing. For a full walkthrough, the Incident Root-Cause Analysis Agent tutorial builds an agent from the boundary outward: it names the worst case first, gates the two tools that publish anything, and deliberately omits any tool that could remediate — which is exactly what keeps it safe to point at a live incident.

Ten minutes of this audit is cheaper than one production incident. Name the worst case, gate the tool behind it, bound the rest, log everything — and let the scanner catch what you miss.

Reading the report: from verdict to framework crosswalk

The scanner doesn't just return a pass/fail. It returns a Trust Level verdict, a per-gate breakdown with the exact evidence for each finding, a risk radar across the governance dimensions, and a fix block you can paste straight into your system prompt. Where a gap exists, it also shows a framework crosswalk: the specific control your missing safeguard maps to in the references security and compliance teams already use — OWASP's Agentic threats, the NIST AI RMF, and ISO/IEC 42001. A missing approval gate, for instance, maps to OWASP's Excessive Agency, NIST MANAGE-2.1 (oversight and intervention), and ISO 42001's human-oversight control.

Those mappings are interpretive — they take judgment, not a lookup — so each one is human-verified and dated rather than auto-generated, and you can see the full set on the regulatory crosswalk. The point isn't to claim certification; it's to translate an agent's governance gaps into language a reviewer already speaks, so the conversation moves from "is this safe?" to "here's exactly what to fix, and here's the standard it satisfies."