Your AI Agent Was Safe Last Week. Is It Still? Catching Governance Drift
The dangerous change is rarely the obvious one. It's a tool quietly moved to auto-execute, a bound deleted, a tier that crept up while no one was looking. Here's how agent governance drifts — and how to catch it.
Here's a failure mode that doesn't show up in demos. You design an agent carefully. You name its worst-case action, you gate the dangerous tools, you set bounds. It ships at a sensible Trust Level — say A3, able to act but with every irreversible step behind a human. It's safe. Then, three weeks later, it isn't — and nobody decided to make it unsafe. It drifted there.
Governance drift is the quiet way agents become dangerous. Not a jailbreak, not a bad design — a series of small, reasonable-looking changes that each pass review and together move the agent past the limits it was approved under. This is arguably the most under-watched risk in production agents, precisely because every individual step looks fine.
How an agent drifts
The pattern is always the same: a change that reads as a convenience is actually a loosening of a boundary. A few concrete ways it happens:
A tool slips out from behind its gate. Someone notices the agent "gets stuck waiting for approval" on a common, low-risk case, and moves the tool from approval-required to auto-execute. Reasonable — except the same tool also handles the high-risk case, and now that runs without a human too. This is the single most common drift, and it's the one that causes refunds to the wrong account and messages sent that shouldn't have been.
A bound gets removed. The cost ceiling or loop limit trips during a legitimate edge case, so someone raises it "temporarily" or deletes it. The bound that would have contained a future runaway is now gone.
A capability is added without re-checking the tier. A new tool is wired in to handle a new case. It's useful. It also happens to be able to change records — which just moved the agent from advisory to acting, from A2 to A3, or from gated to autonomous — but nobody re-ran the classification, so on paper it's still whatever it was.
In every case, the agent's real Trust Level rose while its documented one stayed put. That gap — between what the agent can now do and what it was approved to do — is where incidents live.
Why normal review misses it
You'd think code review would catch this. Mostly it doesn't, for a specific reason: in a diff of raw code, moving a tool to auto-execute looks like a one-line config change, not a safety regression. The reviewer sees approval_required: false and reads "small tweak," not "this tool now runs without a human on the action that could refund the wrong account." The risk is real but it isn't legible in a code diff.
What makes it legible is diffing the governance, not the code. If you can see "Trust Level rose A3 → A4" or "issue_refund moved from gated to auto-executable — now runs without approval," the regression is obvious to anyone, no deep reading required. That's the whole idea: make the Trust Level a thing you regression-test.
Catch it by testing the tier, not trusting it
The fix is a habit, then a tool. The habit: whenever an agent's spec changes, re-check its Trust Level and compare against the last known-good version — the same way you'd run tests after a change. The tool: Agent Watch does exactly this in your browser. You paste your agent's agentaz.json, it computes the tier and a snapshot of the tool boundary, and every time you re-check after a change it tells you what moved — a tier that rose, a tool that lost its gate, a new autonomous action. It's the regression test for governance that a code diff can't give you.
It runs entirely client-side (your spec is never uploaded), so it's zero-friction to point at an agent you're actively developing. The point isn't ceremony — it's that the one change most likely to cause an incident is also the one most likely to be waved through, and a tier diff is what stops that.
Where this fits
Drift-checking is the last step in a chain worth running end to end: specify the agent's boundaries in an AgentAz spec, audit the design with the 10-minute governance audit and the Compliance Scanner, build it from the boundary outward (the Incident RCA tutorial is a worked example), and then keep it safe by watching for drift. For teams with a compliance obligation, this is also how you satisfy the continuous-monitoring expectations in frameworks like NIST AI RMF and ISO 42001 — see the AgentAz regulatory crosswalk for the mapping.
An agent that was safe at launch is not safe forever. It's safe until the next change — and the next change is where you need to be looking.
Frequently asked questions
It's when an agent becomes riskier over time through small, unremarkable changes — a tool moved out from behind its approval gate, a cost or loop bound removed, a new capability added — none of which get a second look individually, but which together move the agent past the limits it was approved under.
Make the agent's Trust Level something you regression-test, not something you set once. Re-check the spec whenever it changes and compare against the last known-good version. Agent Watch does this in the browser: it computes the tier from your agentaz.json and flags what changed since your last check.
Code review catches a lot, but governance drift is exactly the kind of change reviewers wave through — moving a tool to auto-execute looks like a small convenience, not a safety regression. An explicit tier diff makes the risk legible where a diff of raw code doesn't.