Governing AI agents that act on your systems of record

Most enterprise AI governance programs were designed for a world in which AI produces an output and a person decides what to do with it. A model scores a loan application. An assistant drafts an email. A chatbot answers a policy question. The human stays in the middle, and the human is accountable for the action.

AI agents change that arrangement. An agent can read a customer record, decide that something needs to happen, and then do it: update a field in the CRM, raise a purchase order in the ERP, close a service case, or adjust an account setting in a core banking platform. The output is no longer a suggestion. It is a change to a system of record.

I have spent more than two decades implementing and running those systems of record, and the lesson is consistent: the risk sits in what gets written, not in what gets said. This article sets out why agents need different controls and what a workable control framework looks like.

Why answers and actions carry different risks

A model that generates an answer can be wrong, biased or poorly explained. Those are real risks, but they are usually contained by the person who reads the answer. An agent that takes an action adds a new set of risks that sit closer to operational risk than to model risk.

• Scale. A flawed instruction can be executed thousands of times before anyone notices, because agents do not get tired or pause to question a pattern.
• Reach. Agents often use integration credentials that were set up for system-to-system work, which can grant far broader access than any single employee would have.
• Chaining. One agent’s output can become another agent’s input, so an error can spread across systems and teams.
• Reversibility. Some actions, such as sending a customer communication or moving money, cannot simply be undone.
• Accountability. When a record changes, auditors and regulators will ask who made the change and on whose authority. “The AI did it” is not an acceptable answer.

None of this means agents should be avoided. It means the governance model has to move from reviewing outputs to controlling actions.

Know every agent, and bind it to an identity

You cannot govern what you cannot see. The first control is a living inventory of every agent in use, whether built in-house, embedded in a vendor platform or switched on by a business team inside an existing application.

For each agent, the inventory should record at least:

• Purpose and owner. What the agent is for, and the named business owner accountable for its behavior.
• Systems and data. Which systems of record it can read from and write to, and which data classifications it handles.
• Actions. The specific operations it can perform, such as create, update, delete, approve, send or pay.
• Underlying models and vendors. Which models it relies on and who provides them.
• Risk tier. A rating based on the impact of its actions, which then drives every other control below.

Many organizations find that agents are already live inside platforms they license, enabled through a configuration setting rather than a formal project. Discovery is often the most valuable first step.

Next, every agent should act under its own identity, not under a shared service account and not under the credentials of the person who set it up. That identity should be registered in your identity and access management processes like any other privileged user.

Permissions should follow least privilege at the level of objects, fields and operations. An agent that updates case status does not need to edit customer bank details. An agent that drafts payment instructions does not need authority to release them. Where an agent acts on behalf of a specific user, it should inherit no more than that user’s own access.

This is familiar ground for anyone who has run access reviews on CRM or ERP platforms. The difference is that agent permissions tend to be granted quickly, during a pilot, and then never revisited. Include agent identities in periodic access certification from day one.

Set approval thresholds by risk, not by habit

“Human in the loop” is often treated as a single switch. In practice it should be a set of thresholds tied to the risk tier of each action.

• Autonomous. Low-impact, easily reversed actions, such as tagging a case or updating an internal note, can run without review, subject to monitoring.
• Human on the loop. Moderate-impact actions run automatically, but a person reviews samples or exceptions after the fact and can intervene.
• Human in the loop. High-impact actions, such as changes to customer financial data, credit decisions, payments or external communications, require explicit approval before they take effect.
• Prohibited. Some actions should not be delegated to an agent at all, whatever the controls.

Thresholds should also account for volume and value. An agent may be trusted to issue a small goodwill credit, but not to issue hundreds in an hour, or one above a set amount. Approvers need enough context to make a real decision, or the approval step becomes a rubber stamp.

Build guardrails, logging and monitoring into the design

Guardrails are the technical limits that hold regardless of what the model decides. They include input and output filtering, validation rules on the records an agent writes, rate limits, spending and value caps, restrictions on which tools an agent can call, and protections against prompt injection from untrusted content such as inbound emails or documents.

Activity logging is what turns an agent from a black box into something you can audit. For every action, you should be able to reconstruct:

• which agent acted, under which identity and on whose behalf;
• what triggered it and what information it relied on;
• which model and configuration version were in use;
• what it changed, with before and after values;
• who approved it, where approval was required.

Logs should be tamper-resistant, retained according to your records policies, and linked to the audit trail of the system of record itself, so that an agent’s change looks no different from a human change when an auditor traces it.

Monitoring then uses those logs to spot drift and misuse: unusual volumes, actions outside normal hours, rising rates of human overrides, or a pattern of changes clustered on particular customers or accounts. Agents behave differently as data, prompts and underlying models change, so monitoring has to be continuous rather than a one-time test before launch.

Plan for incidents and rollback before you need them

Every agent that writes to a system of record needs an incident plan that is written and tested before go-live. At a minimum, that plan should cover:

• A kill switch. A fast, well-understood way to suspend an agent or revoke its permissions without taking down the wider platform.
• Blast-radius analysis. The ability to identify every record an agent touched in a given period, using the activity log.
• Rollback. A tested method to restore affected records to their prior state, and a clear process for actions that cannot be reversed, such as customer remediation.
• Escalation. Defined routes into your existing operational risk, information security and, where relevant, regulatory notification processes.

The good news is that most organizations already have these capabilities for other systems. The work is extending them to agents and making sure the people who run them know agents are in scope.

Where to begin

Agent governance does not need a new bureaucracy. It needs existing disciplines, including identity management, change control, operational risk and audit, applied deliberately to a new kind of actor. A sensible starting sequence is to inventory what is already live, tier it by the impact of its actions, tighten permissions and approvals on the highest tiers, and confirm that you can log, monitor and reverse what those agents do.

Done well, this does not slow adoption. It gives business leaders, risk teams and regulators the confidence to let agents do more.

This article is general information, not legal or regulatory advice.