A practical AI governance framework assigns an owner, defines permitted behavior, tests consequential failures and keeps evidence of decisions and actions. Apply risk controls throughout delivery and operations. A policy document matters when the running system can enforce it and a responsible person can respond.
What the current standards actually establish
NIST’s AI Risk Management Framework page describes the framework as voluntary and states that AI RMF 1.0 is being revised. Its 2026 critical-infrastructure profile announcement is a concept note, not a completed replacement framework. Keep these statuses distinct when writing an internal compliance claim.
The AI RMF Core organizes risk work into Govern, Map, Measure and Manage, with governance spanning the other functions. ISO/IEC 42001:2023 addresses an AI management system. Neither an editorial checklist nor a reference to these frameworks establishes that your particular system complies with applicable law or has achieved certification.
For technical action risk, OWASP’s excessive-agency guidance is a useful complement. The operating model below translates the general risk-management principles into concrete implementation recommendations. It does not claim that the exact tables are mandated by NIST or ISO.
Start with an inventory that includes what the AI can change
List the system’s purpose, business owner, affected people, data sources, provider, model configuration, tools and deployment environment. Describe the completed business outcome in plain language. “Uses a large language model” is not a useful inventory entry; “drafts support responses from an approved knowledge base” is.
Then record the authority boundary. An assistant that suggests a refund and an agent that submits it have different consequences even if they use the same model. Identify which operations read data, send communications, modify records or move value. Include indirect capabilities: a general browser or shell tool may permit much more than its friendly tool name suggests.
Record supplier dependencies and change paths. Which team can replace a model, edit a policy prompt or expand a connector’s permissions? What evidence is required before that change reaches production? A well-reviewed launch can become unsafe if updates bypass the original controls.
A compact risk classification
Classify impact, data sensitivity, action scope and reversibility separately. A low-cost workflow can still expose private information. A reversible draft may still mislead a reviewer. Use the classification to decide review depth and runtime restrictions, then document the rationale.
Bring process owners, engineering, security, privacy and relevant domain reviewers together for consequential uses. Clarify who makes the release decision and who can stop the workflow. A committee without an operational owner can produce agreement while leaving incidents unanswered.
A control matrix tied to observable behavior
| Decision area | Control to implement | Evidence to retain |
|---|---|---|
| Purpose and ownership | A bounded workflow with a named owner and excluded uses. | System record, approved scope and escalation contact. |
| Data access | Server-enforced user and tenant authorization, retention and source selection. | Access tests, data-flow map and source revisions. |
| Model behavior | Representative evaluations with severity-based acceptance criteria. | Versioned test cases, outputs, review decisions and unresolved failures. |
| Action authority | Typed tools, operation limits, approval gates and idempotency. | Proposal, approval, submitted command and verified result. |
| Change management | Model, prompt, retrieval and tool changes pass an appropriate release review. | Change record, regression results and rollback reference. |
| Incident response | A tested stop path, exception queue and communication owner. | Incident timeline, containment, remediation and follow-up tests. |
Keep controls close to the point they govern. A system prompt asking the model to respect a spending limit is weaker than a write service that rejects commands exceeding that limit. A user interface showing an approval checkbox is insufficient if the backend accepts the action without proof of approval.
Human review needs a clear job. Show the source evidence, proposed change, affected record and uncertainty. Give reviewers enough time and authority to reject the proposal. Measure whether review actually catches important errors rather than merely recording that someone clicked a button.
Worked example: governing a customer refund agent
Illustrative system: a support agent reads an order, explains return policy and proposes a refund. The business allows automatic submission only for a narrow, approved class of cases; exceptions go to a supervisor. This is a design example, not a report of an existing customer deployment.
The inventory states which order data can be read and which refunds the workflow can propose. The policy service checks product, country, current order state and the applicable policy revision. The write service checks authority and approval, submits a command once, then confirms the resulting refund record.
The evidence pack joins the conversation ID, retrieved policy version, order state, eligibility decision, approval and resulting transaction ID. It avoids unnecessary personal data in logs. The support team can answer a complaint without reconstructing the event from an unstructured transcript alone.
The acceptance test includes an already-refunded order, a changed policy, a duplicated tool call, a revoked user permission and a malicious instruction inside a customer attachment. The system must treat that attachment as evidence to inspect rather than authority to change its own rules.
One incident scenario matters especially: the refund succeeds but the confirmation response is lost. The agent should check the existing operation and report its real state. A blind retry can turn an infrastructure error into a financial loss.
The governance question is concrete: who permitted this action, which evidence supported it and how can the business correct it?
Build a release evidence pack that can be reviewed
- Purpose record: describe intended users, use cases, excluded actions and the accountable owner.
- Data record: list source ownership, allowed access, retention and relevant vendor settings.
- Evaluation record: retain representative examples, severe failure cases, scoring instructions and actual results.
- Authority record: document tool permissions, approval conditions, operation limits and recovery behavior.
- Operating record: assign monitoring, exception handling, incident response and supplier-change review.
- Release decision: state which failures remain, why they are acceptable within the scope and what triggers rollback or renewed review.
Measure risk with denominators and scope. “No failures found” after ten happy-path cases has little meaning. State the sample, conditions and severity definitions. A single unauthorized data disclosure may be a release blocker even when a general accuracy score looks strong.
Choose runtime indicators that correspond to the actual hazards: permission denials, unsupported answers, rejected tool commands, repeated transactions, queue age and unverified completion. More dashboards do not automatically improve governance; each alert needs an owner and a response.
Use the control-plane guide for runtime implementation and the observability guide for event evidence. A framework and an engineering control should connect through an explicit requirement, not merely share the word “governance.”
A rollout sequence that does not outrun its controls
Begin with read-only or draft-only operation in a bounded workflow. Validate data access and representative answers before enabling writes. Use shadow-mode proposals to learn what the system would do without exposing customers to unapproved actions.
Next, enable a limited class of operations with monitored approvals and a staffed recovery path. Review the failures, review burden and benefit together. A workflow that requires reviewers to redo every answer may be controlled but economically unattractive.
Expand tool scope, user population or autonomy through explicit change decisions. Re-run the relevant tests when models, source data or permissions change. Avoid treating a supplier’s model update as automatically equivalent to the version you evaluated.
Record local legal and sector requirements separately from voluntary framework alignment. Have the responsible legal or compliance function assess applicability using current authoritative rules for the actual jurisdiction and use case. A generic checklist cannot establish that determination.
The runtime risk guide helps turn broad principles into boundaries around credentials, delegated actions and sensitive workflows. Governance should make these boundaries understandable to the people operating the system.
Questions about enterprise AI governance
Does using NIST AI RMF establish legal compliance?
No. NIST describes AI RMF as voluntary risk-management guidance. Applicable legal obligations require a separate assessment of the jurisdiction, sector and use case.
Is a system prompt enough to enforce action limits?
No. Enforce permissions, limits and approval conditions in the services that execute actions, and test whether the backend rejects prohibited commands.
What must human review include?
Show the relevant evidence, proposed action, affected record and uncertainty. Give reviewers the authority, time and tools to reject or correct the proposal.
When should governance be reviewed again?
Review material changes to models, prompts, data sources, tools, permissions, users or autonomy, as well as incidents that reveal a previously untested failure.
Primary references checked for this edition
- NIST AI RMF status and purpose
- NIST AI RMF Core functions
- ISO/IEC 42001:2023 overview
- OWASP excessive agency
Checked on 11 October 2026. This edition replaces unsupported failure statistics and deployment anecdotes with a practical operating framework and an explicitly illustrative scenario.
Research Path