Responsible AI in Practice: A NIST AI RMF Walkthrough for Operators

TL;DR

• The NIST AI Risk Management Framework (AI RMF 1.0) is the most useful free resource available for operationalising responsible AI.
• Its four functions (Govern, Map, Measure, Manage) translate cleanly into a mid-market enterprise operating model.
• The deliverable for each function is concrete: a charter, a system inventory, an evaluation harness, and a response runbook.

Why now

Responsible AI is moving from principles to practice. The most cited principles documents (the OECD AI Principles, the IEEE Ethically Aligned Design, the EU's Ethics Guidelines for Trustworthy AI) are useful for setting tone. They do not tell an operator what to do on Monday morning.

The NIST AI Risk Management Framework, published in January 2023 and now at version 1.0, is the closest thing to an operating manual for responsible AI. NIST has separately published a Generative AI Profile (July 2024) and is widely referenced by US, EU, and Asian regulators as a baseline.[^1] For mid-market enterprises in Asia that need a credible responsible AI programme without budget for ground-up design, the AI RMF is the right starting point.

The four functions

The AI RMF organises responsible AI work into four functions. Each function has categories and subcategories, but the high level is enough to drive a programme.

Govern. Establish the policies, accountabilities, and processes that guide AI risk management.

Map. Identify the context: what AI systems exist, who uses them, what could go wrong.

Measure. Quantify and assess risk. Run evaluations. Track metrics.

Manage. Treat the risk. Implement controls. Respond to incidents.

The functions are designed to run in parallel, not sequentially. A maturing programme has all four running, with Govern setting the cadence for the other three.

Govern: the AI charter

The Govern function asks you to write down what your organisation believes about AI and who is accountable for what.

Concrete deliverable: an AI Charter, signed by the executive committee. The charter should cover:

• Scope: which AI systems are in and out
• Principles: 5-7 binding principles (not 25 aspirational ones)
• Accountabilities: who decides, who approves, who monitors
• Approval thresholds: what level of risk needs what level of approval
• Review cadence: how often the charter itself is reviewed

A good charter is 4-8 pages. A charter that runs to 30 pages will not be read. The principles should be specific enough to drive decisions: "We do not use AI to make final decisions about employment outcomes" is binding. "We are committed to fairness" is decoration.

Map: the AI system inventory

The Map function asks you to know what AI you operate. This sounds trivial. It is not. Most mid-market enterprises discover at least 4x more AI use than the central inventory suggests, often consumer SaaS being used for work tasks.

Concrete deliverable: an AI System Inventory. For each system:

• Name, owner, business purpose
• Data inputs and outputs
• Underlying model or vendor
• User population
• Decision impact (does it inform a decision, or make one?)
• Deployment status (pilot, production, sunset)
• Last review date

The inventory should be a living document. Review quarterly. Add a row to the inventory in any procurement workflow that touches AI. Treat the inventory as the master list against which all other AI risk work is scoped.

Measure: the evaluation harness

The Measure function asks you to quantify risk. This is where most mid-market programmes fall short, because measurement requires engineering investment.

Concrete deliverable: an Evaluation Harness for each production AI system. The harness should run:

• Accuracy evaluations on a curated test set, with regression detection
• Fairness evaluations across the demographic dimensions relevant to your context
• Robustness evaluations including adversarial inputs and edge cases
• Privacy evaluations including PII leakage and inference attacks
• Operational evaluations including latency, cost per request, error rates

For LLM-based systems, the harness should include hallucination detection, prompt injection resistance, and refusal behaviour evaluations. Open-source eval frameworks (lm-evaluation-harness, OpenAI Evals, deepeval) provide a starting point.

The harness runs in CI. New model versions, prompt changes, and infrastructure changes all trigger re-evaluation. Results are reviewed monthly by the AI risk function, weekly by the engineering team.

Manage: the response runbook

The Manage function asks you to act on the risk. Detection is necessary but not sufficient. The deliverable is a Response Runbook for each production AI system.

The runbook should cover:

• Incident severity classification (SEV1, SEV2, SEV3)
• On-call paths and escalation
• Containment actions (rate-limit, fall back to a previous version, kill the system)
• Communication templates for customers, regulators, internal stakeholders
• Post-incident review template

The runbook is tested in tabletop exercises at least quarterly. Real incidents are reviewed in a no-blame post-mortem. Patterns across incidents inform the next iteration of the Map and Measure functions.

Implementation playbook

How to stand up a NIST AI RMF-aligned programme in 12 weeks.

1. Weeks 1-2: Charter draft. A 4-8 page document with scope, principles, accountabilities, thresholds. Get executive committee sign-off.
2. Weeks 3-4: Inventory. Survey every team. Include shadow AI. Aim for completeness over precision in v1.
3. Weeks 5-7: Evaluation harness for the top 3 production systems. Pick the systems with the highest risk or impact. Build a baseline eval harness for each. Open-source frameworks accelerate this.
4. Weeks 8-9: Runbooks for the top 3 systems. One page each. Severity, escalation, containment, communications.
5. Week 10: Tabletop exercise. Run a simulated incident through the runbook for one system. Observe what breaks. Fix.
6. Weeks 11-12: Cadence and metrics. Define the monthly AI risk forum, the quarterly executive review, and the metrics each forum will look at.

What good looks like at the end of 12 weeks

By week 12 you should have:

• A signed AI Charter
• An inventory covering at least 80% of AI use in the organisation
• An evaluation harness running in CI for the top 3 production systems
• A response runbook for each of those systems
• A tabletop exercise completed and learnings documented
• A monthly AI risk forum scheduled with named attendees

You will not have a perfect programme. You will have a programme that is real, defensible, and improving. That is what regulators, customers, and your board want to see.

What good does not look like

Anti-patterns to avoid.

A 60-page policy document with no operational mechanism. Common in mid-market enterprises that delegate responsible AI to legal alone. The document gets signed and ignored.

An evaluation harness that runs once at launch. Models drift. Prompts change. Vendors update their APIs. An eval harness that runs only at launch is decorative.

A risk register without thresholds. "Bias risk: medium." Medium according to whom, against what threshold, requiring what action? Risks without thresholds are status updates, not decisions.

A responsible AI programme owned only by legal or only by engineering. Both are wrong. Responsible AI sits across legal, engineering, product, security, and the relevant business functions. The Cynefin framework (Snowden, 1999) is useful here: AI risk is a Complex problem, not a Complicated one, and Complex problems require multiple perspectives in the room.

Counter-arguments

"NIST is a US framework. Our regulators are in Asia." True, and irrelevant. The major Asian regulators reference or accept NIST AI RMF mapping. The Singapore Model AI Governance Framework, the Korean AI Basic Act guidance, and the Japan AI Promotion Act all cite or align with NIST. Building NIST-aligned satisfies the Asian regulators in practice.

"This is too much for our size." It is more than nothing and less than full-bore enterprise governance. The 12-week programme above is achievable with 0.5 FTE for the duration. After launch, ongoing operation is 0.2-0.4 FTE depending on AI portfolio size.

"We are not regulated, so we do not need this." Customers ask for this even when regulators do not. Your EU and Japanese customers will start asking by 2026. Your enterprise customers globally already are. Build it for them.

Bottom line

The NIST AI Risk Management Framework is the most useful free resource available for operationalising responsible AI. Its four functions translate into four concrete deliverables: a charter, an inventory, an evaluation harness, and a response runbook. Twelve weeks of focused work gets a mid-market enterprise from zero to defensible.

The work is not glamorous. It is documentation, plumbing, and tabletop exercises. Done well, it is the difference between a responsible AI programme that survives a regulator inquiry and one that does not.

Next read

• AI Governance for Asian Enterprises: Mapping HK, SG, JP, KR, CN
• Hallucination Risk: A Practical Containment Framework for LLM Deployments

By Maya Tan, Practice Lead, AI Strategy.

[^1]: NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), January 2023; Generative AI Profile, July 2024.