AI Incident

An AI incident is an event in which an AI system caused harm, nearly caused harm, or behaved materially outside its intended operating envelope. The term deliberately echoes aviation's safety-incident vocabulary because the analogy matters: a mature engineering discipline treats failures as learning opportunities, records them systematically, and feeds the lessons back into design and operations. Incident categories include harmful output (toxic, biased, misleading), privacy incident (data leakage, memorisation, training-data exposure), safety incident (dangerous instruction, self-harm enabling), operational incident (wrong action via tool use, integration failure), and reputational incident (visible failure that damages trust even if no direct harm).

The institutional infrastructure grew up during 2022-25. The AI Incident Database (AIID, Partnership on AI / Responsible AI Collaborative) is the public archive — pass 3,000 incidents by 2026. The OECD AI Incidents Monitor aggregates regulator-reported incidents. NIST's AI Risk Management Framework treats incident response as a required practice. In APAC, Japan's AI Bill and Korea's AI Basic Act both mandate incident reporting for high-impact systems. The vocabulary for describing incidents (severity, affected population, root cause category, mitigation class) is converging, which makes cross-organisation learning possible for the first time.

For APAC mid-market teams, the pragmatic incident practice has four parts: **intake** (a single channel for users, operators, and monitoring to report concerns — do not require them to route between security and product), **triage** (severity assignment within 24 hours using a simple rubric — business impact × user harm × scope), **post-mortem** (within 5 days for severity-high and above, blameless, root-cause focused), and **lessons log** (quarterly review of themes, fed into governance and engineering practice). For regulated industries this also includes external reporting against regulator timelines.

The non-obvious failure mode is **silent incidents**. Users complain in support tickets, employees flag odd outputs in Slack, someone screenshots a bad response — and none of it becomes a post-mortem because the intake channel is fragmented. Real incident programmes treat a meaningful fraction of user complaints as proto-incidents, audit them, and only dismiss with documented rationale. Programmes that never raise an incident are not safer than their peers — they are less observant. Zero-incident quarters are a red flag, not a trophy.