Post-Incident Review for an AI System

Run a post-incident review. Blameless — the goal is to change the system, not the people.

What happened: [description] When and how it was detected: [detection source] Duration: [from onset of impact to resolution] Impact: [who was affected, how many, and what the actual harm was]

Produce:

1. Timeline — from introduction of the cause through full resolution. Separate three distinct moments: when impact began, when anyone knew, and when work started. The gap between the first two is usually the most important finding.
2. Root cause — keep asking "why" until the answer is a decision or a process, not a person. "A developer forgot" is not a root cause; "no check catches this" is.
3. AI-specific questions:
   • Did the system return a wrong answer while raising no error at all?
   • Was there gradual quality drift nobody noticed?
   • Was a prompt or model version change the cause, and was that change reviewed?
   • Did the output feed an automated decision without human review?
4. What worked — what prevented worse harm. These are mechanisms to preserve.
5. Corrective actions — each with: description, owner, type (prevention / detection / harm reduction), and estimated cost. Aim for balance: a fix that prevents, and a check that catches it if prevention fails.
6. What remains open — a known risk deliberately not addressed, and the reasoning.

Success measure: this incident cannot recur in the same form, and if something similar occurs it surfaces in minutes rather than days.