Voluntary AI incident disclosure is structurally inadequate for AI safety governance. Current voluntary reporting is fragmented, systematically biased toward underreporting, and subject to strong competitive and legal disincentives. Drawing on mandatory incident reporting regimes in aviation, nuclear power, and pharmaceuticals, this paper proposes a mandatory AI Incident Reporting System (AIRS) specifying coverage criteria, disclosure obligations, confidentiality protections, and governance structures. We argue that the costs of mandatory implementation are substantially lower than the costs of continued opacity, and that voluntary disclosure will not self-correct given current incentive structures.
When an AI system causes harm—misdiagnoses a patient, incorrectly denies a loan, generates dangerous instructions—there is currently no systematic requirement to report the incident, analyze its causes, or share findings with the broader development community. AI failures occur in epistemic darkness: known to those immediately involved, invisible to everyone else.
Several voluntary AI incident databases exist—most notably the AI Incident Database (AIID)—but they suffer from systematic limitations. They rely on media coverage, meaning low-profile harms are invisible. Victims often don't know AI was involved in a decision affecting them. Developers have strong incentives not to self-report. And taxonomic inconsistency makes cross-database comparison unreliable.
Safety information is a public good. Individual organizations bear full disclosure costs but capture only a fraction of social benefits. The rational response under voluntary disclosure is to consume safety information produced by others while minimizing one's own reporting.
Voluntary incident disclosure creates legal exposure. Documents describing failures may be discoverable in litigation. This creates strong incentives for incident documentation to occur, if at all, exclusively in privileged attorney-client communications rather than shareable form.
Incident reports reveal proprietary details about architecture, training, and failure modes. Voluntary disclosure risks providing competitors with exploitable information while imposing reputational costs.
Aviation's mandatory incident reporting regime—including the Aviation Safety Action Program's confidentiality protections for self-reporters—has contributed to a 90%+ decline in commercial aviation fatal accident rates since the 1970s. Nuclear near-miss reporting has driven systematic reactor design improvements. FDA adverse event reporting has identified drug interactions invisible in clinical trials.
Common design principles across these successful regimes: clear definitional criteria; graduated timelines by severity; confidentiality protections that encourage self-disclosure; independent analysis capacity; and feedback mechanisms to the regulated community.
The mandatory AI Incident Reporting System (AIRS) applies to AI systems deployed in high-stakes contexts: healthcare decision support, credit and employment screening, criminal justice risk assessment, critical infrastructure, autonomous vehicles, and systems with more than one million active monthly users.
Reportable events include: serious physical harm; significant financial harm; legal violations; data breaches; and behavior significantly inconsistent with stated capabilities. Timelines: 72-hour initial notification for serious incidents; 30-day full report; quarterly consolidated near-miss reporting.
Confidentiality protections address chilling effects: privilege protection for compliant reports; trade secret protections for proprietary information; safe harbor from enforcement for promptly self-disclosed incidents (except gross negligence or intentional wrongdoing).
Voluntary AI incident disclosure has failed to produce the safety information ecosystem that responsible AI governance requires. Mandatory incident reporting—modeled on aviation, nuclear, and pharmaceutical precedents—is both necessary and feasible. The analogy is direct: in each precedent domain, mandatory reporting was initially resisted as burdensome and ultimately recognized as a foundation of safety progress. AI governance should not wait for a catastrophic event to reach the same recognition.