This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The AI Incident Database editors downgraded several former incidents, including a GPT-3 medical chatbot experiment and a Tesla Autopilot vulnerability, to issues because they represented academic findings, humor, or projected risks rather than real-world harm.
TheFaceTag app, a social networking app developed and deployed within-campus by a student at Harvard raised concerns surrounding its facial recognition, cybersecurity, privacy, and misuse. This incident has been downgraded to an issue as it does not meet current ingestion criteria.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
An experimental GPT-3 medical chatbot suggested that a mock patient kill themselves.
The reports detail various AI systems failing to perform reliably or safely under experimental conditions, such as GPT-3 suggesting suicide or Tesla Autopilot being vulnerable to adversarial stickers.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Other
Without a clearly specified time of occurrence
The underlying reports describe unintentional AI system vulnerabilities, limitations, and unexpected behaviors across various experimental or pre-deployment settings.
Added categories peak at 2: Minor (Epistemic Harm, Inferred)
The incident database administrative action to downgrade several historical reports highlights that these events represented academic findings, humor, or projected risks rather than real-world national security threats. None of the evaluated incidents resulted in physical harm, critical infrastructure compromise, or information warfare.
The report is a compilation of multiple different AI systems (including GPT-3, Tesla Autopilot, and a facial recognition app) and does not characterize a single AI system's mode of operation for the entire incident log.
High Risk: The report includes systems like Tesla Autopilot (critical transport infrastructure) and facial recognition (TheFaceTag), which fall under High Risk (Level 2) applications, alongside chatbots (Level 3).
The report covers multiple AI systems, with GPT-3 (used as a chatbot and writing assistant) and Tesla Autopilot (autonomous driving) being the most prominent examples discussed.
The report clearly outlines the administrative decisions to downgrade several incidents, providing explicit reasons for each (e.g., 'academic finding', 'projected harms', 'not deployed in the real world'). This makes the assessment of zero real-world harms highly certain.
One could interpret the Nabla GPT-3 medical chatbot experiment as a real-world incident of toxic content generation, but the text explicitly notes it was an experiment with a mock patient and not deployed, justifying the downgrade.
Technology
Inconsistent LLM Outputs
GPT-3 displayed highly inconsistent responses during medical triage tests.
Lack of Safety Guardrails
The model lacked hard constraints to prevent recommending self-harm.
Data Inputs
General Web Training Data
The model was trained on general web text rather than verified medical facts.
Human Factors
Experimental Prompting
A mock patient tested the chatbot with highly sensitive mental health queries.
Process and Methods
Inadequate Clinical Validation
The deployment lacked rigorous clinical safety testing and validation processes.
Management
Premature Deployment
Management experimented with LLMs for clinical tasks before verifying safety.
Regulatory Environment
Absence of LLM Medical Standards
There were no regulatory standards governing the medical use of LLMs.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
No population impact data reported.
