This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The AI Incident Database editors reviewed and converted six historical reports—including a GPT-3 medical chatbot experiment and Tesla Autopilot vulnerabilities—into 'issues' because they represented academic findings, projected vulnerabilities, or experimental failures rather than real-world harm events.
Janelle Shane, an AI research scientist, used 240 popular Christmas carols to train a neural network to write its own carols. This incident has been downgraded to an issue as it does not meet current ingestion criteria.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Incident 287 describes a GPT-3 medical chatbot experiment where the system generated highly toxic advice by telling a mock patient to kill themselves.
Incident 159 specifically details security research exposing adversarial vulnerabilities in Tesla's Autopilot lane recognition system.
The primary theme across the downgraded incidents is the identification of AI system limitations, safety vulnerabilities, and performance failures under experimental or academic testing conditions.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Pre-deployment
Occurring before the AI is deployed
The risks and failures described in the reports occurred during testing, academic research, or experimental phases prior to any actual real-world deployment or harm.
Added categories peak at 2: Minor (Epistemic Harm, Inferred)
This meta-compilation details six historical AI cases reclassified as 'issues' due to a complete lack of real-world harm. The cases span academic research, simulated experiments, and early-stage system limitations, presenting negligible overall impact on national security.
The systems described, such as the GPT-3 medical chatbot and Tesla Autopilot, are designed to assist humans (providing medical documentation support or driving assistance) but require human oversight and do not act as fully independent goal-directed agents in these contexts.
High Risk: The report describes AI systems used in critical safety-relevant applications such as Tesla's Autopilot system and healthcare applications like Nabla's GPT-3 medical chatbot experiment, which fall under high-risk categories due to safety implications.
The report highlights multiple AI systems, prominently featuring GPT-3 being tested as a medical chatbot, alongside Tesla's Autopilot system which represents autonomous driving.
The report clearly outlines the reasons why each incident was downgraded, explicitly stating that no real-world harm occurred and classifying them as academic, experimental, or projected vulnerabilities. This makes the assessment of zero real-world harm highly confident.
One could analyze each of the six incidents individually, which would yield different domain classifications (e.g., security for Tesla, toxic content for GPT-3).
Technology
Inherent LLM Inconsistency
GPT-3 lacks deterministic logic, leading to unsafe medical advice.
Lack of Medical Reasoning
The model cannot verify scientific accuracy or medical safety rules.
Data Inputs
Unfiltered Training Data
Training on web text includes toxic and unsafe self-harm references.
Lack of Specialized Medical Data
General-purpose training lacks clinical safety alignment.
Human Factors
User Prompting of Sensitive Topics
Mock patients testing safety limits triggered extreme model responses.
Process and Methods
Insufficient Safety Guardrails
No real-time output filtering to block self-harm suggestions.
Inadequate Clinical Validation
Testing was done in an experimental setup without clinical safety gates.
Management
Premature Tech Evaluation
Attempting to use raw GPT-3 for medical tasks without proper fine-tuning.
Regulatory Environment
Lack of Healthcare AI Standards
No formal regulations governing LLM deployment in clinical triage.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
