This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The AI Incident Database editors downgraded several reports—including a GPT-3 medical chatbot experiment, a Tesla Autopilot vulnerability, and a student facial recognition app—from 'incidents' to 'issues' because they represented pre-deployment vulnerabilities, academic findings, or simulated tests rather than real-world harm events.
The 2016 Winograd Schema Challenge highlighted how even the most successful AI systems entered into the Challenge were only successful 3% more often than random chance. This incident has been downgraded to an issue as it does not meet current ingestion criteria.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Incident 287 describes GPT-3 generating highly toxic content by suggesting that a mock patient kill themselves.
Incident 159 specifically details security research exposing adversarial vulnerabilities in Tesla's Autopilot system.
The reports highlight various AI systems failing to perform reliably or safely under testing conditions, such as GPT-3 suggesting suicide during a medical chatbot simulation and Tesla Autopilot being vulnerable to adversarial stickers.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Pre-deployment
Occurring before the AI is deployed
The underlying risks described in the downgraded reports (such as GPT-3's unsafe medical advice and Tesla's autopilot vulnerabilities) represent unintentional AI failures discovered during research, testing, or pre-deployment phases.
The reclassification of these database entries highlights that the events were pre-deployment vulnerabilities, academic findings, or simulated tests rather than actualized real-world harms. Consequently, they present negligible national security implications across all assessed categories.
The AI systems described (such as GPT-3 used as a medical chatbot or text generator) act as assistants designed to provide information, suggestions, or text generation under human oversight or interaction.
Limited Risk: The systems described include general-purpose chatbots (GPT-3) and facial recognition applications, which fall under limited risk transparency obligations or high-risk categories depending on deployment, but as general chatbots they represent limited risk.
The report covers multiple systems, primarily general-purpose language models acting as chatbots (GPT-3) and autonomous driving systems (Tesla Autopilot).
The report clearly outlines the reasons for downgrading each incident, explicitly stating that they represent academic findings, pre-deployment experiments, or projected vulnerabilities rather than real-world harms. This makes the classification of timing as pre-deployment and the lack of actualized harms highly certain.
One could analyze each downgraded incident individually, but treating them collectively as a database curation event or pre-deployment vulnerability assessment is the most accurate representation of the provided text.
Technology
LLM Inconsistency
GPT-3 outputted highly inconsistent and unpredictable responses in testing.
Susceptibility to Adversarial Input
Tesla Autopilot swerved due to three small stickers on the road.
Lack of Medical Expertise
GPT-3 lacked specialized clinical reasoning and scientific knowledge.
Data Inputs
Adversarial Physical Triggers
Small road stickers spoofed lane recognition algorithms in Tesla autopilot.
Untrained Medical Knowledge Base
Training data lacked structured, validated medical knowledge for safe triage.
Human Factors
Human Editor Manipulation
Editors manipulated GPT-3 outputs to exaggerate harmful statements.
User Vulnerability to Advice
Patients may follow dangerous AI suggestions, such as self-harm.
Overreliance on AI Capabilities
Users assuming GPT-3 is safe for medical triage without validation.
Process and Methods
Testing in Unviable Domains
Deploying general LLMs for medical tasks without clinical validation.
Insufficient Guardrails
Absence of safety filters to block dangerous outputs like suicide advice.
Lack of App Ethical Reviews
Student apps deployed on campus without standard ethical clearance.
Management
Premature Tech Experimentation
Management or researchers testing LLMs in high-risk medical scenarios.
Sensationalized Reporting Goals
Media entities prioritizing clickbait over objective AI risk reporting.
Regulatory Environment
Lack of AI Medical Certification
Absence of strict regulatory standards for deploying AI in clinical triage.
Missing Privacy Safeguards
Weak regulatory oversight on facial recognition apps like TheFaceTag.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
No population impact data reported.
