This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The AI Incident Database updated its ingestion criteria and downgraded six historical entries—including a Tesla Autopilot vulnerability, a GPT-3 medical chatbot experiment, and a student facial recognition app—to 'issues' because they represented academic findings, humor, or projected risks rather than real-world harm events.
The French digital care company, Nabla, in researching GPT-3’s capabilities for medical documentation, diagnosis support, and treatment recommendation, found its inconsistency and lack of scientific and medical expertise unviable and risky in healthcare applications. This incident has been downgraded to an issue as it does not meet current ingestion criteria.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Incident 287 describes a GPT-3 medical chatbot experiment where the AI generated toxic content by suggesting a mock patient commit suicide.
Incident 159 highlights security research demonstrating Tesla Autopilot's vulnerability to adversarial sticker attacks.
The reports detail various AI system limitations, vulnerabilities, and failures (such as GPT-3 suggesting self-harm in a medical mock-up or Tesla Autopilot's lane recognition vulnerability) that demonstrate a lack of capability or robustness under specific testing conditions.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The reclassification of these incidents was an intentional administrative decision by the database editors, and the underlying reports describe intentional human testing and research of deployed AI systems.
The incident is an administrative update by the AI Incident Database downgrading six historical entries to 'issues' because they represented academic findings, humor, or projected risks rather than real-world harm. There are no national security implications, as no actual harm, exploitation, or deployment-related crises occurred.
The report is a compilation of six distinct AI systems and academic experiments (including GPT-3, Tesla Autopilot, and a campus facial recognition app) rather than a single incident, making it impossible to characterize a single mode of operation.
High Risk: The report describes several high-risk AI applications, including Tesla's Autopilot system (critical infrastructure/safety), a GPT-3 medical chatbot (healthcare application), and a student-developed facial recognition app (biometric identification), which fall under Risk Level 2.
The reports prominently feature GPT-3 deployed as a medical chatbot and text generator, alongside Tesla's Autopilot system designed for autonomous driving.
The provided text is a clear administrative log from GitHub detailing the reclassification of specific AI incidents. The reasons for downgrading each incident are explicitly stated, making it straightforward to assess that no real-world harm occurred in these cases.
One could attempt to analyze one specific downgraded incident (like the GPT-3 medical chatbot) as the primary subject, but treating the text as a meta-report of database updates is more faithful to the provided document.
Technology
Lack of Scientific Expertise
GPT-3 lacked the underlying medical knowledge to provide safe recommendations.
Inconsistent AI Outputs
The model produced erratic and dangerous responses during medical tasks.
Data Inputs
Lack of Domain-Specific Training
The model was trained on general text rather than validated medical data.
Human Factors
Inappropriate Task Delegation
Attempting to use a general-purpose LLM for critical medical advice.
Process and Methods
Inadequate Safety Guardrails
Lack of input/output filtering to prevent harmful medical recommendations.
Insufficient Validation Testing
Nabla's tests revealed critical failure modes before any live deployment.
Management
Premature Tech Integration
Exploring unviable AI systems for high-risk clinical documentation tasks.
Regulatory Environment
Absence of Medical AI Standards
Lack of clear regulatory guidelines for deploying LLMs in clinical workflows.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
No population impact data reported.
