This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
In December 2016, Uber deployed autonomous vehicles in San Francisco without required permits. The vehicles were captured on video running red lights. While Uber initially blamed human error and suspended safety drivers, internal documents and employee reports later confirmed that the autonomous mapping system failed to recognize traffic signals, leading to multiple safety violations.
Uber vehicles equipped with technology allowing for autonomous driving running red lights in San Francisco street testing.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Uber deployed the autonomous vehicles on public roads without obtaining the required DMV testing permits and in defiance of regulatory demands.
The incident was primarily caused by the AI system failing to perform reliably under normal driving conditions, specifically failing to recognize traffic lights.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The safety risk was caused by the autonomous driving system failing to recognize traffic lights after being deployed on public roads.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Uber deployed unauthorized autonomous vehicles in San Francisco in 2016, which committed safety violations such as running red lights due to mapping failures. While representing a challenge to local regulatory authority and public safety, the incident has negligible national security implications and is managed through standard regulatory and law enforcement procedures.
The self-driving cars operated independently in real-time, responding to environmental inputs using learned patterns without human intervention during the red-light incidents.
High Risk: The system is used in critical infrastructure (traffic management/road transport). The reports describe self-driving cars operating autonomously on public roads, which directly impacts public safety and transport infrastructure.
The AI system was designed to autonomously pilot vehicles on public roads, navigating traffic and obeying traffic signals.
Multiple independent news outlets confirmed the details of the incident, backed by internal Uber documents and employee statements that contradicted Ubers initial public claims.
Uber initially claimed human error was to blame, but this was later debunked by internal documents.
Technology
Mapping Program Failure
Mapping program failed to recognize six traffic lights in San Francisco.
Inability to Navigate Bike Lanes
Autonomous system failed to safely merge and navigate near bike lanes.
Data Inputs
Inaccurate Map Data
The system relied on maps that did not properly identify traffic signals.
Lack of Sensor Integration
Sensor data failed to override incorrect mapping data for red lights.
Human Factors
Safety Driver Inaction
The backup driver failed to intervene and apply brakes to prevent violations.
Process and Methods
Skipping Permit Requirements
Uber bypassed DMV permit processes, avoiding mandatory safety reporting.
Inadequate System Testing
Deploying autonomous vehicles on public streets with unresolved mapping bugs.
Management
Prioritizing Speed Over Safety
Management deployed vehicles without permits to accelerate public rollouts.
Deceptive Public Communication
Management falsely blamed human error instead of admitting system failure.
Regulatory Environment
Lack of Strict Local Enforcement
Initial lack of immediate enforcement allowed unpermitted testing on streets.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
