This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
On March 18, 2018, an Uber autonomous vehicle in autonomous mode struck and killed a pedestrian, Elaine Herzberg, in Tempe, Arizona. The vehicle's perception system failed to correctly classify the pedestrian, and Uber had disabled the car's factory-installed automatic emergency braking system. The human safety driver was distracted by a streaming video service at the time of the collision, which was deemed entirely avoidable.
An Uber autonomous vehicle (AV) in autonomous mode struck and killed a pedestrian in Tempe, Arizona.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The safety driver exhibited automation complacency, a known effect of overreliance, by streaming a video instead of keeping eyes on the road.
Arizona officials established a low-regulation environment to attract self-driving companies, failing to provide adequate safety oversight.
The primary failure was the autonomous vehicle's perception system failing to correctly classify a pedestrian and predict her path, representing a critical lack of capability and robustness in real-world conditions.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by the Uber autonomous vehicle's perception system failing to correctly classify a pedestrian and take evasive action during public road testing. This unexpected failure occurred post-deployment during real-world operations.
Added categories peak at 2: Minor (Psychological Harm, Direct)
This incident involved a fatal collision where an Uber autonomous vehicle struck and killed a pedestrian due to perception system failures and safety driver distraction. While a significant milestone in AI safety and regulatory history, it remains a localized traffic accident with negligible national security implications.
The Uber vehicle operated independently in autonomous mode for 19 minutes prior to the crash, making real-time driving decisions based on sensor inputs and learned patterns. The report notes that the SUV had been driving autonomously, with no input from its human backup driver, calculating speeds and object trajectories.
High Risk: The system is an autonomous vehicle operating in public transport infrastructure. The report describes a 'semi-autonomous Uber vehicle' and 'fully autonomous vehicle technology' which falls under AI in critical infrastructure and safety-critical transport applications, posing significant implications for safety and public interest.
The AI system's primary function was to operate a self-driving Volvo SUV autonomously on public roads. A secondary function was the real-time identification and classification of objects in its path.
The reports provide exceptionally detailed, multi-perspective accounts of the incident, including technical details from the NTSB investigation, personal backgrounds of the operator and victim, and the regulatory context. The facts of the collision, the system's classification failures, and the operator's distraction are thoroughly documented and consistent across sources.
One could argue the primary failure was human negligence (the distracted safety driver), but the NTSB and technical reports make it clear that the AI system's perception design flaws and the disabling of the emergency braking system were critical causal factors.
Technology
Unstable Object Classification
System constantly reclassified the pedestrian, resetting path prediction.
Disabled Volvo Emergency Braking
Uber disabled Volvo's factory AEB to prevent system conflicts.
Braking Suppression Logic
System delayed hard braking to prevent erratic, false-alarm stops.
Data Inputs
Lack of Jaywalker Labeling
System was not trained to recognize pedestrians outside of crosswalks.
Confusing Sensor Data
Victim carried a bike with bags, which confused classification models.
Poor Night Lighting
The crash occurred at night on a poorly illuminated four-lane road.
Human Factors
Automation Complacency
Safety driver trusted the automation and stopped paying active attention.
Driver Distraction
Safety driver was streaming a TV show on her personal phone while driving.
Delayed Human Intervention
Driver took control only two-hundredths of a second before the impact.
Process and Methods
Transition to Solo Operators
Uber cut the second safety driver, increasing the chance of distraction.
Inadequate Driver Monitoring
Management did not perform real-time monitoring or frequent video checks.
Lack of System Transparency
Proprietary software made it difficult to audit and verify safety limits.
Management
Prioritizing Miles Over Safety
Uber pressured operators to 'crush miles' to show rapid progress.
Inadequate Safety Culture
NTSB found Uber did not make safety a top priority in its operations.
Ignoring Internal Warnings
Executives ignored manager warnings about poor training and crash risks.
Regulatory Environment
Lax State Oversight
Arizona established a regulation-free zone to attract tech companies.
Permissive Testing Approvals
State allowed testing on public roads without strict safety verification.
Weak Federal Standards
Federal regulators relied on voluntary safety reporting from companies.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
