This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
On May 7, 2016, a Tesla Model S operating with its Autopilot system engaged collided with a tractor-trailer in Williston, Florida. The vehicle's sensors failed to distinguish the white side of the trailer against a bright sky, and the system did not apply the brakes. The collision resulted in the death of the driver, Joshua D. Brown. This incident was the first known fatality involving a vehicle in self-driving mode and triggered a federal investigation into the design and performance of the Autopilot system.
A Tesla Model S on autopilot crashed into a white articulated tractor-trailer on Highway US 27A in Williston, Florida, killing the driver.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The driver exhibited extreme overreliance on the system, keeping his hands off the wheel for nearly the entire trip and ignoring multiple system warnings.
The Autopilot system failed to perform reliably under varying environmental conditions, specifically failing to distinguish a white truck against a bright sky.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by the failure of the deployed Tesla Autopilot system to detect the tractor-trailer and apply the brakes, leading to an unexpected fatal collision.
The first reported fatal accident involving a semi-autonomous vehicle (Tesla Autopilot) in May 2016. While a significant milestone in AI safety and regulatory history, the incident was an unintentional consumer product failure with negligible direct implications for national security.
The Tesla Autopilot system operates as an autonomous driving assist feature, controlling steering, speed, and lane-keeping in real-time based on sensor inputs without high-level strategic planning.
High Risk: The Autopilot system is a safety component of a passenger vehicle, directly impacting road safety and human life, which classifies it as a High Risk AI system under the EU AI Act.
The primary function of Tesla's Autopilot is to control the vehicle's steering, speed, and braking autonomously on highways, acting as an advanced driver assistance system.
The reports provide extensive, detailed, and consistent factual information regarding the crash, the technical failure of the sensors, the regulatory investigations, and the driver's behavior. Conflicting witness statements regarding the DVD player are clearly documented and resolved by official NTSB findings.
The crash could be interpreted primarily as human error due to driver inattention and overreliance, rather than a pure AI system failure.
Technology
Sensor fusion failure
Radar and camera failed to detect perpendicular white tractor-trailer.
Radar false positive filtering
System ignored stationary radar returns to avoid braking at overpasses.
Perpendicular vehicle detection gap
Vision algorithms were not trained to detect perpendicular trucks.
Data Inputs
Low contrast lighting
Bright sky and white trailer created low contrast, blinding the camera.
Perpendicular vehicle signature
Perpendicular orientation is rare and poorly represented in training data.
Human Factors
Driver complacency
Overreliance on Autopilot led driver to watch a movie or disengage.
Lack of hands-on-wheel compliance
Driver ignored multiple hands-on-wheel warnings during the trip.
Speeding behavior
Set cruise control to 74 mph in a 65 mph zone, reducing reaction time.
Process and Methods
Inadequate driver monitoring
Torque-based steering wheel detection failed to ensure active driver attention.
Lack of public beta safety gates
Releasing autopilot as public beta shifted safety testing to consumers.
Opaque safety data sharing
Proprietary code and lack of public exposure data hindered safety analysis.
Management
Overhyped capabilities
Marketing and tweets by CEO fostered driver overconfidence in autopilot.
Shift of safety responsibility
Corporate policy placed ultimate liability on the driver despite automation.
Failure to resolve edge cases
Management allowed deployment without resolving known perpendicular truck risks.
Regulatory Environment
Lack of autonomous standards
Absence of clear federal rules allowed Level 2 systems to be used hands-free.
Insufficient oversight of beta tests
Regulators did not restrict the deployment of unproven beta autopilot systems.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
