This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
On March 1, 2019, a Tesla Model 3 operating with Autopilot engaged collided with a tractor-trailer in Delray Beach, Florida, resulting in the death of the driver. The NTSB investigation concluded that the crash was caused by the truck driver's failure to yield and the Tesla driver's inattention due to overreliance on the automation system. The report further noted that the Autopilot system failed to detect the truck, did not warn the driver to take control, and operated outside its intended design conditions.
A Tesla Model 3 driver switched on Autopilot seconds before the crash into the underbelly of a tractor-trailer on a highway in Florida, killing the Tesla driver.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
Also classified as
The Autopilot system failed to detect the crossing truck and did not initiate automatic emergency braking due to design limitations.
The driver suffered from inattention and disengagement due to overreliance on the Level 2 Autopilot system, which was operated outside its intended design conditions.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by the Autopilot system failing to detect the crossing truck and apply brakes during real-world operation after deployment.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
A fatal traffic accident involving a Tesla Model 3 on Autopilot. The incident is a consumer safety and regulatory issue with negligible national security implications, as it does not target critical infrastructure, government functions, or strategic national assets.
The Autopilot system dynamically controlled steering, braking, and acceleration in real-time based on environmental inputs, operating independently within its lane prior to the collision.
High Risk: The system is an AI safety component used in road traffic control and vehicle automation, which has significant implications for physical safety and public interest.
The primary goal of Tesla's Autopilot is to automate steering, braking, and speed regulation to assist or perform driving tasks on public roads.
The incident is thoroughly documented by official NTSB investigation reports and court proceedings. The technical details of the Autopilot engagement, vehicle speed, and driver actions are based on physical telemetry and are highly reliable.
The incident could be viewed purely as a human driver error (both the truck driver and Tesla driver), but the documented failure of the Autopilot's sensors and the systemic issue of overreliance make it a clear AI safety failure.
Technology
System Design Limitations
The collision avoidance system did not warn or initiate AEB for the truck.
Inadequate Driver Monitoring
System failed to send visual or audible warnings to put hands on the wheel.
Lack of ODD Restrictions
Autopilot was not restricted from operating outside its design environment.
Data Inputs
Sensor Detection Failure
Tesla sensors failed to detect the crossing semitrailer in time to brake.
Inaccurate Hands-on Sensing
Torque sensors failed to accurately detect driver hand presence on the wheel.
Human Factors
Driver Overreliance on Automation
Driver was disengaged, expecting the system to handle all hazards.
Driver Inattention
The driver did not apply brakes or take evasive actions to avoid the truck.
False Sense of Security
Marketing names like 'Autopilot' led to unrealistic safety expectations.
Process and Methods
Unsafe System Safeguards
Tesla did not incorporate safeguards to limit Autopilot to its ODD.
Failure to Adopt Recommendations
Tesla failed to implement NTSB recommendations on driver engagement.
Management
Prioritizing Sales Over Safety
Company pushed advanced marketing while ignoring known safety limitations.
Ignoring Safety Feedback
Management rejected NTSB recommendations to improve driver monitoring.
Regulatory Environment
Lack of NHTSA Verification Method
Regulators lacked methods to verify safety safeguards for Level 2 systems.
Inadequate Oversight on ODD
No regulatory rules prevented automation use outside design conditions.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
