This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
In the summer of 2017, a Tesla sedan operating on Autopilot failed to maintain its lane, striking a curb in Redmond, Washington. The incident resulted in shredded tires and rear suspension damage to the vehicle. The driver, Microsoft AI researcher Eric Horvitz, was unharmed but used the incident to advocate for more robust safety reporting and ethical oversight in AI deployment.
A Tesla Sedan operating on Autopilot mode was not able to center itself on the road and drove over a yellow dividing curb in Redmond, Washington, causing minor damage to the vehicle’s rear suspension.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The report highlights a lack of regulatory reporting frameworks for AI safety incidents, comparing it to the lack of an FDA-like reporting system for autonomous vehicles.
The incident represents a physical failure of the Tesla Autopilot's lane-keeping capability, where the system failed to center the vehicle properly on a curving road, leading to a collision with a curb.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by the Tesla Autopilot system failing to properly center the vehicle post-deployment, which was an unintentional failure of the system's driving capabilities.
A minor 2017 traffic incident in Washington where a Tesla on Autopilot struck a curb, causing vehicle damage but no injuries. The event holds negligible national security implications and served primarily as a catalyst for advocacy on autonomous vehicle safety reporting standards.
The report describes the Tesla sedan operating on Autopilot, where the car steered itself down a curving road near Microsoft's campus. This represents an autonomous mode of operation where the system responds to environmental inputs and makes real-time steering decisions without explicit human planning.
High Risk: The system is used in critical infrastructure and transport (autonomous driving). The report describes a safety-critical system where failure can lead to physical accidents, stating 'Both driver's side tires of the vehicle nicked a raised yellow curb marking the center line, and shredded.'
The system's primary function is to steer and control the vehicle autonomously along roads, which is the definition of Autonomous Driving.
The primary incident is described consistently across two independent reports with specific details regarding the driver, location, vehicle behavior, and physical outcomes. The role of the AI system (Tesla Autopilot) is explicitly stated, and there are no conflicting accounts of the crash itself.
One could argue the incident was caused by human driver inattentiveness since Tesla Autopilot requires a fully attentive driver, but the physical failure of the lane-keeping algorithm itself is the clear technical trigger.
Technology
Flawed Centering Algorithm
The Autopilot algorithm failed to center the vehicle properly on a curve.
Inadequate Edge Case Handling
The system failed to handle a curving road with a raised yellow curb.
Data Inputs
Inadequate Real-Time Data
Vehicle sensors failed to detect or correctly process the yellow curb.
Human Factors
Driver Distraction
Driver was focused on a phone call about AI ethics instead of the road.
Overreliance on Autopilot
Driver trusted the system to steer along a curving road without intervention.
Process and Methods
Poor Feedback Mechanisms
Tesla lacked a formal system for users to report safety-critical AI bugs.
Inadequate Risk Assessment
Deploying Autopilot on curving roads without robust safety guardrails.
Management
Focus on Liability over Safety
Tesla prioritized establishing liability limits over resolving safety issues.
Regulatory Environment
Lack of AI Reporting Standards
No FDA-like federal reporting mandate existed for autonomous vehicle issues.
Weak External Governance
Absence of national regulations or standards governing AI safety limits.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
