This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
On December 11, 2023, two Waymo autonomous vehicles in Phoenix, Arizona, collided with the same pickup truck being towed in an unusual, backwards-facing orientation. Waymo determined that its software incorrectly predicted the motion of the towed vehicle due to the orientation mismatch. The company subsequently issued a voluntary recall to deploy a software update across its fleet to address the issue.
Two Waymo autonomous vehicles reportedly hit the same tow truck under unusual towing conditions due to a software misinterpretation in Phoenix, Arizona. Waymo reportedly issued a software recall and updated its fleet to prevent future such incidents.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
The Waymo Driver failed to perform robustly when encountering an unusual towing configuration, leading to incorrect motion prediction and subsequent collisions.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The collisions were caused by the Waymo autonomous vehicle's software incorrectly predicting the motion of a towed vehicle post-deployment, which was an unintentional outcome of its driving operations.
Two Waymo autonomous vehicles in Phoenix, Arizona, collided with the same improperly towed pickup truck due to a software error in predicting the vehicle's motion. The incident resulted in only minor property damage and no injuries. Waymo successfully deployed a software patch to resolve the issue, representing a standard commercial safety event with negligible national security implications.
The Waymo vehicles operated independently in real-time traffic, responding to environmental inputs using learned patterns to navigate without explicit human planning or direct control during the driving task.
High Risk: The AI system is used in critical infrastructure (road traffic and transport), where failures can have significant implications for public safety.
The primary function of the Waymo Driver is to operate self-driving vehicles autonomously on public roads.
The reports provide consistent, detailed accounts of the incident from both the developer (Waymo) and independent media outlets, clearly explaining the technical failure, the sequence of events, and the subsequent mitigation steps.
None
Technology
Incorrect Motion Prediction
The AI software wrongly predicted how the towed vehicle would move.
Sensor Input Misinterpretation
Cameras and sensors failed to correctly interpret the angled towed truck.
Lack of General World Model
The system lacked a conceptual understanding of a backwards-towed vehicle.
Data Inputs
Unfamiliar Towing Configuration
Training data lacked scenarios with angled, backwards-facing towed trucks.
Out-of-Distribution Scenario
The persistent orientation mismatch was outside standard training inputs.
Human Factors
Lack of Human Fallback
No human operator was present to override the AI system decision.
Process and Methods
Inadequate Edge Case Handling
System struggled to handle novel, complex, and dynamic road scenarios.
No Real-Time Fleet Shutdown
The fleet was not halted immediately after the first collision occurred.
Management
Continuous Fleet Operation
Vehicles continued operating despite the unresolved software failure.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
