This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
This report details two separate incidents involving Waymo autonomous vehicles in San Francisco. In the first, passengers felt trapped when an individual obstructed the vehicle's sensors, prompting Waymo support to advise them to remain inside. In the second, a Waymo vehicle collided with a cyclist who was occluded by a truck, resulting in minor injuries. Experts note that while the technology is advancing, it has not yet reached human-level safety.
During a nighttime Waymo ride in San Francisco, an individual reportedly blocked the sensors of an autonomous vehicle, leaving passengers Robert Moreno and his husband feeling trapped and unsure how to proceed. Waymo's support team reportedly advised the riders to stay inside the vehicle for safety. The individual reportedly left after a few minutes without further escalation.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
The Waymo vehicle failed to avoid a collision with a cyclist due to occlusion and stopped completely when its sensors were blocked, demonstrating a lack of capability or robustness in complex urban environments.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incidents occurred post-deployment due to the unintentional actions and decisions of the Waymo autonomous driving system during real-world operations.
Two localized incidents involving Waymo autonomous vehicles in San Francisco highlight ongoing safety and robustness challenges in fully autonomous driving systems. While demonstrating physical safety risks to road users and passenger distress, the national security impact remains negligible to minor, with no systemic threat to critical infrastructure or state sovereignty.
The Waymo vehicle operates independently in real-time, responding to environmental inputs (like braking for a cyclist or stopping when sensors are blocked) using learned patterns without explicit human intervention or high-level planning.
High Risk: The system is an autonomous vehicle operating in critical transport infrastructure. According to the definitions, AI in critical infrastructure (such as traffic management or autonomous transport) is classified as High Risk.
The Waymo vehicle's primary function is to autonomously navigate and drive passengers to their destinations without a human driver.
The report provides clear, firsthand accounts of two distinct incidents involving Waymo vehicles, including statements from the passengers, the company, and an academic expert. The details of the events are consistent and sufficient to classify the risks and harms.
The sensor-blocking incident could be interpreted primarily as a human harassment/vandalism issue rather than an AI safety failure, though the vehicle's resulting state of trapping passengers represents an AI safety concern.
Technology
Sensor Occlusion by Large Vehicles
The AV could not detect the cyclist hidden behind a large truck.
Vulnerability to Sensor Blocking
Pedestrian covering sensors blinded the vehicle, causing it to stop.
Lack of Human-Like Anticipation
System failed to assume hidden hazards behind moving trucks.
Data Inputs
Incomplete Environmental Data
Occluded line of sight prevented real-time tracking of the cyclist.
Loss of Sensor Perception
Physical obstruction of sensors cut off critical environmental inputs.
Human Factors
Passenger Panic and Helplessness
Riders felt trapped and lacked control when the vehicle was immobilized.
Unpredictable Pedestrian Behavior
Pedestrian intentionally blocked sensors, exploiting AV vulnerability.
Cyclist Riding Behind Truck
Cyclist crossed path quickly while occluded by a larger vehicle.
Process and Methods
Defensive Driving Limitations
The AV's driving logic did not proactively stop for potential occlusion.
Inadequate Emergency Guidance
Passengers lacked clear, immediate instructions during a sensor attack.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
