This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Cruise autonomous vehicles have repeatedly interfered with emergency response operations in San Francisco, including driving over active fire hoses and blocking emergency vehicles. These incidents have raised significant safety concerns among city officials and the San Francisco Fire Department, who warn that such interference could lead to catastrophic injuries or fatalities. While the companies involved emphasize their safety records and communication with officials, the frequency of these encounters has prompted calls for stricter regulation and data transparency.
A Cruise AV ran over a fire hose that was being used in an active firefighting area.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Inadequate regulatory oversight and lack of data transparency from state regulators like the CPUC and DMV hindered the management of AI safety risks.
The autonomous vehicles failed to perform reliably under complex real-world conditions, specifically active emergency scenes, demonstrating a lack of capability and robustness in critical situations.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incidents were caused by autonomous vehicles failing to navigate emergency scenes correctly after being deployed on public roads, which represents an unintentional failure of the AI system.
Fully autonomous commercial vehicles in San Francisco repeatedly interfered with emergency responders, including blocking fire trucks and driving over active hoses. While causing no casualties, these incidents highlight emerging physical security and critical infrastructure challenges as autonomous systems integrate into public spaces without robust edge-case handling.
The report describes robotaxis operating independently in real-world traffic environments, making real-time driving decisions based on sensor inputs without human intervention, which aligns with the Autonomous behavior type.
High Risk: The AI systems are deployed in critical infrastructure (traffic management and transport) and their failures directly impact emergency response services and public safety.
The primary function of the Cruise and Waymo systems is to autonomously drive passengers to their destinations in urban environments.
The reports provide clear, detailed accounts of specific incidents with dates, locations, and official statements from the San Francisco Fire Department, police union, and AV companies. The nature of the AI failures and their impacts are well-documented, leaving little ambiguity about the core events.
None
Technology
Failure to Detect Emergency Scenes
AVs failed to recognize active fire scenes and avoid emergency apparatus.
Sensor Vulnerability to Obstacles
Downed wires got tangled in rooftop sensors, immobilizing the vehicle.
Inadequate Navigation Logic
AVs blocked firehouse driveways and stopped behind responding trucks.
Data Inputs
Poor Sensor Detection of Hoses
Sensors and AI failed to identify flat fire hoses on the ground.
Human Factors
Delayed Remote Operator Action
Remote operators could not quickly move stuck vehicles out of harm's way.
Forced Physical Intervention
Firefighters had to shatter a window to stop the autonomous vehicle.
Process and Methods
Ineffective Stalled Vehicle Removal
Firefighters had to wait for human employees to arrive and move cars.
Management
Premature Commercial Deployment
Companies deployed vehicles before perfecting emergency scene interaction.
Inadequate Safety Transparency
Management relied on trade secrets to avoid sharing safety performance data.
Regulatory Environment
Shielding of Safety Information
DMV agreement allowed companies to hide crash safety data from the public.
Premature Operation Expansion
Regulators planned to allow unlimited robotaxis despite known issues.
Regulatory Conflict of Interest
A commissioner previously served as the top attorney for Cruise.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
