This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
On July 7, 2016, a Knightscope K5 autonomous security robot patrolling the Stanford Shopping Center in Palo Alto, California, collided with a 16-month-old toddler. The robot knocked the child to the ground and continued moving, running over the child's foot and causing swelling and scrapes. The incident led the shopping center to temporarily suspend the use of the robots while an investigation was conducted.
On July 7, 2016, a Knightscope K5 autonomous security robot collided with a 16-month old boy while patrolling the Stanford Shopping Center in Palo Alto, CA.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
The robot failed to perform reliably under normal operating conditions by failing to detect and avoid a small child in its path.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The physical collision was caused by the autonomous navigation decisions of the deployed K5 robot, which unintentionally failed to detect and avoid the toddler.
In July 2016, an autonomous Knightscope K5 security robot collided with and minorly injured a toddler at a California shopping center. While highlighting early public safety challenges of integrating physical AI systems into crowded pedestrian environments, the incident was an unintentional localized accident with negligible national security implications.
The K5 robot operates independently on pre-programmed patrol routes, making real-time navigation decisions based on sensor inputs without human intervention or explicit planning.
High Risk: The K5 is an autonomous security robot deployed in public spaces with direct physical safety implications, which fits the definition of systems having significant implications for safety and public interests.
The K5 robot is designed to patrol shopping centers, corporate campuses, and malls to monitor the environment, detect abnormal noises or changes, and report suspicious activity to human security.
The reports provide consistent core facts about the collision, the parties involved, the location, and the resulting minor injuries. While there is a slight conflict between the mother's account and the company's account regarding the exact dynamics of the collision, the physical outcome and the failure of the robot's sensors to prevent the collision are clear.
Knightscope suggested the child ran backwards into the robot, framing it as a 'freakish accident' rather than a system failure.
None
Technology
Sensor Blind Spots
Sensors failed to detect the small stature of the toddler.
No Vibration Alert Triggered
The machine failed to register the collision via its vibration sensors.
Lack of Collision Shutdown
The motors did not fault and the robot kept moving forward after impact.
Data Inputs
Inadequate Object Classification
The small size of the toddler confused the robot's sensor fusion algorithms.
Lack of Impact Sensor Data
No physical feedback was registered to indicate an obstacle was run over.
Human Factors
Unpredictable Toddler Movement
The child ran towards the robot or backwards into its path.
No Warning Labels on Machine
The robot lacked signs warning parents to keep children away.
Attractive Friendly Appearance
The R2-D2-like design attracted the child to approach the machine.
Process and Methods
Insufficient Public Space Testing
The system was not fully validated for navigating around small children.
No External Manual Override
The parent tried to push the robot but could not manually stop it.
Management
Unaddressed Prior Incidents
A similar incident days earlier was mentioned but not investigated.
Prioritizing Cost over Safety
Deploying robots as cheap security labor before resolving navigation bugs.
Regulatory Environment
Lack of Autonomous Bot Standards
No regulations governed the safety of heavy security robots in public spaces.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
