This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The Omnilert AI security system triggered multiple false positive gun alerts at Baltimore County schools, including Parkville High and Kenwood High. These incidents resulted in police deployments, student relocations, and the unnecessary handcuffing of students. The recurring errors have prompted school officials to mandate increased human oversight for AI-generated alerts.
Police in Baltimore County responded to Parkville High School after the Omnilert AI security system reportedly detected what it believed to be a gun. Officers searched the school, and students were relocated to a supervised area during the investigation. No weapon was found, and normal activities resumed after an all-clear.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
School officials called for more human oversight because police responded directly to the AI's unverified reports, demonstrating unsafe overreliance on the system's outputs.
The incident was caused by the AI system's lack of robustness and capability, as it failed to accurately distinguish between a bag of chips or other non-threatening objects and a gun.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The false gun alerts were caused by the Omnilert AI system incorrectly identifying objects post-deployment, which was an unintentional failure of its threat detection capabilities.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
An AI threat detection system at Baltimore County schools generated multiple false positives, leading to police deployments and the temporary handcuffing of innocent students. While the incident highlights the risks of AI overreliance and its localized impact on civil liberties, it has negligible national security implications.
The report notes that Omnilert 'only flags potential threats and sends them to humans to verify,' which fits the Assistant profile of augmenting human capabilities with required human oversight.
Risk Level 2. The system is used for safety and security in an educational institution, and its outputs directly trigger law enforcement responses, which has significant implications for safety and fundamental rights.
The Omnilert system is explicitly deployed to detect potential threats, specifically guns, in school environments.
The report clearly outlines the AI system involved, the nature of the false positives, the specific schools affected, and the real-world consequences such as police response and student detention.
None
Technology
False Positive Weapon Detection
AI algorithm misidentified a bag of chips as a firearm.
Inability to Differentiate Objects
The system lacked precision to distinguish everyday items from weapons.
Data Inputs
Visual Similarity in Inputs
Visual features of a bag of chips closely resembled a gun profile.
Human Factors
Inadequate Human Verification
Humans failed to properly verify the AI threat alert before police action.
Lack of Operator Training
Operators lacked sufficient training on verifying AI-flagged alerts.
Process and Methods
Flawed Response Protocol
Police responded to unverified Omnilert reports without confirmation.
Lack of Verification Step
The verification workflow was bypassed or rushed during the incident.
Management
Deficient Training Programs
Management failed to provide annual training on Omnilert protocols.
Inadequate Risk Assessment
Lack of clear escalation procedures for AI-generated alerts.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
