This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An automated license plate reader system developed by Flock Safety misidentified a license plate character, leading Toledo Police officers to stop an innocent driver at gunpoint and deploy a police dog that caused severe physical injuries.
Toledo, Ohio, police stopped motorist Brandon Upchurch at gunpoint after a Flock Safety license-plate reader incorrectly matched his truck to a stolen plate. Officers released a K-9 that bit him and arrested him. Police later said the alert was a misread; one officer was reprimanded for failing to verify the plate, and all charges against Upchurch were later dropped.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Police officers overrelied on the automated alert and initiated a high-risk, armed intervention without performing the mandatory visual verification of the license plate.
The primary failure was the AI system's lack of robustness in accurately reading license plate characters under real-world conditions, misidentifying a '7' as a '2' and triggering a false police response.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The physical and legal harms resulted from an unexpected character misidentification error by the Flock Safety AI system after it was deployed in the field.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
An automated license plate reader system misidentified a plate character, leading local police to detain an innocent driver at gunpoint and deploy a police dog. This incident is part of a nationwide pattern of false positive arrests and civil rights violations stemming from AI-driven mass surveillance, highlighting systemic risks to societal stability, civil liberties, and public trust in law enforcement.
The Flock Safety camera operates as a tool that performs a specific, well-defined function of scanning license plates and matching them against a database, producing deterministic alerts based on direct visual inputs without maintaining an independent planning state.
High Risk: The AI system is used in law enforcement applications to identify vehicles of interest, which has significant implications for public safety, personal liberty, and fundamental human rights.
The primary function of the Flock Safety system is automated license plate recognition (ALPR) to identify and flag vehicles of interest for law enforcement.
The reports provide highly detailed, consistent, and verified accounts of the specific incident in Toledo, including body camera transcripts, police department statements, and legal settlement details. The role of the Flock Safety camera in misidentifying the plate is explicitly documented and acknowledged by both the police department and the court.
The incident could be viewed purely as a failure of police procedure (failure to visually verify), but the underlying trigger was an algorithmic misidentification, making it a clear AI-human interaction failure.
Technology
Character Misidentification
The Flock camera misread the character '7' on the license plate for a '2'.
Database Sync Failure
Stolen status of vehicles already recovered was not updated in the system.
Unreliable LPR Performance
System accuracy is impacted by dirty plates, covers, and lighting conditions.
Data Inputs
Incorrect Camera Hit
The camera fed a false-positive stolen plate alert directly to the officers.
Obscured License Plate
Dirt or covers on license plates caused the AI model to misread characters.
Human Factors
Failure to Verify Alert
The officer did not visually verify the license plate before deploying force.
Adrenaline and Information Overload
Inundation of automated pings increased officer adrenaline and rushed actions.
Confirmation Bias
Officers assumed the suspect was fleeing instead of tripping during the stop.
Process and Methods
High-Risk Felony Stop Protocols
Officers immediately drew guns and used high-risk tactics based on an ALPR hit.
Inadequate K-9 Deployment Policy
Policy allowed canine deployment without confirming the severity of the crime.
Lack of Real-Time Tag Running
Officers failed to run the plate through NCIC manually to confirm the hit.
Management
Prioritizing Growth Over Accuracy
Vendor prioritized rapid expansion without disclosing operational error rates.
Allowing Lower Confidence Searches
Management allowed officers to lower confidence thresholds, increasing errors.
Inadequate Officer Training
Inadequate training on the limitations and verification protocols of ALPR tech.
Regulatory Environment
Lack of Independent Efficacy Data
No independent regulatory body measures or standardizes LPR accuracy rates.
Absence of Error-Rate Disclosure
No regulatory mandate forces vendors to publicize local system misread rates.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
