This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A Flock Safety license plate recognition camera in Rio Arriba County, New Mexico, misread a license plate digit, leading police to conduct an armed, high-stress traffic stop and mistakenly handcuff two innocent sisters.
In July 2023, Española police stopped Jaclynn Gonzales and her 12-year-old sister after a Flock Safety license-plate reader reportedly read the final digit of Gonzales's plate as 7 rather than 2, matching a stolen plate. Body-camera reporting showed officers drew guns and handcuffed both before confirming the mismatch. The sisters later sued; Española police said officers followed procedure.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Police officers overrelied on the automated camera alert, immediately conducting high-risk, armed traffic stops and handcuffing minors without independent verification of the vehicle details.
The primary failure was the AI system's lack of capability or robustness in accurately reading the license plate digit under real-world conditions, misidentifying a '2' as a '7'. Boundary rule B2 applies because the system performed below reasonable expectations (a direct reading error), making the system the locus of failure.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by a deployed Flock Safety license plate recognition camera system misreading a digit, which was an unexpected and unintentional failure of the system's computer vision capabilities.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
A commercial license plate recognition system misidentified a digit, leading local police to mistakenly detain innocent civilians at gunpoint. While raising significant civil liberties and local public safety concerns, the incident has negligible national security implications.
The Flock Safety camera operates as a tool that performs a specific, well-defined function of reading license plates and matching them against a database, producing outputs based on direct visual inputs without autonomous planning or context retention.
High Risk: The AI system is used in a law enforcement application (license plate recognition and traffic surveillance), which has significant implications for public safety and fundamental civil rights.
The primary function of the Flock Safety camera system is to read and recognize vehicle license plates to flag stolen vehicles or suspects.
The report provides clear, factual details of two specific incidents in Espanola, New Mexico, naming the victims, the law enforcement agencies, the specific technology vendor (Flock Safety), and the exact nature of the system error (reading a 2 as a 7).
None
Technology
OCR Character Misclassification
The Flock Safety camera incorrectly read the license plate digit 2 as a 7.
Algorithmic Flaw in Plate Matching
The automated system flagged a vehicle with a different plate digit as stolen.
Data Inputs
Incorrect License Plate Reading
The system captured and processed incorrect alphanumeric data from the plate.
Inaccurate Alert Database Entry
An officer entered faulty vehicle details into the alert system manually.
Human Factors
Blind Reliance on Camera Alerts
Officers acted aggressively on the automated alert without verifying the plate.
Failure to Verify Vehicle Details
Officer ignored differences between the suspect vehicle and the stopped vehicle.
Officer Misinterpretation of Video
An officer used video of a non-matching truck to create a police-wide alert.
Process and Methods
Lack of Manual Plate Verification
No process required officers to double-check the physical plate before a stop.
Flawed Alert Dissemination Process
System allowed faulty alert details to be broadcasted to other agencies.
Over-aggressive Stop Protocols
Standard operating procedures led to immediate handcuffing of minors.
Management
Inadequate Officer Training
Police management failed to train officers on the limitations of ALPR tech.
Lack of System Oversight
Management did not monitor or audit camera errors and subsequent bad stops.
Regulatory Environment
Lack of ALPR Operational Standards
No regulatory standards governing how police verify automated camera alerts.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
