This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Facial recognition systems deployed by the Metropolitan Police and private retailers (Facewatch) have caused multiple incidents of mistaken identity. In one case, a shopper was wrongly accused of shoplifting and banned from stores, while in another, an anti-knife crime activist was detained by police after being incorrectly flagged as a wanted suspect. These incidents highlight concerns regarding the accuracy and civil liberty implications of live facial recognition technology in public and retail spaces.
Sometime in February 2024, Shaun Thompson is reported to have walked by one of the London Metropolitan Police's facial recognition technology vans near London Bridge. He was almost immediately arrested because the technology is reported to have misidentified him as a suspect in an unrelated and unspecified crime.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Police officers and retail staff immediately acted on the AI's alerts to detain and accuse individuals without sufficient independent verification.
The incident was primarily caused by the facial recognition systems failing to perform reliably, resulting in false positive matches and mistaken identity.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The false positive matches were caused by the AI system's algorithmic errors after it was deployed in public and retail spaces.
Added categories peak at 2: Minor (Psychological Harm, Direct)
The deployment of live facial recognition by UK police and private retailers has led to incidents of mistaken identity, resulting in wrongful detentions and privacy violations. While raising significant domestic civil liberties and human rights concerns regarding mass surveillance, the national security impact remains minor as it does not involve foreign adversaries, kinetic threats, or critical infrastructure compromise.
The facial recognition systems perform a specific, well-defined function of matching live video feeds against a database of watchlists. They operate on direct inputs without planning or context retention, producing deterministic similarity outputs for human operators.
High Risk: The AI system is used for law enforcement applications, specifically live facial recognition in public spaces, which is explicitly listed as a High Risk application under the EU AI Act.
The AI systems (Facewatch and the Met Police system) are explicitly designed to perform live facial recognition to identify individuals from watchlists.
The reports provide clear, firsthand accounts of the individuals affected by the facial recognition errors, along with official statistics from the Metropolitan Police regarding error rates (1 in 40 alerts being false positives). The roles of the AI systems (Facewatch and Met Police LFR) are explicitly stated, and the resulting harms are well-documented.
None
Technology
High False Positive Rate
One in 40 alerts was a false positive, causing wrongful detentions.
Family Resemblance Mismatch
System struggled to differentiate individuals with strong family resemblance.
Flawed Biometric Matching
Facewatch algorithm incorrectly matched innocent shopper to shoplifter.
Data Inputs
Poor Watchlist Image Quality
Low-quality database photos can lead to algorithmic mismatching.
Suboptimal Camera Angles
Roof-mounted van cameras captured images prone to distortion.
Human Factors
Automation Bias
Operators trusted algorithmic alerts without sufficient skepticism.
Immediate Accusation by Staff
Store worker accused shopper of theft based solely on system alert.
Process and Methods
Lack of Pre-Confrontation Checks
No rigorous human verification step before detaining flagged targets.
Intrusive Identity Verification
Detainees forced to provide fingerprints or passports to prove innocence.
Management
Rapid Technology Scaling
Met Police scaled deployment from 9 to 67 times without fixing error rates.
Prioritizing Speed Over Accuracy
Focusing on rapid arrests led to overlooking high false positive risks.
Regulatory Environment
Absence of Clear Legislation
Lack of specific laws governing facial recognition creates legal uncertainty.
Insufficient Oversight
No independent watchdog regulating private use of biometric databases.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
