This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
UK police facial recognition technology was found to have significantly higher false positive rates for Black and Asian individuals, particularly Black women, compared to white subjects. Despite internal findings of bias, police forces lobbied to revert to a lower confidence threshold to maintain the number of investigative leads, raising concerns about the prioritization of operational convenience over civil rights and the lack of transparency regarding these system failures.
UK government testing of police facial recognition technology reportedly found significantly higher false positive identification rates for Black and Asian individuals compared with white subjects, with particularly elevated error rates for Black women. The findings reportedly emerged from analysis of retrospective searches of the police national database and were disclosed by the Home Office amid plans for expanded national deployment.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
Police forces lobbied to maintain a lower confidence threshold despite knowing it introduced significant demographic bias, prioritizing operational convenience over safe and equitable use.
The facial recognition system exhibited significantly different false positive rates across demographic groups, performing worst for Black and Asian individuals, particularly Black women.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The demographic bias and high false positive rates were caused by the performance limitations of the deployed AI system, which was an unintended outcome of its operational use.
UK police forces deployed facial recognition technology with significant demographic bias, particularly affecting Black and Asian individuals. Despite internal findings, forces lobbied to maintain a lower confidence threshold for operational convenience, leading to civil rights concerns and potential erosion of public trust in domestic law enforcement.
The facial recognition system acts as an Assistant by scanning images and cross-referencing them to suggest potential matches, which then require manual verification and visual assessment by trained officers before any action is taken.
High Risk: The report describes the use of facial recognition technology in law enforcement, which is explicitly listed as a high-risk application. The reports state: 'UK forces use the police national database (PND) to conduct retrospective facial recognition searches' and 'Live facial recognition... is currently used by 13 police forces'.
The primary function of the AI system is to perform facial recognition by scanning faces and cross-referencing them against a database of custody photos to identify suspects.
The reports are highly detailed, citing official research from the National Physical Laboratory (NPL) with specific statistical figures on false positive rates across different demographic groups. The accounts from the Home Office, police representatives, and civil rights groups are consistent regarding the existence of the bias and the operational decisions made around the confidence thresholds.
None. The bias and the decision to revert the threshold are clearly documented.
Technology
Biased Matching Algorithm
Algorithm shows significantly higher false positive rates for minorities.
Low Algorithm Confidence Settings
Lowering confidence thresholds to get more leads increased false positives.
Performance Decline on Women
The software performs worse on women and younger demographics.
Data Inputs
Custody Photo Database Bias
Matching against 19 million custody images introduces demographic skew.
Poor Quality Probe Images
CCTV and mobile photos used as inputs can degrade matching accuracy.
Unbalanced Training Datasets
The underlying AI model was trained on demographically unrepresentative data.
Human Factors
Preference for Quantity over Quality
Police preferred more investigative leads despite higher false positive rates.
Risk of Cognitive Bias in Review
Officers reviewing matches may trust the AI suggestion too readily.
Inadequate User Training
Officers lacked sufficient training to mitigate algorithmic bias effects.
Process and Methods
Reverting to Low Thresholds
Reversing a high threshold decision to restore operational search volume.
Inadequate Pre-Deployment Testing
Technology was deployed into operational policing without safeguards.
Delayed Public Disclosure
Known system bias was kept quiet for over a year before publication.
Management
Prioritizing Leads over Fairness
Police leadership prioritized operational volume over reducing racial bias.
Poor Communication with Communities
Management failed to share known bias issues with affected communities.
Inadequate Risk Mitigation
Failed to establish robust safeguards before wide system rollout.
Regulatory Environment
Absence of Strict National Standards
No robust national standards exist to regulate facial recognition use.
Weak Independent Scrutiny
Lack of independent oversight bodies to audit police AI systems.
Lack of Clear Legal Framework
Deployment proceeded without detailed legislative debate or restrictions.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
