This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The ACLU conducted a test of Amazon's Rekognition facial recognition software, which resulted in 28 members of Congress being falsely matched to individuals in a mugshot database. The results showed a disproportionate error rate for people of color, raising significant concerns about the potential for law enforcement to use such biased tools to unfairly target vulnerable communities. The report highlights the risks of wrongful police encounters and calls for a moratorium on law enforcement use of face surveillance technology.
Rekognition's face comparison feature was shown by the ACLU to have misidentified members of congress, and particularly members of colors, as other people who have been arrested using a mugshot database built on publicly available arrest photos.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
The system failed to perform reliably by generating 28 false matches against a mugshot database under default settings.
The facial recognition system demonstrated unequal accuracy across demographic groups, with a disproportionately higher rate of false matches for people of color.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The false matches were generated by the deployed Amazon Rekognition AI system, representing an unintentional failure of the technology's accuracy.
The ACLU test of Amazon Rekognition exposed significant demographic bias and false-positive rates in commercial facial recognition software by falsely matching US Congress members to mugshots. While the test itself did not disrupt national security, it highlighted substantial civil rights risks and potential for biased law enforcement actions, fueling debate over domestic surveillance policy.
Amazon Rekognition operates as a tool that performs a specific, well-defined function (facial matching) on direct input (images) without context retention or autonomous goal-directed planning. The ACLU used it by uploading photos and receiving deterministic match outputs based on default settings.
High Risk: The report describes a facial recognition system, which is explicitly classified under 'Law enforcement applications, including facial recognition' as a High Risk AI system under the EU AI Act.
The AI system, Amazon Rekognition, was explicitly used to perform facial recognition and identify matches between photos of members of Congress and a mugshot database.
The report from the ACLU is highly detailed regarding the methodology of the test, the specific AI system used (Amazon Rekognition), the exact number of false matches (28), and the demographic breakdown of the errors. There is little ambiguity about the events of the test itself.
None
Technology
Inherent Algorithmic Bias
Academic research shows lower accuracy for darker-skinned faces and women.
Low Default Confidence Threshold
Using default match settings allowed false positives to be generated easily.
Data Inputs
Biased Mugshot Database
The 25,000 arrest photos database contains historical demographic imbalances.
Unstandardized Input Photos
Public photos of Congress members may differ in lighting and quality.
Human Factors
Automation Bias
Users may blindly trust facial recognition matches without verification.
Process and Methods
Lack of Rigorous Testing
Insufficient pre-deployment testing on diverse demographic groups.
Flawed Default Settings
Default settings are not optimized for high-stakes law enforcement use.
Management
Aggressive Marketing
Amazon markets face surveillance to police before addressing accuracy issues.
Neglecting Ethical Feedback
Shareholder and civil rights group warnings about harms were ignored.
Regulatory Environment
Absence of Federal Regulations
No federal laws govern or restrict the deployment of face surveillance.
Lack of Oversight
Police departments adopt the technology without public debate or oversight.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
