This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Porcha Woodruff, a pregnant woman in Detroit, was falsely arrested for robbery and carjacking after an unreliable facial recognition match incorrectly identified her as a suspect. The charges were later dropped due to insufficient evidence. This incident is one of several wrongful arrests in Detroit attributed to the misuse of facial recognition technology, leading to a lawsuit and subsequent policy changes within the Detroit Police Department.
Porcha Woodruff was arrested and subsequently had charges dropped due to an unreliable facial recognition match. Despite being visibly pregnant, she was implicated in a robbery and carjacking based on an outdated photo used in a lineup.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
Law enforcement officers over-relied on the facial recognition match as the sole basis for a photo lineup and subsequent arrest warrant.
The facial recognition system failed to perform reliably, generating an incorrect match from a low-quality video.
The facial recognition technology exhibited unequal performance, having significantly higher false positive rates for Black individuals, directly leading to the wrongful arrest of Porcha Woodruff.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was initiated by an incorrect match generated by a deployed facial recognition AI system, which was an unintentional error in its performance.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
A pregnant woman was falsely arrested in Detroit due to an incorrect facial recognition match. While representing a serious violation of civil rights and highlighting systemic algorithmic bias in municipal policing, the incident remains a localized domestic law enforcement issue with minor national security implications.
The facial recognition system operates as a tool, performing a specific, well-defined function of matching an input face against a database of images. It does not make autonomous decisions to arrest, but rather provides a static match output for human investigators to act upon.
High Risk: The report describes the use of facial recognition in law enforcement, which is explicitly classified as a high-risk application under the EU AI Act due to its significant implications for safety, fundamental rights, and public interests.
The primary function of the AI system was to perform facial recognition by matching a probe image against a database of mugshots to identify a suspect.
The reports provide highly detailed, consistent accounts of Porcha Woodruff's arrest, the role of the facial recognition technology, and the broader context of similar incidents in Detroit. The facts are well-documented through court filings and official statements, leaving little ambiguity about the sequence of events or the technology's failure.
One could argue the primary failure was human investigative error rather than AI failure, though the AI's false match was the indispensable trigger for the chain of events.
Technology
Algorithmic Bias in FRT
FRT has poor accuracy rates for Black and female individuals.
High False Positive Rates
Algorithms produce many false matches, especially on non-white faces.
Use of Low Quality Images
Grainy surveillance footage reduces accuracy and reliability of matches.
Data Inputs
Outdated Mugshot Database
System matched suspect using an eight-year-old photo instead of current ID.
Biased Training Data
Training datasets predominantly feature white faces, causing racial bias.
Scraped Internet Images
Unregulated scraping of public images leads to unverified database entries.
Human Factors
Automation Bias
Officers over-relied on the technology's suggestions without verification.
Inadequate Visual Review
Detectives failed to notice the suspect in video was not pregnant.
Eyewitness Misidentification
Victim selected Woodruff from a biased photo lineup presented by police.
Process and Methods
Flawed Photo Lineup Process
Lineups were not double-blind, leading to suggestive identification.
Lack of Corroborating Evidence
Arrests were made based solely on FRT leads without secondary evidence.
Poor Investigative Follow-up
Failed to interview other suspects or cross-reference physical traits.
Management
Inadequate Officer Training
Management failed to train officers on the limitations of facial recognition.
Over-reliance on Tech Metrics
Prioritized technology speed over rigorous investigative standards.
Lack of Oversight Audits
No regular audits of search outcomes to identify misidentifications.
Regulatory Environment
Absence of Use Limitations
Lack of legal bans or strict limits on police use of biometric surveillance.
Lack of National Standards
No standardized federal guardrails for deploying facial recognition.
Unregulated Data Scraping
No laws preventing companies from scraping public photos for databases.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
