This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The Detroit Police Department used facial recognition software from DataWorks Plus to identify suspects in criminal investigations. This practice led to the wrongful arrest and detention of multiple Black men, including Robert Williams and another individual, due to algorithmic errors and a lack of proper investigative due diligence. The incidents resulted in significant personal and economic harm to the victims and sparked widespread criticism regarding racial bias in facial recognition technology.
A Black man was wrongfully detained by the Detroit Police Department as a result of a false facial recognition (FRT) result.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
Police officers over-relied on the AI's output, treating a mere lead as definitive proof of identity without conducting basic due diligence.
The AI system failed to perform reliably, yielding a high rate of false matches (up to 96% according to the police chief).
The facial recognition software exhibited unequal performance across demographic groups, misidentifying Black faces at substantially higher rates than White faces, directly leading to the wrongful arrest of Black men.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The wrongful arrests were triggered by an erroneous match generated by the deployed facial recognition AI, which was an unintended failure of the system.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
The Detroit Police Department's reliance on biased facial recognition technology led to multiple wrongful arrests of Black men. While presenting no direct threat to national infrastructure, sovereignty, or economic security, the incident highlights substantial domestic concerns regarding civil rights violations, systemic algorithmic bias, and the erosion of societal trust in law enforcement.
The facial recognition software operates as a Tool. It takes a specific input (a still image) and performs a well-defined function (measuring facial points to generate a face print and searching a database for matches) without autonomous planning or context retention.
High Risk: The system is used for law enforcement applications, specifically facial recognition, which is explicitly listed as a High Risk application under the EU AI Act.
The AI system's primary function is to perform facial recognition by measuring facial features to identify individuals from a database.
The reports provide consistent, detailed accounts of the two wrongful arrests, including names, dates, the specific software used (DataWorks Plus), and the consequences. The role of the AI is explicitly stated and corroborated by police statements.
None; the causal link between the AI's false match and the wrongful arrests is clear.
Technology
High Error Rate for Black Faces
The software has a substantial error rate among Black and brown individuals.
Algorithmic Bias
Algorithms replicate racial bias programmed into them by developers.
High General Misidentification Rate
Police chief acknowledged a 96 percent misidentification rate if used alone.
Data Inputs
Low Quality Source Image
The system used a single blurry screen-grab from a cellphone video.
Biased Database Photos
Matches were queried against mugshots and driver's license databases.
Human Factors
Lack of Due Diligence
Detective failed to compare obvious physical features like tattoos.
Confirmation Bias
The witness and detective uncritically accepted the algorithmic match.
Lack of Witness Interview
Officer failed to interview a key student witness named in the video.
Process and Methods
No Written Policy
The department had no written policy governing facial recognition at the time.
Inadequate Photo Lineup Validation
An unverified algorithmic match was used directly to create a photo lineup.
Lack of Prior Prosecutor Review
Supervisor did not review the facial recognition evidence before charging.
Management
Prioritizing Tech Over Safety
Renewed controversial contracts despite known high error rates.
Lack of Investigator Training
Management did not ensure investigators followed up on algorithmic leads.
Insufficient Initial Protocols
No strict protocols were established for warrants using facial recognition.
Regulatory Environment
Absence of Local Ban
Detroit renewed its contract unlike other cities that banned the technology.
Lack of Disclosure Requirements
Police were not required to share when matches were used, hiding errors.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
