This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
[name withheld] was wrongfully arrested in Oklahoma based on an incorrect facial recognition lead provided to Maryland police. She spent six months in jail for bank fraud crimes she did not commit, as she was in Oklahoma at the time of the incidents. The case highlights systemic issues with police reliance on flawed facial recognition results and the failure to conduct independent investigations.
Kimberlee Williams was reportedly arrested in Oklahoma on Maryland warrants after police allegedly relied on a facial recognition lead that incorrectly identified her as a suspect in bank fraud cases. The ACLU said Williams had never been to Maryland, police failed to investigate alibi evidence placing her in Oklahoma, and she spent about six months in jail before the charges were dropped.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
Police officers overrelied on the facial recognition lead, failing to conduct independent investigations or verify the suspect's identity.
The AI system failed to perform reliably, generating a false match that misidentified an innocent person as a suspect.
The facial recognition technology exhibits unequal performance across demographic groups, producing higher false match rates for people of color, women, and the elderly, which directly led to disproportionate wrongful arrests.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was initiated by an incorrect match generated by a deployed facial recognition AI system, which was an unintentional failure of the technology.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
An innocent individual was wrongfully jailed for six months due to an incorrect facial recognition match by Maryland police. This incident highlights ongoing domestic civil rights and societal stability concerns regarding the overreliance on biased facial recognition technology in law enforcement, though it presents negligible direct threat to national security.
The AI system acts as an Assistant because it is designed to augment law enforcement capabilities by suggesting potential matches from a database, which explicitly requires human verification and oversight before any action is taken.
Risk Level 2. High Risk: AI systems that have significant implications for safety, fundamental rights, and public interests. Examples include law enforcement applications, including facial recognition.
The primary goal of the AI system was to perform face recognition by comparing a suspect's image against a database to identify a matching individual.
The report provides clear, detailed, and consistent accounts of the wrongful arrest of [name withheld] and lists 13 other specific cases. The role of the facial recognition technology is explicitly stated, and the resulting harms (wrongful arrest, job loss, emotional distress) are directly documented.
None. The causal link between the false AI match and the subsequent wrongful arrest is clear and undisputed.
Technology
High False Match Rates
FRT often produces false matches, especially for women and minorities.
Inherent Demographic Bias
Studies show FRT has higher error rates for people of color and women.
Data Inputs
Low-Quality Security Images
Using low-quality security camera footage led to inaccurate FRT matches.
Lack of Database Context
Matches sent via listserv lacked context on how search was conducted.
Human Factors
Confirmation Bias in Officers
Officers visually confirmed false matches due to cognitive bias from FRT.
Overreliance on Automation
Police treated unverified FRT leads as sufficient proof for arrest.
Process and Methods
Inadequate Alibi Verification
Detectives failed to verify alibis or investigate other obvious leads.
Flawed Photo Lineup Practices
Using FRT leads directly in lineups biased witnesses toward false matches.
Concealment of FRT Use
Police hid their reliance on FRT from courts during warrant applications.
Management
Failure to Enforce Policies
Management failed to enforce policies stating FRT is not grounds for arrest.
Regulatory Environment
Absence of Use Restrictions
Lack of legal bans or restrictions on police use of FRT in Maryland.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
