This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A 14-year-old Black teenager was denied entry to a roller skating rink in Livonia, Michigan, after the facility's facial recognition system incorrectly matched her to a previously banned individual. The rink's management claimed the system reported a 97 percent match, leading to the teenager being ejected from the premises. The incident highlights ongoing concerns regarding the accuracy of facial recognition technology, particularly its higher error rates for people of color.
A Black teenager living in Livonia, Michigan was incorrectly stopped from entering a roller skating rink after its facial-recognition cameras misidentified her as another person who had been previously banned for starting a skirmish with other skaters.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
The skating rink staff blindly trusted the AI's 97 percent match output to eject a minor without proper manual verification.
The facial recognition system failed to accurately identify a Black teenager, reflecting documented demographic disparities where such algorithms perform worse on darker skin tones and women.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by a false match decision made by the deployed facial recognition AI system, which was an unexpected outcome of its operation.
A localized commercial incident where a 14-year-old Black teenager was falsely identified and ejected from a skating rink due to facial recognition bias. While it highlights ongoing civil rights concerns regarding algorithmic bias, it poses negligible threat to national security.
The facial recognition system operates as a tool that performs a specific, well-defined function of scanning faces and matching them against a database, producing a match percentage for human operators to act upon.
Risk Level 2: High Risk. The system is a biometric facial recognition system used for identification and access control, which has significant implications for fundamental rights and public interests.
The AI system's primary function was to scan faces and match them against a database of banned individuals to identify them.
The reports provide clear, consistent details about the specific incident involving Lamya Robinson, the technology's failure, and the broader context of demographic bias in facial recognition.
None, the incident is a clear-cut case of AI misidentification leading to wrongful denial of service.
Technology
Demographic Accuracy Bias
Algorithms have higher false match rates for Black women than other groups.
High False Match Rates
Systems match different Black women's faces at rates up to ten times higher.
Physical and Makeup Variances
Algorithm performance varies due to physical differences and makeup use.
Data Inputs
Biased Training Datasets
Algorithms are trained on datasets lacking sufficient demographic diversity.
Unoptimized Database Records
Banned list database relied on static photos prone to misidentification.
Low Quality Reference Photos
The database of banned individuals contained low-quality thumbnail photos.
Human Factors
Automation Bias
Staff blindly trusted the 97 percent match score over visual differences.
Lack of Staff Verification
Employees failed to manually check the thumbnail photos before banning her.
Overreliance on Technology
Staff assumed the software was infallible and skipped visual verification.
Process and Methods
Flawed Verification Process
Rink relied on automatic matches rather than human visual verification.
Inadequate Dispute Resolution
Manager asked mother to call during the week instead of resolving it onsite.
Ineffective Screening Workflow
The screening setup did not allow real-time manual review during busy hours.
Management
Inadequate Staff Training
Management failed to train staff on the limitations of AI match scores.
Lack of Safeguard Policies
No fallback procedures existed to prevent wrongful ejections of minors.
Prioritizing Speed Over Safety
Management prioritized quick throughput over accurate identity verification.
Regulatory Environment
Absence of Federal Regulation
No federal-level laws exist to govern commercial facial recognition use.
Lax State and Local Rules
Michigan lacks strict restrictions on private use of facial recognition.
Lack of Industry Standards
No mandatory accuracy standards exist for commercial biometric systems.
