This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Uber drivers in India have faced widespread account lockouts due to the 'Real-Time ID Check' facial recognition system, which uses Microsoft Azure's Face API. Drivers report that the system fails to recognize them due to minor appearance changes, poor lighting, or low-quality camera equipment, resulting in significant loss of income. Research indicates that the underlying software may have higher error rates for Indian faces, and drivers report that grievance redressal mechanisms are ineffective and opaque.
Uber drivers in India reported being locked out of their accounts allegedly due to Real-Time ID Check's facial recognition failing to recognize appearance changes or faces in low lighting conditions.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
The automated lockouts degrade gig work quality and cause sudden loss of livelihood for vulnerable platform workers.
The underlying AI system fails to perform robustly under common real-world conditions like low lighting or minor appearance changes.
The facial recognition system exhibits unequal performance and higher error rates for Indian faces and minority ethnic groups, leading to disproportionate account lockouts.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The lockouts were caused by the unexpected failure of the deployed facial recognition AI system to accurately identify drivers under varying real-world conditions.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
Widespread account lockouts of Uber drivers in India and the UK due to facial recognition failures in Microsoft Azure's Face API, which exhibits higher error rates on Indian and minority ethnic faces. This algorithmic bias caused immediate financial harm and highlighted labor rights concerns, but carries minor national security implications.
The Real-Time ID Check system automatically processes driver selfies, compares them to database records using learned facial recognition patterns, and autonomously triggers account lockouts when a mismatch is detected.
High Risk: The system is used for employment and worker management, specifically controlling access to work and livelihood, and utilizes biometric facial recognition, both of which are classified as high-risk applications under the EU AI Act.
The primary function of the AI system is to perform facial recognition by comparing real-time selfies against database photos. Secondarily, it serves as a workforce monitoring and access control tool for gig economy platforms.
The reports are highly detailed, drawing on a structured survey by MIT Technology Review, academic audits of the Microsoft Face API, specific driver case studies, and statements from both driver unions and Uber.
Uber suggests some lockouts are due to community guideline violations rather than facial recognition failures, presenting an alternative explanation for specific cases like Neradi Srikanth.
Technology
Brittle Facial Recognition Software
Microsoft Face API struggles to recognize faces with facial hair or shaved heads.
Poor Edge Detection in Low Light
The software fails to detect facial edges in low lighting or low contrast.
Failure on Diverse Demographics
The facial recognition tool has higher error rates on Indian and minority faces.
Data Inputs
Inadequate Demographic Training Data
The underlying AI model was not sufficiently trained on diverse Indian faces.
Low-Quality Selfie Submissions
Scratched phone cameras and low-budget smartphones produce low-quality inputs.
Variable Lighting Conditions
Selfies taken at dusk or night lack the contrast required for verification.
Human Factors
Natural Changes in Appearance
Drivers shaving heads, growing facial hair, or changing haircuts triggers fails.
Driver Desperation and Fatigue
Drivers working extremely long hours face high pressure to bypass lockout bugs.
Lack of Driver Tech Literacy
Drivers struggle to understand how to optimize lighting and angles for the AI.
Process and Methods
Flawed Manual Review Process
Human reviewers fail to timely override false negatives, causing long lockouts.
Ineffective Grievance Redress
Support systems are tedious, slow, and unhelpful for locked-out drivers.
Frequent Verification Prompts
Insisting on real-time ID checks before rides increases the chance of failure.
Management
Risk Shifted to Workers
Uber shifts the operational and financial risk of AI failures onto drivers.
Prioritizing Security over Usability
Implementing strict fraud checks without safeguards for false matches.
Lack of Transparency in Bans
Management fails to provide clear explanations for account deactivations.
Regulatory Environment
Lack of Gig Worker Protections
India lacks policy frameworks to protect gig workers from algorithmic errors.
Absence of Transparency Mandates
No regulations force platforms to disclose why an AI model rejected a driver.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
