This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Amazon Flex drivers reported being unfairly terminated by automated performance evaluation algorithms that failed to account for external factors beyond the drivers' control, such as locked apartment complexes or vehicle malfunctions. The system, which operates with minimal human oversight, frequently issues automated termination notices without providing drivers a meaningful opportunity to appeal or defend their performance. Former employees noted that Amazon management was aware of the system's flaws but prioritized cost-cutting over manual investigation of wrongful terminations.
Amazon Flex's contract delivery drivers were dismissed using a minimally human-interfered automated employee performance evaluation based on indicators impacted by out-of-driver's-control factors and without having a chance to defend against or appeal the decision.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Delegating by humans of key decisions to AI systems, or AI systems that make decisions that diminish human control and autonomy, potentially leading to humans feeling disempowered, losing the ability to shape a fulfilling life trajectory, or becoming cognitively enfeebled.
Also classified as
The automated management system degraded the quality of employment for gig workers, creating an exploitative dependency with no human support.
The algorithms failed to perform reliably under varying real-world conditions like snow, locked gates, and flat tires, leading to erroneous performance drops.
The incident represents a severe loss of human agency as Amazon delegated critical employment decisions (firing) entirely to AI systems, leaving workers with no meaningful human recourse or appeal.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The wrongful terminations were directly caused by the automated decisions of the Amazon Flex algorithms after they were deployed.
Added categories peak at 2: Minor (Psychological Harm, Inferred)
An automated HR and performance evaluation system used by Amazon to manage Flex drivers led to unfair terminations due to algorithmic rigidity. While causing financial and psychological distress to individual gig workers, the incident remains a corporate labor issue with negligible implications for national security.
The Amazon Flex algorithms operate independently to track, rate, and terminate drivers based on real-time data inputs (GPS, delivery times, selfies) with minimal to no human intervention.
High Risk: The system is an employment and worker management system used to monitor, rate, and terminate contract workers, which falls under Risk Level 2.
The system's core function is to track and evaluate driver performance metrics, automatically administering ratings and deactivations.
The reports provide detailed, consistent accounts of Amazon's automated HR system, including interviews with multiple affected drivers, former managers, and engineers. The mechanics of the algorithmic rating and termination process are well-documented.
Amazon argues that the driver complaints are anecdotal and do not represent the experience of the vast majority of drivers, suggesting the system is highly accurate overall.
Technology
Flawed Facial Recognition
Image algorithms failed to verify identity due to low light or hair changes.
Inflexible Rating Algorithms
Algorithms penalized drivers for delays caused by locked gates and snow.
Automated Termination Emails
Systems automatically deactivated drivers without human verification of details.
Data Inputs
Incomplete Environmental Data
GPS and routing systems failed to account for snow or locked apartment gates.
Low Quality Selfie Images
Poor lighting during nighttime shifts led to failed identity verification.
Unfiltered Performance Metrics
System processed package returns due to flat tires as driver failures.
Human Factors
Overreliance on Automation
Executives trusted algorithmic speed and cost-reduction over human oversight.
Lack of Driver Context Input
Drivers could not input real-world obstacles directly into the system.
Process and Methods
Automated Support Loops
Appeals were processed by automated bots, offering generic, unhelpful replies.
Hasty Human Reviews
Support staff performed rapid reviews to meet their own performance targets.
Costly Arbitration Process
A 200 dollar arbitration fee deterred drivers from appealing terminations.
Management
Prioritizing Cost over Safety
Management decided it was cheaper to trust faulty algorithms than pay humans.
Acceptance of System Errors
Executives accepted collateral damage and errors due to high driver supply.
Understaffed Support Centers
Dozens of untrained seasonal workers monitored millions of drivers.
Regulatory Environment
Lack of Algorithmic Regulation
No active laws forced companies to be transparent about algorithmic decisions.
Failure of Fairness Legislation
Proposals like the Algorithmic Fairness Act failed to pass in congress.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
