This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An Italian court ruled that Deliveroo's 'reliability' algorithm was discriminatory because it penalized delivery riders for cancelling shifts, even when the cancellations were due to legally protected reasons like illness or striking. The algorithm's ranking system prioritized riders who did not cancel, effectively reducing work opportunities for those who had legitimate absences. The court ordered Deliveroo to pay damages to the affected workers, marking a significant legal precedent for algorithmic management in the gig economy.
In December 2020, an Italian court ruled that Deliveroo’s employee ‘reliability’ algorithm illegally discriminated against workers with legitimate reasons for cancelling shifts.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
Also classified as
The algorithm was used to micromanage gig workers, reducing their employment quality and creating exploitative dependencies.
The algorithm discriminated against riders by penalizing them for exercising legally protected rights like striking or taking sick leave, leading to unfair work distribution.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The discriminatory outcomes were caused by the automated decisions of the deployed Deliveroo ranking algorithm, which unintentionally penalized riders for legally protected absences.
An Italian court ruled that Deliveroo's shift-allocation algorithm was discriminatory because it penalized riders for taking sick leave or striking. While representing an important legal precedent for gig-worker rights and algorithmic transparency, the national security implications of this commercial labor dispute are negligible.
The algorithm operated independently to calculate a reliability index and allocate shifts based on rider cancellation data without real-time human intervention. As reported, 'the particular algorithm examined by the court allegedly was used to determine the reliability of a rider' and automatically penalized those who cancelled shifts.
High Risk: The system is an employment and worker management system used to manage and evaluate gig workers. According to the definitions, High Risk includes 'Employment and worker management systems, such as AI used in recruitment' and managing workforces.
The algorithm was designed to monitor rider cancellations, evaluate their reliability index, and automatically schedule/allocate shifts based on that evaluation.
The reports provide clear, consistent details about the court ruling, the specific mechanics of the algorithm (penalizing cancellations within 24 hours), the parties involved, and the legal outcomes. There is minimal ambiguity regarding the AI's role or the nature of the harm.
None. The court's legal ruling explicitly establishes the discriminatory nature of the algorithm under Italian labor law.
Technology
Flawed Ranking Algorithm
Algorithm did not distinguish between protected and trivial cancellations.
Opaque Reliability Index
Blackbox system calculated metrics without transparent logic.
Data Inputs
Incomplete Absence Data
System only recorded absence, ignoring reasons like sickness or strikes.
Human Factors
Lack of Worker Recourse
Couriers could not appeal automated decisions or explain absences.
Process and Methods
No Algorithmic Auditing
Absence of systematic reviews to identify indirect bias in algorithms.
Discriminatory Shift Allocation
Booking system restricted shifts based on biased reliability metrics.
Management
Prioritizing Efficiency
Management prioritized supply consistency over labor rights.
Inadequate Risk Assessment
Failed to evaluate potential legal liabilities of automated decisions.
Regulatory Environment
Evolving Gig Work Laws
Unclear regulations on algorithmic management in the self-employed sector.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
