This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
In December 2020, Stanford Medical Center implemented a rules-based algorithm to prioritize 5,000 COVID-19 vaccine doses among its staff. The algorithm failed to account for the high exposure risk of frontline medical residents and fellows, resulting in only seven residents being included in the first wave of vaccinations, while administrators and remote staff were prioritized. This led to protests by frontline workers and a public apology from hospital leadership, who acknowledged the algorithm's flaws and committed to revising the distribution plan.
In 2020, Stanford Medical Center's distribution algorithm only designated 7 of 5,000 vaccines to Medical Residents, who are frontline workers regularly exposed to COVID-19.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The algorithm's age-based scoring system systematically disadvantaged younger frontline residents relative to older, non-frontline staff.
Hospital leadership initially relied on the algorithmic output and failed to correct the distribution list even after being alerted to the errors.
The algorithm failed to perform effectively under the condition of residents rotating departments, leading to a flawed prioritization list.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by the Stanford vaccine distribution algorithm post-deployment, leading to an unintentional and unexpected deprioritization of frontline residents.
Added categories peak at 2: Minor (Psychological Harm, Direct)
In December 2020, Stanford Medicine's rules-based vaccine allocation algorithm failed to prioritize frontline medical residents, leading to localized protests and a subsequent policy revision. The incident was an internal, unintentional administrative error with negligible implications for national security.
The algorithm operated as a deterministic tool, calculating priority scores based on direct inputs (age, job variables, department) without maintaining an independent state or planning capabilities.
High Risk: The system is an algorithmic tool used in healthcare to allocate critical medical resources (vaccines), which has significant implications for the safety and health of essential workers.
The primary function of the algorithm was to allocate a limited resource (5,000 vaccine doses) among Stanford Medicine's workforce.
The reports provide consistent, detailed accounts of the algorithm's variables, the specific number of residents affected, and the organizational response. There is high consensus across multiple reputable news sources regarding the facts of the incident.
The incident could be viewed purely as a human management/ethical failure in designing the rules, rather than an AI failure, given that it was a simple rules-based formula rather than machine learning.
Technology
Flawed Rules-Based Formula
Age-based scoring and complex weights disadvantaged middle-aged residents.
Overcomplicated Algorithm
Too many variables made the system hard to understand and verify.
Data Inputs
Proxy Variables For Risk
Used department testing rates instead of actual direct patient exposure.
Rotation Data Exclusion
Rotating residents lacked a single department assignment, losing points.
Human Factors
Lack of Stakeholder Input
Residents and department chairs were excluded from the design process.
Overreliance on Proxies
Designers assumed department testing rates reflected individual exposure risk.
Process and Methods
Lack of Outcome Verification
System output was not validated against real frontline status before rollout.
No System Override Mechanism
Hospital lacked a clear mechanism to quickly adjust or fix flawed outputs.
Management
Dismissal of Early Warnings
Leadership ignored flaws flagged on Tuesday and proceeded with rollout.
Prioritizing Automation Over Logic
Relied on algorithm outputs instead of manual verification by chiefs.
Regulatory Environment
Misapplied CDPH Guidelines
State guidelines were meant for counties, not for internal hospital triage.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
