This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An analysis of California's COVID-19 vaccine distribution algorithm revealed that using ZIP codes instead of census tracts to allocate supply likely excluded over 2 million people in high-need, underserved neighborhoods. The algorithm, built by Blue Shield, prioritized larger geographic areas, which critics argue masked the needs of specific low-income census tracts and disproportionately impacted Black and Latinx populations.
California's vaccine-distribution algorithm used ZIP codes as opposed to census tracts in its decision-making, which critics said undermined equity and access for vulnerable communities who are largely low-income, underserved neighborhoods with low Healthy Places Index scores.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
The algorithm lacked the geographic resolution (robustness) to accurately identify vulnerable pockets within larger ZIP codes, leading to system failure in critical resource allocation.
The vaccine distribution algorithm's effectiveness in prioritizing high-need areas was significantly lower for communities of color, resulting in unequal outcomes and reduced benefits for Latinx and Black populations.
Human
Due to a decision or action made by humans
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The inequitable distribution risk was caused by human design decisions to use ZIP codes instead of census tracts in the algorithm, which unintentionally undermined the state's equity goals during post-deployment planning.
An analysis of California's COVID-19 vaccine distribution algorithm developed by Blue Shield showed that using ZIP codes instead of census tracts risked excluding 2 million high-need individuals, disproportionately affecting communities of color. While representing a significant domestic policy and civil rights concern during a public health crisis, the incident does not present a direct threat to US national security.
The vaccine distribution algorithm operates as a tool, performing a specific, well-defined function of allocating vaccine supply based on direct geographic inputs (ZIP codes) without context retention or autonomous planning.
High Risk: The system is used in healthcare and critical public resource allocation (vaccine distribution), which has significant implications for safety, fundamental rights, and public interests.
The primary goal of the Blue Shield algorithm was to allocate scarce COVID-19 vaccine resources across different geographic regions in California.
The report provides a clear, detailed analysis of the algorithm's design choices, the specific geographic metrics used, and the quantified demographic impact on the affected population. There is little ambiguity about the system's role or the nature of the risk.
The incident could be viewed purely as a policy/administrative failure rather than an AI safety failure, though the use of an automated algorithm to execute this policy makes it highly relevant to algorithmic safety.
Technology
ZIP Code Resolution Flaw
Using ZIP codes instead of finer-grained census tracts diluted equity scores.
Inflexible Allocation Logic
The system could not handle sub-ZIP code level targeting for vaccine supply.
Data Inputs
Coarse Geographic Data
ZIP codes were used instead of socioeconomic census tracts for tracking.
Omission of Demographic Factors
The algorithm did not explicitly prioritize high-risk racial or ethnic groups.
Human Factors
Prioritizing Simplicity
Decision-makers favored operational ease over precise equitable distribution.
Process and Methods
Inadequate Impact Assessment
The state failed to model how ZIP code grouping would impact vulnerable areas.
Flawed Metric Application
Applying the Healthy Places Index to ZIP codes instead of census tracts.
Management
Operational Bias in Design
Blue Shield and state management prioritized deployment speed over equity.
Inadequate Oversight of Vendor
Lack of rigorous state oversight on Blue Shield's algorithmic design choices.
Regulatory Environment
Lack of Algorithmic Standards
No regulatory mandates existed to audit public policy algorithms for equity.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
