This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Clinical decision algorithms, specifically the eGFR formula used to assess kidney function, have been found to incorporate race-based multipliers that systematically assign healthier scores to Black patients. This bias results in the underestimation of disease severity, leading to delayed medical interventions such as specialist referrals and kidney transplant waitlisting. The issue has prompted petitions and policy changes at several major academic medical centers, highlighting broader concerns about systemic bias in medical algorithms.
Decades-long use of the estimated glomerular filtration rate (eGFR) method to test kidney function which considers race has been criticized by physicians and medical students for its racist history and inaccuracy against Black patients.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
Doctors relied on the algorithm's race-based recommendations without questioning them, leading to zero of the 64 qualifying Black patients being referred for transplants.
The algorithm performs with unequal accuracy across demographic groups, systematically overestimating kidney health for Black patients compared to white patients, leading to unequal medical outcomes.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The clinical harm is caused by the automated output of the deployed eGFR algorithm, which unintentionally underestimated kidney disease severity in Black patients.
Added categories peak at 2: Minor (Psychological Harm, Inferred)
A widely used clinical algorithm for assessing kidney function incorporated a race-based multiplier that systematically underestimated the severity of kidney disease in Black patients, delaying critical treatments like transplants. While presenting significant ethical and civil rights concerns within the healthcare sector, the incident has minor direct national security implications and is being addressed through institutional policy reforms.
The eGFR formula is a deterministic mathematical equation that performs a specific, well-defined function on direct inputs (creatinine, age, sex, race) without planning or context retention.
High Risk: The system is an AI tool used in healthcare for clinical decision-making and diagnostic support, which has significant implications for safety and fundamental rights.
The algorithm's primary function is to estimate kidney function to support clinical diagnosis, which in turn determines the allocation of medical resources like transplant waitlists.
The reports provide clear, peer-reviewed scientific study results detailing the exact number of patients affected within a specific healthcare system, along with the precise mathematical adjustment used.
None
Technology
Flawed Race-Correction Formula
Formula boosts Black patient scores by 15.9 percent, masking kidney disease.
Lack of Physiological Basis
Equation incorrectly equates social category of race with biological difference.
Data Inputs
Small Black Patient Sample
Original 2009 study used small sample size of Black patients to build formula.
Binary Categorization of Race
Formula simplistically groups patients into only Black or non-Black.
Confounding Creatinine Factors
Formula ignored non-racial factors affecting creatinine like diet.
Human Factors
Uncritical Trust in Algorithm
Doctors accepted race-based scores without questioning clinical validity.
Education in Biased Science
Medical training historically taught race as a biological variable.
Process and Methods
Outdated Medical Guidelines
Societies maintained guidelines endorsing the race-adjusted formula.
Lack of Care Plan Reviews
Hospitals failed to review historical patient care plans for racial bias.
Management
Institutional Inertia
Hospitals delayed changing formula until prompted by student petitions.
Delayed Policy Updates
Management waited for society guidelines before updating local formulas.
Regulatory Environment
Delayed Federal Investigation
HHS and CMS did not investigate race-adjusted algorithms for years.
Lack of Algorithmic Standards
No regulatory standards existed to prevent bias in clinical calculators.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
