This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A study published in JAMA Network Open demonstrates that medical diagnostic software for lung function incorporates race-based adjustments that result in the underdiagnosis of respiratory conditions in Black men. By artificially raising the diagnostic threshold for Black patients, these algorithms may prevent timely access to medications, procedures, and lung transplants. The findings highlight a broader issue of race-based assumptions embedded in clinical decision-support systems across various medical fields.
A study published in JAMA Network Open reveals that racial bias built into a commonly used medical diagnostic algorithm for lung function may be leading to underdiagnoses of breathing problems in Black men. The study suggests that as many as 40% more Black male patients might have been accurately diagnosed if the software were not racially biased. The software algorithm adjusts diagnostic thresholds based on race, affecting medical treatments and interventions.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
The diagnostic software performs with unequal accuracy and effectiveness across demographic groups, specifically underdiagnosing Black patients due to race-based adjustments built into its design.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The underdiagnosis is directly caused by the outputs of the deployed diagnostic software, which unintentionally perpetuates historical racial biases while attempting to assess lung function.
Added categories peak at 2: Minor (Psychological Harm, Inferred)
The incident involves embedded racial bias in medical diagnostic software that underdiagnoses respiratory conditions in Black patients. While it poses a serious concern for healthcare equity and civil rights, its direct national security implications are minor and restricted to domestic public health software systems.
The spirometry software acts as a clinical decision support tool, processing test results to provide a score that assists doctors in making diagnostic and treatment decisions, requiring human oversight.
High Risk: The report describes an AI system used in healthcare as a diagnostic tool, which is explicitly classified as High Risk under the EU AI Act due to its significant implications for patient safety and fundamental rights.
The primary function of the software is to analyze spirometry measurements to assist clinicians in diagnosing respiratory conditions.
The report is highly detailed, citing a specific peer-reviewed study in 'JAMA Network Open' with clear statistics (e.g., 40% more diagnoses, nearly 400 additional cases, sample sizes of 2,700 Black men and 5,700 white men). The role of the software and the nature of the bias are explicitly described.
None. The findings of the study regarding algorithmic bias are clear and uncontested in the text.
Technology
Race-adjusting software design
Software raises diagnostic thresholds for Black patients based on race.
Fragmented system software
Hospitals use varied software versions, delaying system-wide updates.
Data Inputs
Biased baseline measurements
Historical reference data adjusted scores based on patient race.
Human Factors
Historical clinical bias
Clinicians historically assumed Black lungs were innately inferior.
Automation bias in diagnosis
Clinicians rely on software scores instead of clinical symptoms.
Process and Methods
Biased diagnostic guidelines
Medical standards institutionalized race-based adjustments in software.
Management
Delayed software modernization
Hospital management fails to quickly deploy updated, unbiased algorithms.
Regulatory Environment
Lack of algorithmic standards
No regulatory body mandated the removal of race-based software adjustments.
