This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Between 1982 and 1986, St [name withheld] Hospital Medical School used an automated admissions screening program that replicated and exacerbated existing human biases. The system, trained on historical selection data, systematically disadvantaged women, ethnic minorities, and working-class applicants by assigning them lower scores, leading to the denial of interview opportunities for approximately 60 applicants annually.
From 1982 to 1986, St George's Hospital Medical School used a program to automate a portion of their admissions process that resulted in discrimination against women and members of ethnic minorities.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
Also classified as
Staff relied entirely on the automated system to screen candidates without human oversight or understanding how the program functioned.
The AI system systematically discriminated against female and non-European applicants, leading to unfair admissions outcomes based on gender and race.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The rejections were directly decided by the automated scoring system after it was deployed, though the bias originated from human historical data.
Added categories peak at 2: Minor (Psychological Harm, Inferred)
A historical case from 1982-1986 where St George's Hospital Medical School used an automated admissions program that systematically discriminated against female and non-European applicants. While presenting negligible threats to physical, economic, or state security, it represents an early, prominent example of algorithmic bias violating civil rights, though localized to a single institution.
The system operated independently to score and filter applicants without human review, using learned patterns from historical data.
High Risk: The system is an educational admissions tool that directly affects access to vocational training and higher education.
The system was explicitly designed to evaluate and screen student applications for medical school admissions.
The reports provide consistent, historically documented details about the system's development, deployment, bias, and the subsequent investigation.
None, the case is a well-established historical example of algorithmic bias.
Technology
Algorithmic Mimicry of Human Bias
The program was designed to match historical human selection panel decisions.
Proxy Variable Deduction
Deduced race and class from surnames, birthplaces, and postcodes.
Data Inputs
Biased Historical Training Data
Trained on historical admissions data reflecting past human discrimination.
Use of Demographics as Proxies
Relied on surname, birthplace, and postcode to infer candidate profiles.
Human Factors
Blind Trust in Automation
Staff assumed the computer program was objective and lacked bias.
Lack of AI Literacy among Staff
Admissions staff did not understand how the algorithm processed data.
Process and Methods
Automation of Unchecked Processes
Automated the admissions stage without auditing the underlying human biases.
No Human-in-the-Loop Review
Candidates were rejected by the model without any human review of files.
Inadequate Validation Testing
Validated model only on matching historical decisions, not on fairness.
Management
Prioritizing Efficiency Over Equity
Adopted the software primarily to reduce administrative workload and costs.
Lack of Systemic Accountability
No single manager took responsibility for auditing the algorithm's contents.
Inadequate Risk Assessment
Management failed to assess the risk of replicating historical discrimination.
Regulatory Environment
Lack of Algorithmic Standards
No regulatory frameworks existed to govern algorithmic fairness in the 1980s.
Retrospective Oversight Only
Regulatory body (CRE) could only act after formal complaints were filed.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
