This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The University of Texas at Austin used a machine-learning tool called GRADE from 2013 to 2019 to screen graduate school applications. The system, trained on historical admissions data, was criticized for potentially encoding and reinforcing biases against marginalized applicants. Following public criticism, the university suspended the use of the algorithm, citing concerns about fairness and maintenance difficulties.
The University of Texas at Austin's Department of Computer Science's assistive algorithm to assess PhD applicants "GRADE" raised concerns among faculty about worsening historical inequalities for marginalized candidates, prompting its suspension.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
Also classified as
Human reviewers' opinions were likely colored by the algorithm's low scores, creating an overreliance on the automated screening.
The machine-learning algorithm did not provide clear guidance on how it classified or scored applicants, making its decisions difficult to interpret.
The GRADE system was criticized for encoding and replicating historical biases, leading to potential unfair discrimination against applicants based on gender, race, and academic background.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The potential bias and unfair screening of applicants were unintended consequences of deploying the GRADE algorithm to streamline the admissions process.
The University of Texas at Austin's use of the GRADE machine-learning tool from 2013 to 2019 highlighted concerns about algorithmic bias replicating historical human prejudices in admissions. While raising significant ethical and equity concerns regarding access to higher education, the incident remains a localized academic issue with negligible direct national security implications.
The GRADE system acted as an assistant by predicting admissions outcomes and assigning scores to help human reviewers prioritize applications, rather than making final autonomous decisions.
High Risk: The system is used in education and vocational training to evaluate and screen applicants for admission, which directly affects access to education.
The GRADE system was explicitly designed to evaluate graduate school applications and assign scores to streamline the review process and allocate faculty time efficiently.
The reports provide clear, consistent details about the GRADE algorithm's purpose, training data, deployment timeline, and the specific criticisms regarding bias. The university's response and the creators' perspectives are also well-documented.
The university claimed the system was discontinued primarily due to maintenance difficulties, whereas critics attribute it entirely to bias and ethical concerns.
An empirical audit of the admissions decisions during the years GRADE was active would provide definitive proof of the scale of bias, but this is not strictly necessary for classification.
Technology
Algorithmic Bias Replication
Replicated and locked in historical biases of the 2013 admissions committee.
Lack of Model Interpretability
ML algorithms do not give clear guidance on how they classify applicants.
Software Maintenance Issues
Changes in data and software environments made the system hard to maintain.
Data Inputs
Biased Training Data
System trained on historical data reflecting past inequities and human bias.
Proxy Variables for Bias
Variables like elite university names and letter keywords encoded social bias.
Unstructured Data Simplification
Turned letters of recommendation and personal statements into a bag of words.
Human Factors
Automation Bias
Low scores from the algorithm colored human reviewers' opinions of applicants.
Overreliance on Efficiency
Staff accepted reduced reviews for low-scored applicants to save time.
Unconscious Bias in Scoring
Reviewers relied on subjective factors like letters containing specific words.
Process and Methods
Lack of Algorithmic Auditing
No effort was made to audit the impact of scores across demographic groups.
Inadequate Disclosure
Applicants were not told they were screened by an algorithm.
Single Human Reviewer Safeguard
Using only one human reviewer instead of multiple passes reduced oversight.
Management
Prioritizing Speed over Equity
Management prioritized saving review time over ensuring fair admissions.
Lack of Risk Assessment
Deployed the tool without assessing long-term impacts on student diversity.
Regulatory Environment
Lack of External Standards
No clear regulations or standards governed AI use in university admissions.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
