This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The Gates Foundation's 'Intensive Partnerships for Effective Teaching' initiative used algorithmic value-added models to evaluate teacher performance. An independent assessment by the Rand Corporation found the program failed to improve student outcomes and that the algorithmic models were statistically weak and flawed. The program resulted in professional harm to teachers and wasted significant resources.
Gates-Foundation-funded Intensive Partnerships for Effective Teaching Initiative’s algorithmic program to assess teacher performance reportedly failed to achieve its goals for student outcomes, particularly for minority students, and was criticized for potentially causing harm against teachers.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The flawed algorithmic evaluations unfairly ruined careers and drove teachers out of the profession.
The value-added models were secret black boxes whose inner workings were kept hidden from experts and teachers.
The primary failure was that the value-added models were statistically weak and performed poorly, failing to reliably measure teacher effectiveness as intended.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The harm was caused by the flawed evaluations generated by the value-added model algorithms, which were deployed to make high-stakes employment decisions.
Added categories peak at 3: Substantial (Epistemic Harm, Direct)
An educational initiative funded by the Gates Foundation used flawed algorithmic value-added models to evaluate teachers, resulting in career disruptions and 575 million dollars in wasted resources. The incident represents negligible national security risk, highlighting domestic challenges in algorithmic transparency and public sector labor management rather than geopolitical threats.
The value-added models functioned as statistical tools that processed inputs like test scores and surveys to output performance ratings without autonomous planning or independent agency.
High Risk: The system is an educational and worker management system used for high-stakes decisions such as the promotion or firing of teachers, which directly affects employment and access to education.
The algorithmic system was explicitly designed to assess teacher performance and determine whether they were adding value to decide on promotions or terminations.
The report is based on an independent assessment by the Rand Corporation of a major public initiative, providing clear details on the system's failure, financial scale, and real-world harms.
One could argue the failure was primarily organizational rather than algorithmic, but the report explicitly highlights the statistical weakness of the value-added models themselves.
Technology
Black-Box Value-Added Models
Value-added models used to rate teachers were secret black boxes.
Statistically Weak Algorithms
Algorithms were little better than random number generators.
Data Inputs
Biased Parent and Student Surveys
Surveys were biased based on how questions were framed.
Flawed Principal Observations
Principals tended to give almost all teachers great scores.
Human Factors
Overreliance on Algorithm
High-stakes decisions were made based on unproven algorithmic outputs.
Process and Methods
Secret Assessment Formulas
Keeping formulas secret prevented experts from identifying flaws.
Lack of Pre-Deployment Testing
The program was unleashed on people without prior scientific testing.
Management
Flawed Big Data Assumptions
Management assumed gathering more data would automatically improve outcomes.
Premature High-Stakes Deployment
Decisions on firing and promotion were based on weak statistical models.
Regulatory Environment
Lack of Algorithmic Oversight
No regulatory bodies oversaw algorithmic teacher evaluations.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
