This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A Yale Executive MBA student was suspended for a year and given an F after an AI detection tool, GPTZero, flagged his exam as AI-generated, leading to a major lawsuit alleging bias against non-native English speakers.
In 2024, Yale University reportedly used GPTZero as one factor in investigating whether Executive MBA student Thierry Rignol used AI on an exam. Rignol reportedly denied cheating and alleged the detector falsely classified his writing as AI-generated. Yale reportedly later suspended him for a year and gave him an F, while citing additional evidence and his conduct during the investigation. Rignol has challenged the process in federal court.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
The AI detection tool GPTZero demonstrated a severe lack of robustness and reliability by generating false positives, including flagging 30-year-old human-written texts as 100% AI-generated.
The incident centers on the AI detector's alleged bias against non-native English speakers, where the tool's unequal accuracy across demographic groups led to a false cheating accusation and severe academic penalties.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was triggered by the biased and unreliable output of the deployed GPTZero AI detection tool, which unintentionally flagged a non-native English speaker's writing as AI-generated.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
A Yale EMBA student was suspended following a false positive flag from the AI detector GPTZero. The incident highlights ongoing issues with AI detection reliability and demographic bias, but it is a localized academic and legal dispute with negligible national security implications.
GPTZero operates as a tool because it performs the specific, well-defined function of scanning text inputs to calculate the probability of AI generation without maintaining an ongoing autonomous state or planning capabilities.
Risk Level 2: High Risk. The AI system was used in an educational setting to assess student work and determine disciplinary action, which directly affects access to education as defined under High Risk systems ('Educational and vocational training systems affecting access to education').
The primary AI system involved, GPTZero, was deployed specifically to detect whether the student had cheated by using generative AI on his final exam.
The report provides a highly detailed account of the dispute between Thierry Rignol and Yale University, including specific dates, the name of the AI tool (GPTZero), the academic and financial consequences, and the arguments from both sides. The role of the AI detector as a trigger for the incident is clear, and the limitations of the tool are well-documented.
The incident could be interpreted primarily as an administrative and disciplinary dispute regarding student cooperation rather than an AI safety failure, given that the suspension was triggered by Rignol's failure to provide his Pages file.
Technology
Unreliable AI Detection Tools
GPTZero and similar tools are highly unreliable for detecting AI text.
Detector Bias Against Non-Native Writers
Detectors mistake formal non-native English prose for AI-generated text.
High False Positive Rates
Scans of human-written historical texts falsely flagged as 100% AI.
Data Inputs
Linguistic Patterns of Test Text
Formal, structured, and long test prose triggered detector flags.
Overlap with ChatGPT Output
Student answers had substantial overlap with ChatGPT generated outputs.
Human Factors
Over-reliance on AI Detectors
Teaching staff relied heavily on automated tools to flag cheating.
Communication Failures
Student withheld Pages file because Yale asked for a Word file.
Misinterpretation of Writing Style
Professors mistook high-quality, polished writing for AI output.
Process and Methods
Infeasible AI Policing Methods
Using automated detectors to police academic integrity is highly flawed.
Lack of Clear AI Verification Steps
No robust process to verify AI use beyond detector tools.
Inadequate Due Process
Student suspended for lack of candor without a formal charge.
Management
Poor Academic Risk Assessment
Management failed to assess risks of using biased AI detection tools.
Escalation of Disciplinary Pressure
Administrators used high-pressure tactics to force a confession.
Regulatory Environment
Lack of Standardized AI Rules
No clear institutional standards on validating AI detection tool use.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
