This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
During the October 2020 California bar exam, an AI-powered proctoring system flagged over one-third of the 9,301 test-takers for alleged rule violations. This high rate of flagging led to widespread accusations of cheating, forcing many applicants to undergo a stressful review process to prove their innocence. Critics and legal representatives argued that the system was overly sensitive and that the process for challenging these allegations was opaque and unfair, as applicants were often denied access to the video evidence used against them.
The proctoring algorithm used in a California bar exam cited a third of thousands of applicants as cheaters, resulting in allegations where exam takers were instructed to prove otherwise without seeing their incriminating video evidence.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The reports note that the algorithm flagged Black and other minority applicants for merely existing, indicating potential demographic bias and unequal performance.
The bar examiners over-relied on the AI's automated flags, issuing violation notices to thousands of applicants without first conducting human verification of the video evidence.
The AI proctoring software failed to perform reliably under normal test-taking conditions, resulting in an extremely high false-positive rate where 34% of innocent applicants were flagged for cheating.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by the ExamSoft AI proctoring system unintentionally flagging a massive number of innocent test-takers due to over-sensitivity, occurring after the software was deployed for the online bar exam.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
An AI-powered proctoring system unintentionally flagged over a third of California bar exam takers for cheating due to high sensitivity and potential demographic bias. While causing significant stress, financial costs, and privacy concerns for thousands of applicants, the incident is a domestic administrative failure with negligible direct impact on national security.
The AI proctoring software operated independently during the exam, monitoring test-takers' webcams and microphones in real-time and autonomously generating flags for suspicious behavior without human intervention.
High Risk: The AI system is used for professional licensing and vocational assessment, which directly affects access to employment. Under the EU AI Act, AI systems used in education, vocational training, or employment management are classified as High Risk.
The AI system was explicitly designed and deployed to monitor test-takers via webcam and microphone to detect and flag potential cheating behaviors during the online bar exam.
The reports provide precise statistics (9,301 test-takers, 3,190 flagged) and clear descriptions of the AI system's role, the developer (ExamSoft), and the specific harms (stress, legal costs, potential licensing delays) experienced by the affected parties.
None. The role of the AI in generating the flags that led to the administrative actions is clear and undisputed.
Technology
Biased Flagging Algorithm
Algorithm flagged Black and diverse applicants at disproportionate rates.
Overly Sensitive AI Proctoring
System flagged benign behaviors like gazing off-screen as cheating.
Technical Software Issues
15 percent of test takers experienced technical issues on exam day.
Data Inputs
Biased Training Data
Lack of diverse representation led to false cheating flags for minorities.
Video and Audio Infractions
Environmental inputs like silence or eye gaze triggered automatic flags.
Prohibited Items in View
Presence of food or digital clocks in video feed triggered alerts.
Human Factors
Natural Test-Taking Behaviors
Applicants naturally looked away or remained silent, triggering flags.
Lack of Proctor Guidance
No live proctors to prevent minor rule infractions before the exam.
Remote Environment Confusion
Applicants took the test in home environments with unavoidable distractions.
Process and Methods
Lack of Human Pre-Verification
Examiners sent notices without reviewing videos to verify AI flags.
Vague Chapter 6 Notices
Notices contained generic descriptions of alleged violations.
Denial of Evidence Access
Applicants had to respond to allegations without seeing their videos.
Management
Over-reliance on Automated Flags
Management assumed the AI could accurately detect academic dishonesty.
Inadequate Resource Planning
Bar was unprepared to review the massive volume of flagged videos.
Lack of Accountability
Bar shifted the burden of proof to applicants rather than verifying.
Regulatory Environment
Pandemic Emergency Mandate
COVID-19 forced a rapid, unregulated shift from in-person to online exams.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
