This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Students at multiple universities filed complaints after discovering professors were using generative AI tools like ChatGPT, Perplexity, and Gamma to generate course materials, grading rubrics, and feedback without disclosure. The incidents highlight growing tensions in higher education regarding the ethical use of AI, transparency, and the value of human-centric instruction versus algorithmic assistance.
A student at Northeastern University reportedly filed a complaint after discovering that a professor had used generative AI tools, including ChatGPT, to produce course materials despite university policies discouraging undisclosed AI use. The student alleged hypocrisy, citing a syllabus that barred unauthorized AI use by students. The professor acknowledged having used AI-generated materials and expressed regret over not reviewing them more carefully. No tuition refund was reportedly granted.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
Also classified as
Students expressed that outsourcing instruction and feedback to AI devalues the human effort and connection they pay high tuition for, and raises concerns about disrupting the pipeline of future teaching assistants.
The incident represents a clear case of human overreliance on AI systems, where professors trusted chatbots to perform tasks requiring human empathy, grading, and teaching expertise.
Human
Due to a decision or action made by humans
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was caused by the human decision of professors to overrely on AI tools for grading and teaching, leading to an unexpected negative outcome of student backlash and loss of trust.
The incident involves university professors using generative AI tools for grading and course material generation without disclosure. While it highlights ethical concerns and student dissatisfaction regarding educational quality, it presents negligible national security implications across all assessed categories.
The report describes the AI systems being used as assistants to augment human capabilities, providing suggestions, drafting emails, and organizing feedback, while requiring human oversight which was neglected.
Risk Level 2: High Risk. The report describes AI being used in educational and vocational training systems, specifically for grading student essays and generating course materials, which directly affects student evaluation and access to education.
The AI systems were primarily used as writing assistants to generate lecture notes, slides, emails, and student feedback, and secondarily to assist in grading and skill assessment.
The reports provide clear, detailed, and consistent accounts of multiple independent incidents of AI overreliance in higher education, including specific names, universities, and administrative responses.
Some might view this as a standard productivity tool dispute rather than an AI safety incident, but the systemic overreliance and ethical concerns justify the safety classification.
Technology
AI Hallucinations and Errors
AI generated distorted text, weird images, and errors in slides.
Algorithmic Flaws in AI Tools
Chatbots outputted nonsensical or low-quality educational content.
Data Inputs
Unvetted Prompting Inputs
Professors fed rubrics or raw topics to AI without refinement.
Human Factors
Overreliance on AI Automation
Professors trusted AI outputs without reviewing them for accuracy.
Lack of Human Oversight
Instructors failed to proofread slides or read student essays.
Perceived Academic Hypocrisy
Professors banned student AI use while using it themselves.
Process and Methods
No Disclosure of AI Use
Educators hid AI use in grading and course material creation.
Inadequate Output Review Processes
Lack of verification steps before publishing AI-generated materials.
Outsourcing Empathy to AI
Using AI for sensitive communications, causing student backlash.
Management
High Faculty Workloads
Heavy workloads pressured professors to use AI as a shortcut.
Inadequate Faculty Training
Universities failed to train faculty on AI risks and limitations.
Lack of Admin Accountability
Management failed to address student complaints with restitution.
Regulatory Environment
Lagging Institutional AI Policies
Universities lacked clear rules on acceptable AI use for faculty.
Inconsistent Integrity Rules
Rules restricted student AI use but ignored faculty AI usage.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
