This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
High school students in New York created and posted deepfake videos on TikTok featuring a local middle school principal and a sheriff's deputy making racist slurs and violent threats against Black students. The school district and law enforcement investigated the incident, determining that while the content was highly offensive, no specific laws were broken. Parents expressed significant anger regarding the school's lack of transparent communication and the perceived failure to treat the deepfaked threats as serious safety concerns.
Three Carmel High School students posted on TikTok a video featuring a nearby middle school's principal making aggressive racist remarks and violent threats against Black students.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The deepfake videos exposed the school community to highly toxic content, including explicit hate speech, racial slurs, and threats of gun violence.
The incident involved the targeted creation of deepfake videos to impersonate and humiliate specific individuals (the principal and deputy) while directing racist threats at Black students.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human actors intentionally utilizing deployed AI deepfake generation tools to create harmful impersonation videos.
Added categories peak at 2: Minor (Psychological Harm, Direct)
High school students used consumer AI tools to create racist and threatening deepfakes of a principal and deputy. While causing localized community distress and highlighting legislative gaps regarding synthetic media harassment, the incident has negligible direct national security implications.
The AI system was used as a tool by the high school students to generate synthetic video and voice outputs based on direct inputs. The AI did not exhibit autonomous planning, goal-directed behavior, or interactive assistant capabilities, but rather functioned deterministically to execute the deepfake generation task.
Limited Risk: The report describes the creation and dissemination of deepfakes, which are subject to transparency obligations under the EU AI Act to ensure users are aware the content is synthetic.
The students used AI technology specifically to generate deepfake videos and clone the voices of the principal and the sheriff's deputy.
The reports from Vice News and PetaPixel provide consistent, detailed accounts of the incident, including specific quotes from the deepfake videos, the reactions of parents and school board officials, and the legal/investigative outcomes. The role of AI (deepfake technology) is explicitly stated, and the resulting harms are well-documented.
None. The facts of the deepfake creation, the content of the videos, and the community response are clear and uncontested.
While the specific AI software used is not named, this does not prevent a high-confidence classification of the incident's domain, causes, and harms.
Technology
Advanced AI Deepfake Tech
Highly advanced AI enabled realistic voice cloning and face synthesis.
High Accessibility of AI Tools
Easy access to deepfake tools allowed students to generate harmful content.
Data Inputs
Use of Principal's Likeness
Audio/visual data of school staff was used without consent to train AI.
Lack of Audio/Video Verification
No digital signatures existed to verify the authenticity of the video.
Human Factors
Student Misuse of AI
High school students deliberately used AI to target and harass school staff.
Racial Bias and Hatred
Pre-existing racist attitudes in the community motivated the deepfakes.
Process and Methods
Delayed School Notification
School failed to immediately notify parents about the threatening videos.
Lack of Reporting Systems
Absence of anonymous reporting systems delayed detection of the videos.
Management
Inadequate Risk Assessment
School administration failed to anticipate or prepare for AI-enabled threats.
Poor Communication Strategy
Management was vague and slow in addressing parents' safety concerns.
Regulatory Environment
Lack of Deepfake Legislation
No federal laws protect individuals from non-consensual deepfakes.
Gaps in Criminal Law
Sheriff's office determined that creating the deepfakes broke no laws.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
