Multiple incidents have been reported across the United States where students used AI-powered 'nudification' applications to create and distribute nonconsensual sexually explicit images of their female classmates. These incidents have prompted state and federal lawmakers to rush new legislation to criminalize the creation and sharing of such deepfakes, as existing laws often fail to cover AI-generated content. The reports highlight a growing trend of technology-facilitated sexual harassment in schools, impacting the mental health and safety of minor victims.
At a high school in Issaquah, Washington, a male student is reported to have used deepfake technology to alter pictures of several female classmates and then shared them.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The creation and circulation of sexually explicit deepfakes of minors exposed students and school communities to highly toxic and inappropriate content.
The incident involves the targeted creation and distribution of humiliating and sexually explicit synthetic imagery of specific victims (female classmates) for personal harassment and exploitation.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The harm was caused by the intentional actions of male students who chose to use AI nudification apps to create and distribute sexually explicit deepfakes of their female classmates.
Added categories peak at 3: Substantial (Child Sexual Exploitation & Abuse (CSEA), Direct)
Multiple incidents of students using AI-powered nudification apps to create nonconsensual explicit deepfakes of classmates highlight growing societal challenges around synthetic media. While causing severe distress to victims and prompting legislative updates, the national security impact remains minor and localized to civil law enforcement domains.
The AI system is described as an app designed to automatically 'strip' clothed photos of real girls. It acts as a tool that takes a specific input (a clothed photo) and performs a single well-defined function to generate a synthetic nude image.
Risk Level 3: Limited Risk. The system is used to generate 'AI-generated content such as deepfakes', which falls under the transparency obligations of Limited Risk systems.
The primary function of the AI apps described is to generate realistic, synthetic nude images ('deepfakes') by altering clothed photos of real individuals.
The reports provide clear, detailed accounts of multiple incidents of AI-generated deepfake creation in schools, including specific locations, legal responses, and the impact on victims. The role of the AI nudification apps is explicitly described.
None. The events clearly represent nonconsensual deepfake creation and harassment.
Technology
Availability of Nudification Apps
Apps designed to strip clothed photos of real people are widely accessible.
Generative AI Image Manipulation
AI algorithms can realistically simulate nude images using real faces.
Data Inputs
Use of Unconsented Clothed Photos
Target photos taken in public or social settings were used without consent.
Human Factors
Peer Exploitation and Harassment
Male classmates surreptitiously generated and shared explicit fake images.
Lack of Digital Responsibility
Minors lack understanding of the severe consequences of using generative AI.
Process and Methods
Lack of App Developer Liability
No mechanisms exist to hold app developers accountable for misuse.
Delayed School Notification
Schools failed to immediately inform parents of the deepfake victimization.
Management
Commercialization of Nudification Apps
App developers profit from releasing tools that enable nonconsensual imagery.
Regulatory Environment
Inadequate Legal Protections
Existing laws failed to address AI-generated synthetic explicit images.
Lack of Federal Legislation
No federal laws specifically criminalize the creation of deepfake nudes.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.