This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A Twitter user used generative AI to modify a trailer for Disney's 'The Little Mermaid', replacing the Black lead actress, Halle Bailey, with a white digital character. The action was widely criticized as racist, resulting in the suspension of the accounts involved by Twitter. The incident highlights the use of generative AI tools to create racially offensive synthetic media.
A Twitter user reportedly modified using generative AI a short clip of Disney's 2022 version of "The Little Mermaid," replacing a Black actress with a white digital character.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
Also classified as
The AI tool was used to misrepresent the race of a Black actress, reinforcing discriminatory attitudes and unfair representation.
The incident involved the creation and viral dissemination of racially offensive synthetic media ('whitewashing' a Black actress), exposing social media users to toxic and discriminatory content.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by a human user intentionally using deployed generative AI tools to modify the trailer and replace a Black actress with a white character.
An individual used generative AI to racially alter a movie trailer, replacing a Black actress with a white character. While the incident generated significant online controversy and highlighted the risks of discriminatory synthetic media, it carried negligible national security implications.
The report describes the AI system being used by a human creator to 'fix' the clip and digitally replace the actress. The AI operated as a tool on direct input to perform this specific image/video manipulation task.
Limited Risk: The system was used to generate AI-altered content (a deepfake or synthetic media), which falls under transparency obligations for AI-generated content under the EU AI Act.
The AI was used to digitally alter a video trailer, replacing the face and skin color of a real actress with a synthetic white character, which is the definition of deepfake video generation.
The report clearly details the incident, the motivation of the users, the reaction of the platform (suspension), and the nature of the AI-generated content. There is little ambiguity about what transpired.
None
Technology
Advanced Deepfake Capabilities
AI models allow rapid, realistic alteration of human faces and skin tones.
High-Performance Compute Access
Access to powerful GPUs enabled rendering the modified trailer in 24 hours.
Data Inputs
Unsanctioned Video Data Use
Using copyrighted trailer footage without permission to run AI modification.
Targeted Demographic Modification
Inputting race-based parameters to alter the main character's appearance.
Human Factors
Racial Bias and Backlash
Users sought to alter the black actress's race due to racist motivations.
Lack of Ethical Awareness
Developer claimed purely educational intent, ignoring the offensive impact.
Process and Methods
Rapid AI Video Generation
Lack of ethical checkpoints during the rapid video modification process.
Viral Social Media Distribution
Sharing controversial AI modifications on Twitter to generate engagement.
Management
Absence of AI Developer Oversight
No institutional oversight to prevent researchers from creating biased AI.
Inadequate Platform Moderation
Social media platforms struggled to proactively block abusive AI content.
Regulatory Environment
Reactive Platform Moderation
Twitter suspended accounts only after the offensive content went viral.
Lack of Deepfake Regulations
Absence of clear legal frameworks governing unauthorized AI whitewashing.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
