This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
YouTube has experienced a significant influx of AI-generated and manipulated media targeting Black celebrities with false narratives and salacious disinformation. These videos, which often use synthetic voiceovers and AI-generated thumbnails, have garnered millions of views and are algorithmically promoted. Despite some channel terminations by YouTube, the content continues to circulate, posing challenges to content moderation and public information ecosystems.
YouTube faced a surge of AI-generated fake news targeting Black celebrities, including fake narratives about Sean “Diddy” Combs and others. These videos, blending AI-generated and manipulated media, amassed millions of views, challenging content moderation efforts and highlighting the spread of disinformation.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The videos expose platform users to unwanted sexualization and harassment through synthetic media depicting lewd acts.
The flood of highly engaging AI-generated fake news videos threatens the broader information ecosystem and degrades shared reality.
The incident involves the intentional creation of humiliating and sexualized synthetic content targeting specific Black celebrities for personal monetization and views.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk is caused by the intentional decisions and actions of human creators who use AI tools to generate and spread fake news for profit.
Added categories peak at 4: Severe (Epistemic Harm, Direct)
An influx of AI-generated disinformation and synthetic media targeted Black celebrities on YouTube to exploit cultural trust and generate ad revenue. While demonstrating the ease of scaling highly engaging fake content using generative AI, the incident remains a commercial fraud and content moderation issue rather than a strategic national security threat.
The AI systems used (text-to-speech and image generators) were operated as tools by human creators to generate specific content based on direct inputs.
Limited Risk: The report describes AI-generated content such as deepfakes, which require specific transparency obligations to ensure users are informed of the AI's nature.
The creators primarily used AI text-to-speech technology to generate synthetic voiceovers and generative AI tools to create fake thumbnails.
The report provides detailed, credible information about the channels, views, and specific celebrities targeted, backed by expert analysis from the University of Buffalo and Reality Defender.
Some channels claimed they only used AI for text-to-speech and not for scripts or thumbnails, suggesting a hybrid of manual and AI manipulation.
Technology
Generative AI Voice Tools
Allowed easy creation of highly realistic fake celebrity voiceovers.
AI Recommendation Algorithms
Prioritized high-engagement fake news, pushing it to top search results.
Automated Thumbnail Generation
AI tools enabled rapid creation of misleading and manipulated thumbnails.
Data Inputs
Unlabeled Synthetic Media
Lack of metadata or labels made it hard to flag AI content automatically.
Remixing Real and Fake Data
Blending real news events with AI falsehoods deceived detection systems.
Manipulated Media Inputs
Using real celebrity photos mixed with AI-generated elements to mislead.
Human Factors
Audience Trust in Alternative Media
Distrust in mainstream media led viewers to believe fake community channels.
Financial Motivations of Creators
Freelancers and creators exploited AI to churn out high-volume clickbait.
Difficulty in Detecting Deepfakes
Average viewers struggled to distinguish synthetic audio from real voices.
Process and Methods
Delayed Policy Implementation
The announced synthetic media labeling policy was not yet active.
Slow Content Moderation Response
Channels flourished for long periods before YouTube took enforcement action.
Flawed Monetization Review
Channels profited from ads on fake news before being demonetized.
Management
Engagement Prioritization
Platform management prioritized metrics that favored sensational fake news.
Inadequate Proactive Safeguards
Management failed to deploy automated preventative filters for deepfakes.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
