This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Scammers are increasingly using AI tools from companies like HeyGen and ElevenLabs to create deepfake videos of influencers and private individuals without their consent. These deepfakes are used to promote offensive products, such as erectile dysfunction supplements, or to spread political disinformation targeting specific demographics. Victims report feeling violated and helpless as their likenesses are manipulated to say things they would never say, with limited recourse for removal or legal action.
Scammers used AI tools from HeyGen and ElevenLabs to create deepfake videos of influencers Michel Janse, Olga Loiek, Shadé Zahrai, and Carrie Williams, misusing Lana Smalls's voice in Williams's case. These videos promoted offensive products and false messages, in some cases targeting nationalist Chinese men to boost China-Russia ties, causing emotional distress and damaging the victims' reputations.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The deepfakes exposed users to highly inappropriate and offensive content, including sexualized ads referencing adult film audio and erectile dysfunction.
The deepfakes were also used in coordinated campaigns on Chinese social media to spread pro-Russia political propaganda and stoke nationalistic sentiment.
The primary harm involves scammers using AI tools to clone individuals' likenesses for unauthorized commercial endorsements, scams, and targeted sexualized humiliation.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk is driven by human scammers who intentionally deploy and use AI cloning tools to create unauthorized deepfakes for financial and political gain.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
Scammers used consumer AI tools from HeyGen and ElevenLabs to clone the likenesses of individuals for unauthorized commercial advertisements and localized pro-Russia political propaganda. While violating individual privacy and contributing to online disinformation, the incident represents a minor national security threat managed through standard platform moderation and legal frameworks.
The report describes the AI systems being used by scammers to perform specific, well-defined functions such as generating synthetic voices and syncing lip movements to audio. The systems operate on direct inputs (short video/audio clips) to produce these outputs without exhibiting independent goal-directed behavior or planning.
Limited Risk: The report describes AI-generated content such as deepfakes, which pose a moderate risk and require specific transparency obligations to ensure users are informed of the AI's nature.
The primary function of the AI systems (HeyGen and ElevenLabs) in these incidents was to generate realistic deepfake videos and synthetic voices of real individuals without their consent.
The reports provide clear, detailed, and consistent accounts of multiple individuals whose likenesses were stolen and cloned using AI tools. The roles of HeyGen and ElevenLabs are explicitly mentioned, and the real-world harms (emotional distress, non-consensual commercial/political exploitation) are well-documented with direct quotes from the victims.
None. The incidents are clearly cases of unauthorized deepfake creation for commercial gain and political propaganda.
Technology
Advanced Voice and Video Cloning
Tools from HeyGen and Eleven Labs allow realistic voice and lip-sync cloning.
Low Cost of AI Generation
Cheap promotional tools drastically lower the cost for scammers to run ads.
Lack of Robust Deepfake Detection
Platforms struggle to automatically detect and flag realistic AI deepfakes.
Data Inputs
Publicly Available Social Media Media
A few seconds of public video/audio are enough to clone a person's likeness.
Cross-Platform Video Scraping
Scammers scrape content from Western social media to use on Chinese platforms.
Lack of Source Verification Data
Platforms lack metadata to verify if uploaded videos contain stolen likenesses.
Human Factors
User Naivety to Deepfake Risks
Victims believed deepfakes only targeted famous political or public figures.
Financial and Nationalist Motives
Scammers leverage nationalistic sentiment to drive traffic and sales.
Masculine and Nationalistic Pride
Target audience consumes objectifying content to reaffirm pride.
Process and Methods
Slow Platform Takedown Processes
YouTube and other platforms were slow to implement deepfake removal policies.
Evasion of Automated Detectors
Scammers bypass landing page detectors by hosting clean redirect pages.
Inadequate Cross-Border Monitoring
Closed internet environments prevent foreign victims from finding stolen content.
Management
Delayed Safety Policy Deployment
YouTube delayed executing promised policies to allow synthetic content removal.
Profit Over Safety Focus
Scammers exploit automated ad systems that prioritize volume over verification.
Lack of Proactive Monitoring
Platform management failed to proactively audit advertiser accounts.
Regulatory Environment
Absence of Federal Deepfake Laws
No federal laws exist to govern deepfakes outside of politics and porn.
Jurisdictional Enforcement Gaps
Global scammers exploit jurisdictional boundaries to avoid legal prosecution.
Costly Legal Recourse for Victims
Bringing civil cases for likeness theft is too expensive for normal victims.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
