This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Organized criminal groups are utilizing AI-powered voice cloning and lip-syncing tools to create convincing deepfake videos of celebrities like Elon Musk. These videos are used to promote fraudulent cryptocurrency investment schemes on social media platforms, resulting in billions of dollars in financial losses for victims globally. Despite platform efforts to remove the content, the scams continue to proliferate due to the low cost and ease of production.
Scammers used AI to create deepfake videos of Elon Musk promoting fraudulent investment opportunities. Over time, these scams have reportedly led to billions in investor losses. The deepfakes also use voice cloning technology. They have been distributed on social media and YouTube. In particular, they target the elderly, such as 82-year-old Steve Beauchamp, to invest significant sums. Despite efforts by platforms to remove these videos, the scams continue to proliferate.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The AI-generated deepfakes spread highly deceptive and false claims about investment opportunities, leading victims to make decisions based on false beliefs.
The incident involves malicious actors using AI-generated deepfakes to impersonate trusted individuals and conduct highly targeted financial fraud and investment scams.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human scammers intentionally deploying AI voice-cloning and lip-syncing tools to defraud victims after the models were trained and released.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
Organized criminal groups are leveraging cheap, highly realistic AI voice-cloning and deepfake tools to conduct mass financial fraud globally. While causing significant economic harm to individual citizens and challenging social media content moderation, the incident represents a transnational crime and consumer protection issue rather than a direct threat to state national security.
The AI voice-cloning and lip-synching systems operate as tools, performing specific, deterministic functions on direct inputs (audio and video samples) without autonomous goal-directed planning.
Limited Risk: The report describes the use of AI-generated content such as deepfakes, which are subject to specific transparency and disclosure obligations under the Act.
The primary function of the AI systems used was to generate realistic deepfake videos by altering mouth movements and cloning celebrity voices to deceive viewers.
The reports provide detailed, first-hand accounts from multiple victims, cybersecurity experts, and platform representatives, clearly outlining the mechanics of the scams and the AI technologies used.
None, as the fraudulent intent and use of deepfake technology are explicitly documented.
Technology
Sophisticated AI Tools
Cheap and accessible voice cloning and lip-synching tools released to public.
Low Cost of Production
Videos cost as little as $10 and take less than 10 minutes to create.
Imperfect Detection Tools
Current detection websites have high false rates and struggle with compression.
Data Inputs
Abundant Celebrity Media
Large volume of online interviews of Elon Musk makes deepfaking easy.
Real Video and Audio Samples
Scammers use genuine clips to train voice clones and lip-sync models.
Single Image Requirement
Only a single image and video recording are needed to make a deepfake.
Human Factors
Targeting of Elderly Users
Older users are targeted due to unfamiliarity with deepfake technology.
Trust in Celebrity Endorsements
Victims believed the realistic Elon Musk videos were genuine endorsements.
Desire for Quick Wealth
Crypto enthusiasts are drawn to alternative paths for earning fortunes.
Process and Methods
Paid Social Media Ads
Scammers use paid ads on Facebook and TikTok to amplify their reach.
Spoofed Live Labels
YouTube live labels are used to make prerecorded deepfakes look real.
Inconsistent Ad Moderation
Social media platforms struggle to detect and block scam ads in real time.
Management
Inadequate Ad Screening
Meta and YouTube failed to prevent malicious actors from buying scam ads.
Cat-and-Mouse Security
Platform security teams struggle to keep pace with shifting scam tactics.
Regulatory Environment
Delayed Law Enforcement
Local police departments are overwhelmed and place fraud cases in queues.
Lack of Platform Liability
Absence of strong regulations holding ad networks liable for scam ads.
FBI and FTC Warning Limits
Federal agencies warn about deepfakes but lack direct enforcement powers.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
