This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Fraudsters are using AI-driven voice and image cloning to create deepfake videos of public figures, such as politicians and celebrities, to promote fraudulent investment schemes on social media platforms like Meta. These scams exploit public trust, leading to significant financial losses for victims across Europe. Despite efforts by 'trusted flaggers' to report these ads, platforms like Meta are criticized for slow response times and ineffective moderation, allowing the scams to proliferate.
Purported deepfake videos reportedly circulated on Meta platforms cloning Irish Fine Gael presidential candidate Heather Humphreys to allegedly portray her endorsing high-return investment schemes. The purportedly AI-generated image and voice cloning aimed to exploit public trust and lure consumers into fraud. The Bank of Ireland warned of further scams and risks.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
The incident involves fraudsters using AI voice and image cloning to impersonate trusted public figures in order to execute financial investment scams targeting individual consumers.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident is caused by human fraudsters intentionally using deployed AI voice-cloning and deepfake tools to deceive victims for financial gain.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
Fraudsters are utilizing advanced AI voice and image cloning to impersonate public figures and politicians in highly convincing deepfake ads, leading to substantial economic losses of over 4 billion euros annually across Europe. While primarily a cybercrime issue rather than a direct state-sponsored national security threat, the scale of economic impact and impersonation of government officials warrants notable attention from security and regulatory agencies.
The voice-cloning and deepfake generation software operates as a tool, performing the specific function of replicating a person's voice and likeness based on direct inputs provided by the human fraudsters.
Limited Risk: The report describes AI-generated content such as deepfakes, which fall under the transparency obligations of Risk Level 3.
The AI system was used to clone the voices and images of public figures to generate highly convincing deepfake videos.
The reports provide clear, detailed accounts of how the deepfakes are created, deployed, and the specific financial harms they cause to named individuals and the broader European public. The role of the AI system (voice and image cloning) is explicitly stated, and there are no conflicting accounts.
None
Technology
Advancing Voice-Cloning Software
AI software easily clones voices to create highly convincing fake endorsements.
Evasion of Automated Filters
AI-generated ads are slightly altered to bypass automated platform detection.
Algorithmic Recommendation Loop
Platform algorithms feed more scam content to users after an initial click.
Data Inputs
Stolen Celebrity Likeness Data
Unauthorized use of public figures' voice and image data to train deepfakes.
Fabricated Trading Screen Data
Scam platforms display false profits to convince victims to invest more.
Human Factors
Trust in Well-Known Figures
Victims trust cloned images of politicians and celebrities endorsing schemes.
Manipulation by Fake Advisors
Social engineering by phone agents who pressure victims to invest more.
Process and Methods
Slow Platform Takedown Process
Meta takes days or weeks to remove flagged fraudulent deepfake ads.
Flawed Reporting Mechanisms
Reporting is managed by chatbots and limits flaggers to 20 URLs per report.
Cloaking and Ad Deactivation
Scammers run ads briefly and use fifty-plus variants to evade detection.
Management
Prioritizing Revenue Over Safety
Platforms allow paid scam ads to run while blocking private posts immediately.
Inadequate Moderation Resources
Platforms rely on chatbots and understaffed human review teams for safety.
Ending Fact-Checking Systems
Meta ended its third-party fact-checking system, reducing oversight.
Regulatory Environment
DSA Article 8 Monitoring Exemption
No general obligation for platforms to monitor third-party published content.
Lack of EU Advertiser Verification
EU regulations do not mandate pre-verification of financial advertisers.
Limited Prosecution Resources
Law enforcement agencies are overwhelmed by the volume of cross-border scams.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
