This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A Tech Transparency Project investigation revealed that 63 scam advertisers used generative AI to create deepfakes of prominent political figures to promote fraudulent government benefit programs on Meta platforms. These advertisers, who collectively spent $49 million on ads, targeted vulnerable groups like seniors. Despite Meta's policies against deceptive practices, many of these scam accounts remained active for extended periods, generating significant revenue for the platform before being removed.
A Tech Transparency Project investigation identified purportedly AI-generated deepfake ads on Facebook impersonating President Trump, Elon Musk, Rep. Alexandria Ocasio-Cortez, Senators Elizabeth Warren and Bernie Sanders, and Press Secretary Karoline Leavitt. The ads allegedly promoted fake $5,000 government rebates and similar scams, reportedly misleading users and generating ad revenue for Meta before removal.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The AI-generated deepfakes were used to promote illegal financial scams, exposing users to deceptive and harmful content.
The incident involves malicious actors using AI-generated deepfakes of public figures to conduct targeted financial scams and fraud against seniors.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human scammers intentionally using deployed generative AI tools to create deepfakes for financial fraud.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
Scammers utilized generative AI to create highly realistic deepfakes of prominent U.S. political figures to run a massive, 49 million dollar fraudulent ad campaign targeting seniors on Meta platforms. While the incident represents a significant evolution in AI-enabled financial fraud and public deception, its national security impact remains minor as it does not target critical infrastructure, state sovereignty, or national economic systems.
The report describes scammers using generative AI as a tool to synthesize realistic videos and voices of politicians. The AI performed the specific function of content generation based on direct inputs from the scammers, without exhibiting autonomous planning or decision-making.
Limited Risk: The incident involves the creation and dissemination of AI-generated content, specifically deepfakes of political figures, which falls under the transparency obligations of this level.
The AI was primarily used to generate highly realistic deepfake videos and synthetic voices of politicians to deceive users.
The reports from the Tech Transparency Project, The New York Times, and other sources provide detailed, consistent, and quantified evidence of the scam campaigns, including ad spend, platforms used, specific politicians deepfaked, and the targeting of seniors. The role of generative AI in creating the deepfakes is clearly established.
None. The evidence clearly points to intentional financial fraud using AI-generated deepfakes.
Technology
Generative AI Deepfakes
Scammers used AI to create realistic fake videos of politicians.
Automated Ad Approval System
Meta relied on automated checks that failed to detect deepfake scams.
Lack of Alteration Detection
Platform lacked technical tools to automatically identify manipulated media.
Data Inputs
Unverified Advertiser Credentials
Advertisers submitted unverified or foreign IDs to bypass political checks.
Missing Digital Disclosures
Scam ads lacked mandatory disclosures for digitally altered content.
Deceptive Landing Page Data
Ads linked to external sites that gathered user info and credit cards.
Human Factors
Targeting of Vulnerable Seniors
Scammers exploited older adults using trusted political figures.
Public Confusion on Benefits
Confusion around social safety net programs made scams more believable.
User Trust in Political Figures
Victims trusted the deepfaked politicians appearing in the video ads.
Process and Methods
Lax Ad Moderation Process
Meta allowed scam ads to run for days or weeks before taking action.
Excessive Account Strike Policy
Meta allowed up to 32 automated strikes before banning scammer accounts.
Crowdsourced Enforcement Model
Meta waited for user reports instead of proactively blocking scam ads.
Management
Prioritizing Revenue Over Safety
Meta focused on short-term ad revenue over long-term platform safety.
Reduced Platform Safety Teams
Meta rolled back moderation efforts and downsized safety divisions.
Inadequate Advertiser Vetting
Management failed to implement rigorous verification for political ads.
Regulatory Environment
Outdated AI Laws
Laws and consumer protections for rapid AI advances are outdated.
Platform Liability Disclaimers
Meta argued in court that it does not owe a duty to protect users.
Blocked Subscription Rules
Court rulings blocked FTC efforts to crack down on negative option scams.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
