This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Multiple TikTok accounts using AI deepfake technology have been created to impersonate fictional nieces of French politician Marine Le Pen. These accounts, which have amassed tens of thousands of followers, promote far-right political parties and have been used to solicit donations from users. The incident has raised significant concerns regarding political manipulation, the spread of misinformation ahead of EU elections, and the violation of personality rights through the unauthorized use of likenesses.
A TikTok account, "Amandine Le Pen," created using AI deepfake technology, impersonated a fictional niece of Marine Le Pen, amassing over 30,000 followers. The account spread pro-RN messages and solicited donations, misleading users and exploiting political influence. Visual inconsistencies revealed the deepfake, raising concerns about AI misuse for political manipulation, identity theft, and violation of personal rights, especially with similar accounts proliferating, such as "Lena Maréchal Lepen."
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The deepfake videos featured sexually suggestive content and xenophobic comments mocking people of color.
The accounts used fake personas to solicit financial donations from followers via virtual coffee purchases.
The deepfake accounts were used to conduct a coordinated campaign aimed at influencing public opinion and political processes ahead of the EU elections.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human creators who intentionally generated and deployed deepfake videos of fictional Le Pen family members to manipulate public opinion and solicit donations.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
This incident involved a coordinated political disinformation campaign in France using AI-generated deepfakes of fictional relatives of politician Marine Le Pen. The accounts amassed millions of views, spreading xenophobic content and far-right propaganda ahead of the EU elections, highlighting the evolving threat of synthetic media in democratic processes.
The AI was used as a tool by human creators to generate deepfake videos by editing faces onto stolen footage. The system performed a specific, well-defined function based on direct human input without autonomous planning or decision-making.
Limited Risk: The EU AI Act classifies AI-generated content such as deepfakes under Limited Risk, which imposes specific transparency obligations to ensure users are informed that they are interacting with AI-generated content.
The primary technology used was deepfake video generation to map juvenile faces of political figures onto stolen videos of influencers, which were then posted as social media content.
The reports from RTS and Euractiv provide consistent, detailed accounts of the deepfake accounts, their reach, the specific content of the videos, and the political context. The facts regarding the creation of fictional Le Pen family members and the use of stolen influencer videos are clearly established.
The campaign could be interpreted purely as a financial scam rather than a coordinated political disinformation campaign, though the heavy use of political hashtags and election references strongly supports the political interpretation.
Technology
Generative AI Face Editing
AI tools allowed editing faces onto stolen influencer videos easily.
Weak Deepfake Detection
TikTok algorithms failed to detect facial and mouth movement anomalies.
Data Inputs
Stolen Influencer Videos
Real videos of influencers were used without consent as base inputs.
Unlabeled AI Content
Synthesized videos lacked metadata or labels indicating AI generation.
Human Factors
Malicious AI Exploitation
Anonymous users created fake personas to manipulate political views.
User Gullibility
Viewers believed the fake Le Pen family members were real people.
Process and Methods
Slow Content Moderation
TikTok failed to remove or label the viral deepfakes in a timely manner.
Lack of Proactive Screening
Platforms relied on manual reports rather than automatic detection.
Management
Poor Platform Oversight
TikTok management did not prioritize election-related deepfake risks.
Regulatory Environment
Delayed DSA Implementation
France lagged in designating trusted flaggers and coordinating bodies.
Weak Legal Protections
MPs voted against banning deepfakes published without consent.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
