This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A coordinated campaign utilized generative AI to create over 100 deepfake video advertisements impersonating former UK Prime Minister Rishi Sunak. These ads, which promoted a fraudulent investment scheme called 'Quantum AI' and featured spoofed BBC news content, were paid for and distributed on Meta's platforms. The campaign reached hundreds of thousands of users, raising significant concerns regarding the use of AI for election interference and financial fraud.
143 deepfake ads, over 100 of which reportedly impersonated former British Prime Minister Rishi Sunak, were promoted on Meta's platform to advertise the fraudulent investment scheme "Quantum AI." Funding for the ads reportedly originated from 23 countries. Up to 462,000 users may have been exposed to the false content. The campaign used generative AI tools to create high-quality misinformation, including spoofed BBC news clips for added legitimacy
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The coordinated campaign used en masse deepfakes of a major political figure to reach up to 400,000 people, raising significant concerns about election manipulation and public opinion influence.
The deepfake videos of Rishi Sunak and the BBC newsreader were created and promoted to deceive users into participating in a fraudulent investment scam for the perpetrators' financial gain.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by malicious human actors who intentionally created and paid to promote deepfake advertisements to scam users.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
A coordinated campaign utilized generative AI to create and distribute over 100 deepfake video ads impersonating UK Prime Minister Rishi Sunak and BBC news presenters. Reaching up to 400,000 users on Meta platforms, the incident represents a substantial information security concern, demonstrating how synthetic media can be scaled to target national leadership and undermine public trust.
The AI systems (voice and face cloning) operated as tools to perform specific, well-defined functions of generating synthetic video and audio based on direct human inputs, without exhibiting independent goal-directed behavior or autonomy.
Limited Risk: The incident involves AI-generated content such as deepfakes, which under the EU AI Act are subject to specific transparency obligations to ensure users are informed that the content is AI-generated.
The AI was used to clone the voice and face of Rishi Sunak and a BBC newsreader to create highly realistic deepfake videos.
The reports provide clear, consistent, and quantified details about the deepfake campaign, including the number of ads (143), the spend (£12,929), the reach (400,000), and the specific nature of the faked content (BBC newsreader and Rishi Sunak promoting an investment scam). There is little ambiguity about the events, though the exact AI models used are not specified.
None
Technology
Cheap Cloning Tools
AI tools enable voice and face cloning with minimal expertise.
High-Quality Deepfakes
Advanced AI generates highly realistic and deceptive video content.
Data Inputs
Public Figure Media Exploitation
Abundant public footage of Rishi Sunak used to train cloning models.
Spoofed BBC News Branding
Using trusted news branding as input data to deceive viewers.
Human Factors
Malicious Actor Intent
Fraudsters exploiting AI to deploy deceptive financial scams.
Voter Vulnerability
Difficulty for ordinary citizens to distinguish deepfakes from reality.
Process and Methods
Lax Ad Moderation Processes
Meta's automated ad screening failed to detect policy-violating deepfakes.
Delayed Threat Response
Platforms failed to swiftly remove reported illegal misinformation.
Management
Prioritizing Ad Revenue
Platform prioritizing ad monetization over robust safety enforcement.
Inadequate Safety Investment
Insufficient resources allocated to scale moderation against AI threats.
Regulatory Environment
Lagging Electoral Laws
Electoral systems struggle to keep pace with rapid AI developments.
Slow Policy Enforcement
Platforms lack strong regulatory pressure for immediate content removal.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
