This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A deepfake video circulating on Facebook impersonated a British Labour leader to promote a fraudulent investment scheme. The video used AI-cloned audio and manipulated footage from a real address to deceive viewers into believing the government was offering a passive income opportunity. Experts identified telltale signs of synthetic media, such as poor lip-syncing and unnatural speech patterns. The incident highlights the growing challenge of deepfake misinformation on social media platforms.
A deepfake video was circulating around social media of British Labour leader Keir Starmer touting an investment scheme.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The video generated and spread false information about a non-existent government-backed investment scheme.
The deepfake video was created to impersonate a trusted public figure for the purpose of promoting a fraudulent investment scam to obtain financial benefit.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human actors who intentionally created and distributed a deepfake video to defraud social media users using post-deployment AI tools.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
A deepfake video impersonated UK Labour leader Keir Starmer to promote a fraudulent investment scam on Facebook. While illustrating the growing accessibility of voice cloning tools and their potential to erode trust in public figures, the incident is a financially motivated scam with minor national security implications.
The AI system operated as a tool, performing the specific function of voice cloning and video manipulation based on direct inputs provided by human creators without autonomous decision-making.
Risk Level 3: Limited Risk. The incident involves AI-generated content such as deepfakes, which are subject to specific transparency obligations under the Act.
The AI system was used to generate a deepfake video and clone the voice of a politician to impersonate him for a scam.
The report clearly details the nature of the deepfake, the debunking process, and the expert analysis of the synthetic media. There is no ambiguity regarding the fact that the video is a deepfake scam.
None.
Technology
Voice Cloning AI Technology
AI tools easily clone voices to convincingly imitate real public figures.
Poor Lip-Syncing Generation
AI struggle to generate natural mouth and teeth looks, causing blurriness.
Data Inputs
Use of Real Public Video Footage
Real footage from the 2023 address was used as input to overlay deepfake.
Lack of Input Verification
No mechanism to verify authenticity of input media used for social sharing.
Human Factors
Rapid Spread of Misinformation
Users sharing unverified realistic deepfakes quickly across social media.
Difficulty in Detecting Deepfakes
Voice cloning is becoming increasingly hard for average users to detect.
Process and Methods
Inadequate Platform Moderation
Social platforms struggle to respond quickly to verified deepfake content.
Verification Challenges
New technology exposes increased challenges for rapid content verification.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
