This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
British broadcaster [name withheld] reported that an AI-generated deepfake video of her was used in an online advertisement to fraudulently promote weight loss pills. The video, which mimicked her appearance and voice, falsely claimed she had achieved significant weight loss using the product. [name withheld] condemned the ad as a violation of her identity and expressed concern that the public might be scammed by the fake endorsement.
An online ad reportedly used a purported AI-generated version of Ashley James, a British broadcaster and former reality television personality, to market weight loss pills through a false celebrity endorsement. James said the video copied her face and voice without consent and described the ad as both a violation of her identity and a misleading sales tactic directed at the public.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
The incident involves the unauthorized use of a public figure's likeness and voice clone to create a fake endorsement for financial gain through a weight loss pill scam.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by malicious human actors intentionally deploying a deepfake video to scam the public using a post-deployment AI tool.
Added categories peak at 2: Minor (Psychological Harm, Direct)
An AI-generated deepfake of a British broadcaster was used in an online weight loss scam. While representing a violation of individual identity and a tool for commercial fraud, the incident has negligible national security implications and does not target critical infrastructure, government functions, or state sovereignty.
The AI system was used as a tool to generate a realistic video and voice clone of [name withheld] based on direct inputs, performing a specific function without autonomous planning or decision-making.
Limited Risk: The system involves AI-generated content such as deepfakes, which pose a moderate risk and require transparency obligations to ensure users are informed of the AI's nature.
The AI system was used to generate a highly realistic video and voice clone of [name withheld] to mimic her appearance and speech.
The report clearly details the nature of the incident, the victim's response, and the medium used (AI-generated video and voice). There is no ambiguity about the fraudulent nature of the ad or the impact on the victim.
None
Technology
Realistic Voice and Image Synthesis
AI tools allowed highly realistic replication of the presenter's likeness.
Deepfake Generation Tools
Accessible AI technology enabled the creation of convincing fake advertisements.
Data Inputs
Unauthorized Likeness Training Data
Publicly available video and audio of the presenter were used without consent.
Human Factors
Targeting of Trusting Audiences
Scammers exploited public trust in the presenter to sell weight loss products.
User Susceptibility to Deepfakes
The AI ad was realistic enough that viewers assumed it was genuine.
Process and Methods
Inadequate Platform Moderation
Social media platforms allowed the distribution of unverified deepfake ads.
Flawed Ad Review Procedures
Platform ad approval processes failed to detect the fraudulent deepfake content.
Management
Platform Profit Over Safety
Social media companies prioritized ad revenue over rigorous safety checks.
Regulatory Environment
Lack of Deepfake Legal Recourse
Unclear legal frameworks make it difficult to stop or penalize deepfake ads.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
