This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A widespread fraudulent campaign used AI-generated deepfake videos of various celebrities to promote fake Le Creuset cookware giveaways on social media platforms like Facebook, Instagram, and TikTok. The ads directed users to phishing websites where they were tricked into paying a small shipping fee, which actually enrolled them in unauthorized, expensive monthly subscription plans. Victims never received the promised cookware and suffered financial loss and potential identity theft.
Scammers reportedly made deepfakes of Taylor Swift, Selena Gomez, Joanna Gaines, Lainey Wilson, Ree Drummond, Oprah, Jennifer Lopez, Trisha Yearwood, Martha Stewart, and Blake Shelton promoting a Le Creuset giveaway. These AI-generated ads, appearing on Meta and TikTok, falsely claimed users could receive free cookware by paying a small shipping fee. Victims were unknowingly enrolled in a costly monthly subscription.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The unauthorized replication and exploitation of the celebrities' public likenesses and voices devalued their personal brands and control over their own likeness.
The incident involved scammers using AI voice-cloning technology to impersonate trusted celebrities for the purpose of defrauding social media users of their money.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by malicious human actors who intentionally deployed AI voice-cloning tools to perpetrate a financial scam.
Added categories peak at 3: Substantial (Epistemic Harm, Indirect)
This incident involved a widespread consumer scam utilizing AI-generated celebrity deepfakes to perpetrate financial fraud. While it highlights the growing sophistication of synthetic media, it represents a commercial crime and consumer protection issue rather than a threat to national security, resulting in negligible national security impact.
The AI voice-cloning and text-to-speech software operated as a tool, performing the specific function of translating text scripts into synthetic celebrity voices based on direct inputs without autonomous decision-making.
Limited Risk: The report describes the use of AI to generate deepfakes, which falls under the category of AI-generated content requiring specific transparency obligations under the Act.
The AI system's primary function in this scam was to generate a synthetic clone of Taylor Swift's voice from training data to deliver a fabricated script, which was then paired with existing video footage.
The reports are highly consistent, originate from reputable news outlets, and clearly detail the mechanics of the scam, the platforms involved, and the impact on victims. There is minimal ambiguity regarding the role of AI or the nature of the harm.
None
Technology
Accessible Text-to-Speech Tools
Free and cheap tools allow anyone to clone realistic celebrity voices quickly.
Advanced Lip-Syncing Software
Programs seamlessly sync cloned audio with existing video footage.
Content Cloaking Techniques
Scammers cloak ad content to evade automated platform detection systems.
Data Inputs
Abundant Public Voice Data
Large amounts of celebrity audio online are used to train voice clones.
Stolen Brand Assets
Scammers used legitimate logos and media to build highly convincing fake sites.
Celebrity Training Footage
Existing videos of celebrities are used as templates for deepfake overlays.
Human Factors
Parasocial Fan Trust
Fans trust celebrity likenesses, making them vulnerable to deceptive ads.
Inability to Spot Deepfakes
Highly realistic AI makes it nearly impossible for users to detect fraud.
Urgency-Driven Decision Making
Fake limited-supply warnings pressure users into paying without verification.
Process and Methods
Evaded Platform Review Systems
Social media review processes failed to catch cloaked fraudulent ads.
Abuse of Account Verification
Verified accounts and sponsored labels lent fake legitimacy to scam ads.
Deceptive Subscription Billing
Hidden terms and recurring fees were buried in fine print on external sites.
Management
Inadequate Ad Pre-Screening
Platforms allowed unverified advertisers to run celebrity-themed ads.
Lack of Endorsement Verification
Platforms failed to verify contractual endorsement rights before hosting ads.
Delayed Ad Takedown Response
Scam ads accumulated thousands of views before being removed by platforms.
Regulatory Environment
No Federal AI Scam Laws
Absence of federal legislation limits legal action against deepfake creators.
International Jurisdictions
Scammers operating from foreign countries evade domestic law enforcement.
Inconsistent State Laws
Only a few states regulate AI-generated content, leaving regulatory gaps.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
