This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A network of TikTok accounts has been identified using AI-generated deepfake influencers and fabricated personal testimonials to promote unverified wellness and beauty products. These accounts employ deceptive tactics, such as creating fake medical backstories and targeting vulnerable users with false claims about fertility treatments and PCOS management. The investigation highlights an evolution in scam sophistication, moving from simple stock photo manipulation to the use of AI-generated personas to drive product sales.
A network of TikTok accounts reportedly used AI-generated and deepfake influencer personas to promote wellness and beauty products through alleged deceptive advertising. The accounts reportedly invented identities and personal testimonials while making unsubstantiated medical claims, including false assertions related to fertility and hair-growth treatments. Some personas were reportedly reused across multiple accounts with inconsistent backstories. The content purportedly targeted vulnerable users and linked to unverified products before the accounts were removed.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The deepfake influencers and slideshows spread unverified and false health claims regarding PCOS treatments and hair growth.
The incident involves scammers using AI-generated deepfake influencers and fabricated stories to deceive users into purchasing unverified wellness products for financial gain.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human scammers who intentionally deployed AI-generated deepfake influencers post-deployment to deceive TikTok users and sell wellness products.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
A network of TikTok accounts utilized AI-generated deepfake influencers to execute deceptive commercial wellness scams targeting vulnerable individuals. While representing an evolution in consumer fraud sophistication, the incident has negligible national security implications and does not target critical infrastructure, government functions, or state stability.
The AI system was used as a tool by scammers to generate synthetic images and videos of influencers. The report describes the technology as 'deepfake technology' used to create specific outputs based on user inputs, without autonomous decision-making or agency.
Limited Risk: The report describes the use of 'AI-generated content such as deepfakes' which falls under Limited Risk due to transparency obligations to inform users they are interacting with AI-generated media.
The scammers primarily used AI to generate deepfake videos of influencers to act as salespeople, as well as synthetic images for slideshows.
The report from Media Matters provides clear, documented examples of specific TikTok accounts, their deceptive strategies, and the types of AI-generated content used. While the exact AI models are not named, the nature of the deepfake technology and its application in the scam is highly transparent.
None
Technology
Sophisticated Deepfake Video
AI tools enable creation of highly realistic talking deepfake influencers.
Generative AI Image Creation
AI generates realistic images of people for fake storytime slideshows.
Data Inputs
Unlabeled AI Content
Deepfake media lacks clear labels indicating it is AI-generated.
Stolen Likeness Data
Using real people's images to anchor or generate deepfake personas.
Human Factors
Trust in Wellness Buzzwords
Users are easily misled by health claims and familiar story formats.
Exploitation of Vulnerabilities
Scammers target women with fertility issues using emotional stories.
Process and Methods
Weak Automated Detection
TikTok's automated filters failed to flag coordinated deepfake networks.
Coordinated Account Networks
Scammers easily deploy the same AI content across multiple accounts.
Management
Reactive Moderation Policies
TikTok only removed the scam accounts after media reports exposed them.
Regulatory Environment
Inadequate AI Ad Regulations
Lack of strict laws governing AI-generated influencers in advertising.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
