This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A widespread campaign of deepfake scam advertisements has been circulating on Meta platforms, impersonating trusted medical professionals and celebrities to sell unproven health supplements. These AI-generated videos have led to significant financial fraud, including unauthorized recurring charges, and have caused serious health risks by convincing patients to abandon prescribed medications for conditions like diabetes. Despite reports from victims and public figures, platforms like Meta have faced criticism for slow response times and inadequate content moderation regarding these sophisticated, industrial-scale scams.
According to an ABC News (Australia) 7.30 report, a second wave of deepfake scam ads impersonating Norman Swan and other public figures circulated widely on Meta platforms in 2025, promoting unproven health supplements. The campaign featured new voice-cloned videos and more impersonation tactics, and it reportedly contributed to financial and health harms. Investigators reportedly linked the ads to businesses in Australia and New Zealand using deceptive redirects and alias branding.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The deepfakes spread false health advice, leading a diabetes patient to stop taking critical prescribed medication.
The incident involves scammers using AI-generated deepfakes of trusted figures to deceive specific consumers for financial gain and unauthorized credit card charges.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The deepfake videos were intentionally created and deployed by human scammers to defraud consumers and promote unproven products.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
A widespread deepfake scam campaign on Meta platforms targeted Australian consumers, impersonating medical professionals to sell unproven supplements. While causing individual health risks and financial losses, the incident represents a low-level consumer fraud threat rather than a direct challenge to national security.
The AI deepfake software acts as a tool, performing the specific, deterministic function of cloning voices and mapping faces based on direct user inputs and scripts without autonomous decision-making.
Limited Risk: The report describes the use of AI-generated content such as deepfakes, which falls under Risk Level 3 and is subject to transparency obligations under the EU AI Act.
The AI system was used to clone the voice and appearance of doctors and celebrities to generate realistic deepfake videos.
The reports provide detailed, first-hand accounts from both the impersonated medical professionals and the victims who fell for the scams. The technical process of creating the deepfakes is explained, and the financial and health impacts are well-documented.
None. The incident is clearly a coordinated deepfake-enabled financial and health scam.
Technology
Accessible Deepfake Software
Low-cost software allows easy cloning of voices and faces of trusted figures.
Sophisticated AI Generation
AI models map faces and clone voices from short video samples accurately.
Exploitation of Ad Algorithms
Scammers leverage Meta's targeting algorithms to reach vulnerable users.
Data Inputs
Public Source Media Use
Scammers easily scrape public YouTube videos to train deepfake models.
Lack of Media Verification
Meta's ad system lacks verification of the media assets uploaded for ads.
Human Factors
Trust in Authority Figures
Victims trust recognized medical experts, making them vulnerable to scams.
Vulnerability of Patients
People with chronic health issues are easily targeted by weight loss ads.
Cognitive Bias Toward Realism
Highly convincing deepfakes make it difficult for users to detect fraud.
Process and Methods
Weakened Content Moderation
Meta's January 2025 policy change led to increased harmful content.
Ineffective Takedown Process
Meta's slow response to user reports allows scam ads to remain active.
Inadequate Ad Screening
Meta fails to proactively filter deepfake ads before they go live.
Management
Prioritizing Ad Revenue
Meta continues to accept ad revenue from scam accounts running deepfakes.
Inadequate Risk Assessment
Failure to evaluate public safety risks before changing moderation policies.
Regulatory Environment
Delayed Regulatory Penalties
Fines for platforms failing to remove scams are not yet fully enforced.
Platform Liability Loophole
Platforms claim they are not publishers to avoid liability for scam ads.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
