This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Fraudulent social media advertisements are using AI-generated deepfake videos of an Australian health advocate to promote fake cures for prostate issues and erectile dysfunction. These videos, which exhibit signs of manipulation such as audio-visual desynchronization, direct users to phishing websites or sites selling unverified health supplements. The impersonated individual has denied involvement, and the scam has been observed circulating widely across several African countries.
Deepfake videos of Barbara O’Neill, an Australian alternative health advocate banned from giving medical advice, have allegedly been used in fraudulent social media ads promoting false health cures. They include treatments for prostate issues and erectile dysfunction. These manipulated videos have been widely circulated across African countries. The scam directs users to deceptive websites, potentially for phishing or financial fraud. O’Neill has denied involvement.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The scam utilizes phishing websites designed to harvest personal information from unsuspecting users.
The deepfake videos spread false health claims and medical misinformation regarding prostate cures.
The incident involves scammers using AI-generated deepfakes of a public figure to deceive social media users for financial gain through phishing and selling unverified supplements.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk is caused by human scammers intentionally deploying AI deepfake tools post-deployment to defraud social media users.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Fraudulent advertising campaigns utilized AI-generated deepfakes of an Australian health advocate to promote fake medical cures and direct users to phishing sites across several African nations. The incident represents a minor national security concern, categorized primarily as consumer fraud and localized cybercrime rather than state-sponsored information warfare.
The report describes scammers using 'artificial intelligence (AI) tools' to generate deepfake videos and audio impersonations. These systems function as tools that perform specific, well-defined functions of video and voice synthesis based on direct inputs.
Limited Risk: The report describes the creation and dissemination of 'AI-generated impersonations' and 'deepfakes', which fall under the transparency obligations of Risk Level 3.
The AI systems were used to generate deepfake videos and synthetic audio impersonating the speaker to promote fraudulent products.
The fact-checking report from Africa Check provides clear, reliable details about the nature of the deepfake scam, the platforms used, the geographic reach, and the specific deceptive tactics employed. While the exact AI models are not named, this does not impede the classification of the incident as a deepfake-enabled scam.
None.
Technology
Accessible AI Deepfake Tools
AI tools enable rapid creation of low-quality but convincing voice clones.
Audio-Video Desynchronization
AI-generated audio fails to sync perfectly with original video movements.
Robotic Audio Synthesis
Synthesized voices sound unnatural and robotic in certain segments.
Data Inputs
Abundant Public Lecture Footage
Large volumes of online lecture videos provide ample source data for training.
Unprotected Personal Identity Data
Publicly available audio clips of the speaker are harvested to train AI.
Human Factors
User Susceptibility to Scams
Audiences trust familiar figures and fall for sensational health claims.
Targeting Vulnerable Demographics
Scammers exploit people seeking quick cures for sensitive health issues.
Overreliance on Social Media Content
Users assume hosted advertisements on major platforms are pre-screened.
Process and Methods
Ineffective Ad Moderation
Social media platforms fail to detect and block deepfake scam ads.
Lack of Proactive Content Filtering
Platforms rely on reactive reporting rather than proactive AI detection.
Facile Account Creation Processes
Scammers easily set up multiple small accounts to run coordinate ads.
Management
Flawed Ad Approval Process
Facebook allowed coordinated ad campaigns using deepfakes to run unchecked.
Inadequate Content Takedown Speed
Platform management failed to quickly remove reported deepfake scam ads.
Regulatory Environment
Ineffective Online Ban Enforcement
Authorities banned the speaker but cannot stop her content spreading online.
Lack of Platform Accountability
No regulatory framework holds social media liable for hosting deepfake scams.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
