This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
In 2023, Australian consumers lost over $8 million to investment scams that utilized AI-generated deepfake videos and fake news articles to impersonate celebrities. The National Anti-Scam Centre received over 400 reports of these incidents, which involved fraudulent trading platforms such as Quantum AI and Immediate Edge. Victims were lured into providing personal details or investing cryptocurrency, often resulting in total financial loss or persistent harassment.
In 2023, Australians reportedly lost over $8 million to scams involving purported deepfake videos and fake news articles that falsely endorsed investment trading platforms. Scammers reportedly used AI-generated content featuring celebrities to mislead victims, leading to significant financial losses. The National Anti-Scam Centre received over 400 reports of these incidents. One man is reported to have lost over $80,000 in cryptocurrency.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
The incident involves scammers using AI-generated deepfakes and fake news articles to impersonate trusted public figures for illegitimate financial benefit.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The harm was caused by human scammers intentionally deploying AI-generated deepfakes and fake articles to defraud victims.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Australian consumers lost over $8 million in 2023 to investment scams leveraging AI-generated deepfakes of public figures. While the incident represents a minor national security threat, it highlights the growing capability of malicious actors to exploit synthetic media for widespread financial fraud and social manipulation.
The AI systems (deepfake generators and text generators) were used as tools by human scammers to produce specific, synthetic media outputs (videos and articles) based on direct inputs, without exhibiting autonomous planning or decision-making capabilities.
Limited Risk: The incident involves 'AI-generated content such as deepfakes', which falls under Risk Level 3 due to transparency obligations requiring users to be informed they are interacting with AI-generated media.
The AI was primarily used to generate deepfake videos of celebrities and fake news articles to deceive victims on social media.
The report provides clear details on the nature of the scams, the role of AI-generated deepfakes and fake articles, the financial losses, and specific victim accounts. The role of the AI as a tool for fraud is unambiguous.
None
Technology
Deepfake Video Generation
AI-generated video of Elon Musk used to deceive victims into investing.
Fake Article Generation
Algorithmic generation of fake news articles featuring celebrities.
Data Inputs
Unauthorised Celebrity Likeness
Scammers used public images and videos of celebrities without consent.
Human Factors
Trust in Celebrity Endorsements
Victims believed fake endorsements from trusted public figures.
High-Pressure Sales Tactics
Scammers pressured victims to invest immediately once contact was made.
Process and Methods
Flawed Social Media Ad Vetting
Social media platforms allowed deepfake scam ads to be published.
Inadequate Reporting Response
Slow response to user reports allowed fake ads to remain active.
Management
Prioritising Ad Revenue
Platforms prioritized ad revenue over rigorous safety checks for ads.
Regulatory Environment
Weak Deepfake Regulations
Lack of strict rules governing AI-generated deepfakes and scams.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
