This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Scammers are increasingly using sophisticated AI-generated deepfakes of prominent Australian politicians, business leaders, and journalists to promote fraudulent investment schemes, resulting in millions of dollars in losses.
An online article reportedly used AI-generated imagery to falsely depict Australian Broadcasting Corporation (ABC) finance presenter Alan Kohler interviewing Reserve Bank of Australia governor Michele Bullock about Boulder Sparhaven, a cryptocurrency trading platform not licensed in Australia. The Australian Securities and Investments Commission (ASIC) cited it as an example of increasingly sophisticated AI-assisted investment scams targeting Australians.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The scams rely on fabricated news articles, fake interviews, and false endorsements to mislead victims.
The unauthorized reproduction of public figures' likenesses and voices devalues their professional reputation and trust.
The incident involves scammers using AI-generated deepfakes of public figures to deceive individuals into investing in fraudulent schemes for financial gain.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident is caused by human scammers intentionally using deployed generative AI tools to create deceptive deepfakes and defraud victims.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
The incident involves the widespread deployment of AI-generated deepfakes of Australian leaders and public figures to conduct highly effective financial scams. While causing millions of dollars in losses and minor epistemic harm to public trust, the incident remains a regulatory and law enforcement issue rather than a direct threat to national security.
The AI systems are used as tools by human scammers to generate synthetic media (deepfake videos and text) based on direct inputs, without exhibiting autonomous goal-directed planning or independent decision-making.
Limited Risk: The report describes the creation of AI-generated content such as deepfakes, which falls under Risk Level 3 due to transparency obligations to ensure users are aware they are interacting with AI-generated content.
The reports repeatedly highlight the use of generative AI to create deepfake videos and images of celebrities and politicians to promote fake investment schemes.
The reports from ASIC and major news outlets provide clear, consistent, and detailed accounts of the deepfake investment scams, including specific figures impersonated, financial loss statistics, and regulatory actions. There is little ambiguity regarding the nature of the AI's involvement or the harm caused.
None
Technology
Generative AI Deepfakes
AI makes fake videos and images highly convincing and hard to detect.
Automated Scam Ad Approval
Ad systems on social media platforms approve fraudulent deepfake ads.
Offshore Platform Operations
Platforms operate offshore, making technical enforcement and takedowns harder.
Data Inputs
Stolen Public Likenesses
Scammers use images and voices of trusted public figures without consent.
Fabricated News and Reviews
AI-generated fake articles and testimonials create a web of deception.
Misused AFSL Details
Scammers copy legitimate license numbers to bypass basic verification.
Human Factors
Trust in Public Figures
Victims trust familiar celebrities and politicians endorsing the schemes.
Difficulty Verifying Legitimacy
Polished AI content makes it nearly impossible for users to spot fakes.
Overreliance on Search Engines
Users rely on simple online searches which lead to spoofed websites.
Process and Methods
Whack-a-Mole Takedowns
Scammers quickly recreate removed accounts and websites under new names.
Inadequate Ad Screening
Social media ad screening processes fail to block deepfake scam ads.
Ineffective AFSL Checks
Standard license checks are bypassed by sophisticated spoofing methods.
Management
Prioritizing Ad Revenue
Meta and other platforms monetize and approve scam ads without strict vetting.
Slow Platform Response
Platforms take down reported accounts slowly, allowing scams to reappear.
Inadequate Proactive Prevention
Platforms fail to implement robust systems to prevent scam ads beforehand.
Regulatory Environment
Lack of Platform Penalties
Absence of heavy fines for platforms hosting fraudulent AI-generated ads.
Incomplete Regulatory Framework
Scams protection framework is still being developed to hold platforms liable.
Offshore Jurisdictional Limits
Regulators struggle to enforce laws on platforms operating from offshore.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
