This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Scammers are increasingly using generative AI tools, such as voice cloning and AI-generated 'ghost sites', to execute highly convincing financial fraud and impersonation scams, resulting in a Georgia couple losing over $800,000.
A retired Gwinnett County, Georgia, couple reported losing nearly $800,000 after a stranger contacted Jerry Dunaway on WhatsApp and encouraged cryptocurrency trading. Police reportedly told the couple that scammers used an AI-generated "ghost site" to manipulate the account information he saw. The specific AI tool and developer were not identified. The loss was discovered when a larger withdrawal failed.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The scammers utilized an AI-generated 'ghost site' to present fabricated financial data and false investment gains to deceive the victim.
The incident involves scammers using AI voice cloning and AI-generated ghost sites to execute highly targeted financial fraud and impersonation scams against specific victims.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk is caused by malicious human actors intentionally deploying AI tools to defraud victims after the models have been trained and released.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
Widespread financial fraud and impersonation scams utilizing generative AI tools, such as voice cloning and 'ghost sites', have led to significant cumulative losses for US citizens. While representing an advanced cybercrime threat, the national security impact remains minor as it does not target critical infrastructure, state sovereignty, or systemic economic stability.
The AI systems were used by scammers as tools to perform specific functions, such as cloning voices from social media audio samples and generating fake screens for a 'ghost site' based on direct inputs.
Limited Risk: The reports describe the use of generative AI to create deepfakes, such as cloned voices and fake photographs, which fall under the category of AI-generated content requiring transparency obligations.
The reports highlight voice cloning (Voice Generation) as a primary mechanism of the AI-enabled scams, alongside the creation of fake photographs (Deepfake Image Generation) to deceive victims.
The reports provide clear, detailed accounts of a specific cryptocurrency scam targeting the Dunaways, including the exact financial loss ($802,000) and the specific role of AI (generating a 'ghost site' and cloning voices). The FBI statistics provide solid context on the broader scale of AI-enabled scams, allowing for highly confident classifications.
The incident could be viewed purely as a traditional cybercrime/phishing scam where AI was merely incidental, rather than an AI-driven safety failure.
Technology
AI Voice Cloning Tools
Scammers clone voices from social media to impersonate family members.
AI-Generated Ghost Sites
AI-generated interfaces show fake balances and buttons to deceive victims.
Generative AI Impersonation
Generative AI creates highly realistic fake photos and profiles to build trust.
Data Inputs
Social Media Audio Exposure
Publicly available audio samples are harvested to train voice-cloning models.
Personal Relationship Data
Scammers research family connections to make emergency requests seem authentic.
Spoofed Contact Information
Legitimate phone numbers are spoofed to bypass caller verification.
Human Factors
Cognitive Bias and Trust
Victims trust realistic cloned voices and professional-looking ghost sites.
Emotional Manipulation
Scammers create artificial urgency, such as fake family emergencies.
Lack of Technical Awareness
Victims are unaware of the sophistication of AI-driven scam capabilities.
Process and Methods
Ineffective Identity Verification
Lack of verification methods like family code words to confirm identity.
Weak App Authentication
Trading apps and communication platforms lack robust identity checks.
Management
Inadequate Risk Mitigation
Financial and tech companies fail to implement robust AI fraud safeguards.
Lack of User Education
Platforms do not proactively warn or educate users about AI scam tactics.
Regulatory Environment
Lack of AI Tool Regulations
Absence of strict regulations on the accessibility of voice-cloning tools.
Insufficient Platform Oversight
Weak regulatory enforcement on communication channels like WhatsApp.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
