This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
This report details the rise of AI-enabled fraud, where scammers leverage language models, voice cloning, and synthetic media to conduct sophisticated social engineering and identity theft. Financial institutions are responding by deploying their own AI-driven verification tools to detect anomalies in user behavior and credentials. Despite these efforts, experts note that criminals are currently outpacing defensive measures, leading to record-breaking financial losses for consumers.
Scammers are reportedly using AI tools such as language models, voice cloning, and synthetic IDs to create more convincing frauds, leading to financial losses and identity theft. Banks have begun deploying AI-driven verification tools to counter these schemes, but experts warn that AI-enabled fraud continues to cause real-world harm and remains difficult to detect.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
Scammers use AI tools to cross-reference leaked passwords and unearth sensitive personal details from data breaches and social media.
The incident involves malicious actors using AI systems to conduct highly targeted financial fraud, impersonation, and scams for personal gain.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk is driven by human scammers intentionally deploying AI tools post-deployment to execute financial fraud.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
The rapid rise of AI-enabled fraud represents a substantial economic security threat, with scammers leveraging voice cloning and LLMs to bypass traditional verification. This has resulted in a record $10 billion in losses in 2023, forcing major financial institutions to deploy advanced defensive AI systems to maintain public trust and secure transactions.
The report describes scammers using ChatGPT and other AI software as assistants to augment their capabilities, making them better writers and more convincing conversationalists, and helping them adapt schemes in real time.
Limited Risk: The report describes the use of chatbots like ChatGPT and AI-generated content such as voice clones and deepfakes, which fall under Risk Level 3 due to transparency obligations for interactive AI and synthetic media.
The AI systems are primarily used as writing assistants to generate highly convincing, personalized scam messages, and as voice generators to clone victims' voices for impersonation.
The report provides clear, detailed accounts of how AI is being used to facilitate financial fraud, citing statistics from the FTC and specific personal anecdotes. The role of AI as an enabler of these scams is explicitly discussed, and the classifications align well with the provided taxonomies.
None. The report is clearly focused on AI-enabled financial fraud.
Technology
AI Voice and Identity Cloning
AI mimics executive voices and identities to request wire transfers.
Generative Text for Phishing
LLMs eliminate spelling errors, making scam messages highly convincing.
Automated Credential Stuffing
AI systems automate testing of stolen passwords across multiple platforms.
Data Inputs
Exploitation of Personal Data
Scammers use AI to mine social media and data breaches for target details.
Synthetic Identity Data
AI generates computer-generated faces to bypass bank verification systems.
Human Factors
Cognitive Overload and Urgency
Victims are manipulated by artificial urgency, ignoring potential red flags.
Emotional Desperation
Job seekers or desperate targets ignore email address discrepancies.
Over-reliance on Spidey Senses
Traditional warning signs like poor grammar are no longer there to warn users.
Process and Methods
Outpaced Prevention Methods
Criminals adopt AI tools faster than banks develop AI defensive measures.
Weak Identity Verification
Traditional ID checks fail against AI-generated faces and fake licenses.
Lack of Multi-Factor Auth
Failure to enable 2FA allows credential stuffing attacks to succeed.
Management
Delayed Defensive AI Investment
Banks lag behind criminals in deploying AI-based fraud detection.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
