This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A deepfake video impersonating a consumer adviser was circulated on social media to promote an auto-insurance quote tool. Victims who interacted with the site reported receiving a flood of unsolicited calls without receiving any actual financial benefit. The incident highlights the ease with which AI can be used to create deceptive content for commercial fraud.
A purported AI-generated video reportedly circulated on social media depicting consumer adviser Clark Howard endorsing an auto-insurance quote tool. A viewer reportedly said she believed it was real and warned others after a similar site led to an alleged flood of unsolicited calls without better rates.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
The incident involves malicious actors using AI-generated deepfakes of a trusted consumer adviser to run an insurance scam, which falls under fraud, scams, and targeted manipulation.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human actors intentionally using deployed AI deepfake generation tools to create deceptive videos for commercial fraud.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
This incident involves the use of AI-generated deepfakes to impersonate a local television consumer adviser for a commercial insurance scam. While demonstrating the increasing ease of creating convincing synthetic media, the incident is a localized cybercrime and fraud issue with minor national security implications, manageable under standard law enforcement and consumer protection frameworks.
The AI system was used to generate synthetic video and voice (deepfakes) based on direct inputs and templates, acting as a tool for content generation without autonomous planning or decision-making capabilities.
Risk Level 3: Limited Risk. The report describes 'AI-created deepfakes' used to impersonate a real person, which falls under the category of AI-generated content requiring transparency obligations under the EU AI Act.
The AI was primarily used to generate a deepfake video of the consumer adviser, which also involved generating a synthetic voice to deliver the scam pitch.
The reports clearly describe the nature of the incident, the specific harm (impersonation, spam calls, deception), and the technology used (deepfakes). There is little ambiguity regarding the events, though the specific AI model used is not identified.
None
Technology
Ease of Deepfake Creation
AI tools make it highly accessible and simple to generate realistic fakes.
High Realism of AI Avatars
Deepfakes are realistic enough to easily deceive average social media users.
Data Inputs
Use of Public Voice and Video
Publicly available media of trusted figures is used to train deepfakes.
Human Factors
User Trust in Public Figures
People trust advice from recognized figures, making them vulnerable to scams.
Difficulty in Verifying Authenticity
Audiences struggle to distinguish real videos from AI-generated deepfakes.
Process and Methods
Inadequate Ad Screening
Social media platforms lack robust verification to block deepfake scam ads.
Management
Inadequate Risk Mitigation
Platforms prioritize ad revenue over proactively scanning for deepfake scams.
Regulatory Environment
Lack of Deepfake Regulations
Absence of strict policies or laws governing AI-generated deceptive ads.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
