This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A report detailing the use of AI-generated deepfakes in Bulgaria to impersonate a physician and a television host for the promotion of fraudulent medical products. The article discusses the increasing difficulty of distinguishing AI-manipulated media and notes that such technology is being used for financial scams and hybrid warfare. It also references other high-profile deepfake incidents involving world leaders and tech executives to illustrate the scale of the issue.
A purportedly AI-manipulated scam advertisement circulating online appeared to use the likenesses and altered media of Bulgarian television host Simeon Ivanov and physician Spas Spaskov to promote an unregistered purported joint-treatment product. They reportedly denied involvement, and the product was described in reporting as ineffective.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The report highlights how deepfakes are being used as weapons of hybrid warfare, referencing fake videos of world leaders like Putin and Zelenskyy to influence public opinion.
The primary incident involves malicious actors using AI-generated deepfakes of public figures to run fraudulent advertisements and scam victims for financial gain.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The harm was caused by malicious human actors intentionally deploying AI deepfake technology post-deployment to commit financial fraud.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
The incident involves the use of AI-generated deepfakes in Bulgaria to impersonate a doctor and TV host for commercial fraud. While the report references global concerns about hybrid warfare and large-scale scams, the immediate national security impact of this specific incident is minor, representing localized criminal activity rather than a state-level threat.
The report describes AI being used to clone voices and generate deepfake images and videos based on existing footage and photos. The AI operates as a tool performing specific generation functions without autonomous goal-directed planning.
Limited Risk: The report describes the creation of deepfakes, which under the EU AI Act are subject to transparency obligations to ensure users are informed they are interacting with AI-generated content.
The AI systems were used to clone the voices and generate realistic deepfake videos of public figures to impersonate them.
The report clearly outlines the specific scam involving the Bulgarian doctor and TV host, and provides expert commentary on the technical ease of creating deepfakes. The primary classifications are well-supported by the text.
The incident could also be viewed through the lens of hybrid warfare due to the mentions of Putin and Zelenskyy, but the primary local event reported is a commercial fraud scam.
Technology
Advanced Deepfake Generation
AI tools make it easy to clone voices and faces with high accuracy.
Lack of Real-time Detection
No effective automated tools exist to block real-time deepfake video streams.
Asynchronous Eye Movements
AI facial synthesis still struggles with natural, synchronized eye gaze.
Data Inputs
Sufficient Public Media Data
Abundant public video and audio of celebrities enable training of AI clones.
Training Duration of Two Weeks
A two-week training period is enough to create nearly indistinguishable clones.
Human Factors
Lack of Critical Thinking
Viewers and journalists fail to verify the identity of people on camera.
Overconfidence in Video Calls
Users trust real-time video communication channels blindly.
Easy Deception of Public
A large part of the public is easily misled by realistic deepfakes.
Process and Methods
Inadequate Verification Methods
Lack of control questions to verify the identity of interviewees.
Delayed Content Moderation
Platforms rely on post-facto reporting to take down fraudulent ads.
Management
Passive Platform Moderation
Tech giants rely on AI-vs-AI battles rather than proactive management.
Regulatory Environment
Lack of Deepfake Regulation
No regulatory or technical frameworks prevent high-level deepfake creation.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
