This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Anonymous Facebook pages have been using AI-generated deepfake videos of prominent Bulgarian public figures, including actor Mihail Bilalov, to market unverified joint-pain products. These videos manipulate original television footage by overlaying synthetic audio and altering lip movements. The advertisements direct users to fraudulent websites that mimic legitimate news outlets to facilitate sales of products not available through standard pharmacies.
A reportedly AI-manipulated Facebook advertisement appeared to repurpose footage of Bulgarian actor and television host Mihail Bilalov, overlaying purportedly fabricated audio and altered lip movements to market an alleged joint-pain product. The ad reportedly linked to a fabricated article imitating a Bulgarian news outlet and to an online sales funnel for a product described as unavailable through pharmacies.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The deepfake videos and associated fake news articles spread fabricated information and false endorsements to mislead consumers.
The incident involves the unauthorized use of public figures' likenesses and cloned voices to deceive social media users into purchasing unverified medical products for financial gain.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human malicious actors who intentionally deployed AI voice-cloning technology to create deceptive advertisements for financial gain.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
A commercial scam in Bulgaria utilized AI-generated voice clones and deepfakes of local celebrities to sell unverified health products. While illustrating the growing accessibility of synthetic media for fraud, the incident has negligible direct national security implications.
The AI system, ElevenLabs, was used as a tool to generate synthetic voice clones based on direct inputs without autonomous planning or agency.
Risk Level 3. Limited Risk: The report describes the creation and dissemination of deepfakes, which are subject to specific transparency obligations under the Act.
The ElevenLabs system was used to clone and generate the voices of Bulgarian celebrities to make it appear as though they were endorsing the joint-pain product.
The report from Factcheck.bg is highly detailed, verifying the deepfake nature of the videos using ElevenLabs tools, identifying the specific public figures targeted, and describing the exact mechanism of the scam.
None.
Technology
Deepfake Voice Generation
AI tools were used to clone the actor's voice with 98% probability.
AI Video Manipulation
AI lip-syncing adjusted mouth movements to match the fake audio.
Data Inputs
Hijacked TV Broadcast Footage
Original video inputs from a TV show were used without authorization.
Unverified Audio Training Data
Public recordings of the actor's voice were used to train the AI.
Human Factors
Targeting Vulnerable Audiences
Targeted Facebook ads reached users seeking joint pain relief.
User Trust in Known Celebrities
Viewers believed the endorsement due to the actor's high profile.
Process and Methods
Flawed Ad Review Process
Facebook's ad review failed to detect the AI-manipulated video.
No Pre-Ad Verification
Lack of identity verification before running celebrity ads.
Management
Ad Platform Profit Prioritization
Facebook accepted ad money without verifying content legitimacy.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
