This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A fraudulent Facebook page used deepfake technology to create a video of Croatian immunologist Stipan Jonjić endorsing an anti-parasite product. The video, which featured manipulated audio and visuals, successfully deceived at least one consumer into purchasing the product. The immunologist confirmed the video was a deepfake and that he had no involvement in the promotion.
A Facebook page reportedly posted a video that purported to show Croatian immunologist Stipan Jonjić endorsing an anti-parasite product and recommending it over conventional treatment. The clip was reported to use AI-generated or manipulated media, and Jonjić denied involvement. A woman was reported to have purchased the product after seeing the ad.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
The incident involves the unauthorized use of a public figure's likeness and voice in an AI-generated deepfake to perpetrate a financial scam and deceive consumers into buying a fake product.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by malicious human actors who intentionally used deployed AI deepfake generation tools to create a fraudulent video for financial gain.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
A deepfake video of a Croatian immunologist was used on Facebook to promote a fraudulent medical product, resulting in minor financial loss for a consumer. The incident represents a common cyber-enabled commercial fraud scam using AI tools, posing negligible threats to national security but highlighting the ongoing challenge of synthetic media in public information spaces.
The AI system functioned as a tool used by human creators to generate synthetic video and audio of Dr. Jonjic. The report notes that 'AI tools' are 'fed' existing authentic recordings to create a synthetic copy of a voice or likeness.
Limited Risk: The incident involves the creation of a deepfake video, which falls under Risk Level 3 (Limited Risk) as it is AI-generated content that requires transparency obligations to inform users of its synthetic nature.
The AI system was used to generate a synthetic video and voice clone of Dr. Jonjic to make it appear as though he was endorsing a product.
The report clearly identifies the nature of the incident as an AI-generated deepfake scam, confirmed by the impersonated individual, and details the resulting financial harm to a consumer.
None. The evidence clearly points to an intentional deepfake scam.
Technology
AI Voice Cloning
AI tools replicated the doctor's voice with high accuracy.
Deepfake Video Generation
AI synthesized lip movements to match fabricated speech.
Deep Learning Tools
Advanced algorithms enable easy replication of human targets.
Data Inputs
Accessible Training Data
Scammers used public YouTube videos to train the AI model.
Human Factors
Trust in Authority
Victims believed the advice because it came from a known doctor.
Detection Difficulty
Deepfakes are increasingly difficult for normal users to spot.
Process and Methods
Weak Platform Moderation
The social network failed to flag or block the deepfake video.
Unverified Page Ad Posting
Suspicious new pages can easily upload and distribute scam media.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
