This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An NHS Foundation Trust in London issued an alert regarding fraudulent, AI-generated videos circulating on Facebook and TikTok. These videos falsely depicted clinicians endorsing weight loss patches to deceive consumers. Experts identified the content as a financial scam, noting inconsistencies in the imagery and social media profiles used to promote the products.
Guy's and St Thomas' NHS Foundation Trust warned that purported AI-generated videos circulated on Facebook and TikTok depicted its clinicians endorsing weight loss patches. The videos allegedly impersonated doctors and used misleading medical claims to market a product.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
The incident involves the use of AI-generated deepfake videos to impersonate NHS clinicians and deceive consumers into purchasing weight loss products for financial gain.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human scammers intentionally using deployed AI generation tools to create fraudulent promotional videos for financial gain.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
An NHS Trust in London issued an alert regarding AI-generated deepfake videos of clinicians falsely endorsing weight loss patches. While the incident represents a malicious use of synthetic media to exploit public trust in state health services, it is a localized financial scam with negligible direct threat to national security.
The AI system was used as a generation tool by human scammers to produce synthetic videos. The report does not describe the AI exhibiting autonomous decision-making, planning, or interactive assistant behavior, but rather acting as a deterministic generator of content based on human inputs.
Limited Risk: The incident involves 'AI-generated content such as deepfakes' which, under the EU AI Act, are classified as limited risk and subject to specific transparency obligations to ensure users are informed of their synthetic nature.
The AI system was used to generate realistic synthetic videos of clinicians to be shared on social media platforms.
The report clearly describes the nature of the scam, the platforms involved, the reaction of the affected NHS Trust, and expert analysis of the fraudulent accounts. There is little ambiguity regarding the fact that AI-generated deepfakes were used for financial deception.
None. The event is clearly a commercial scam leveraging synthetic media.
While the specific AI generator model is unknown, this does not prevent confident classification under the taxonomies.
Technology
Sophisticated AI Generation
AI tools allowed creating realistic videos of fake clinicians.
Platform Detection Gaps
Social media platforms failed to automatically detect AI-generated scams.
Data Inputs
Stolen Identity Data
Scammers used stolen online images, like bank ads, to create fake profiles.
Unverified Promotional Content
Manipulated imagery and false claims were fed into social media feeds.
Human Factors
Emotional Exploitation
Scammers bypassed rational thinking by targeting weight loss desires.
Trust in Medical Authority
Public trust in NHS clinicians was leveraged to make the scam believable.
Process and Methods
Reactive Moderation Processes
Platforms relied on manual reporting rather than proactive prevention.
Lack of Profile Verification
Weak verification allowed fake doctor profiles with mismatched followers.
Management
Slow Takedown Response
Platform management was slow to respond to the Trust's takedown requests.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
