This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A deepfake video of U.S. President Joe Biden was circulated on social media, featuring AI-generated audio and manipulated lip movements to make it appear he was making transphobic remarks. The video used footage from a January 25, 2023, speech about Ukraine, with audio synthesized using ElevenLabs' voice cloning technology. Fact-checkers and digital forensics experts confirmed the video was a deepfake, noting that the president never made such statements.
A deepfaked audio of US President Joe Biden making transphobic remarks played on top of a video showing him giving a speech was released on Instagram and circulated on social media.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The deepfake generated and exposed social media users to highly toxic, transphobic hate speech and references to suicide.
The AI tool was used to impersonate a public figure by cloning his voice without consent to manipulate viewers.
The incident involved the intentional creation and viral dissemination of a political deepfake of President Biden to spread disinformation and influence public opinion.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by a human actor intentionally using a deployed AI voice cloning tool to generate and distribute a deceptive, transphobic deepfake video of President Biden.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
An anonymous actor used ElevenLabs' voice cloning tool to create a transphobic deepfake of U.S. President Joe Biden. While the video circulated widely on social media, it was quickly debunked by fact-checkers, resulting in minor information security and societal impacts without threatening critical infrastructure or government functions.
The reports describe ElevenLabs as a 'voice synthesis platform' and 'voice cloning tool' that allows users to generate realistic audio of any person's voice by 'uploading a few minutes of audio samples and typing in any text for it to say.' This represents a deterministic function operating on direct user input, characteristic of a Tool.
Limited Risk: The system is used to generate AI-generated content such as deepfakes, which under the EU AI Act falls under Limited Risk, requiring specific transparency obligations to ensure users are informed that the content is AI-generated.
The primary AI system identified is ElevenLabs, which is a voice synthesis and cloning platform designed for voice generation. The overall incident resulted in a deepfake video combining this audio with manipulated lip movements.
The reports provide consistent, detailed accounts of the deepfake video, the specific AI tool used (ElevenLabs), the platform safeguards implemented in response, and expert analysis of the digital manipulation techniques. There is no conflicting information regarding the fake nature of the video or the technology used.
None. The video is universally confirmed by multiple fact-checkers and forensics experts to be an AI-generated deepfake.
Technology
Advanced Voice Synthesis
ElevenLabs beta allowed realistic voice cloning with minimal audio samples.
Open-source Lip-sync Algorithms
Free tools enabled matching mouth movements to synthetic audio.
Data Inputs
Public Voice Samples
Abundant public audio of Biden was used to train the voice-cloning model.
Source Video Footage
Real footage from a Jan 25 speech was used to map mouth movements.
Human Factors
Malicious Intent by Creators
Bad actors intentionally generated and spread transphobic disinformation.
User Susceptibility
Some social media users believed the highly realistic deepfake was authentic.
Process and Methods
Unrestricted Beta Release
ElevenLabs initially allowed anonymous, free access without safeguards.
Reactive Content Moderation
Social media platforms relied on post-facto flagging and fact-checking.
Management
Premature Technology Release
Management released the tool to the public before implementing safety controls.
Lack of Prior Risk Assessment
The startup deployed the model without anticipating voice cloning misuse.
Regulatory Environment
Lack of Deepfake Regulation
No legal frameworks existed to restrict the generation of deceptive deepfakes.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
