This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A deepfake audio recording falsely attributed to US President Joe Biden was circulated on social media platforms including Facebook, X, Instagram, and TikTok. The audio claimed Biden conspired with the Pakistani military to remove former Prime Minister Imran Khan from office in 2022. Fact-checkers and forensic experts concluded the audio was likely AI-generated or spliced, citing linguistic inconsistencies, unnatural cadence, and forensic analysis of frequency patterns. The incident highlights the use of generative AI to propagate political misinformation during sensitive election periods.
A purported deepfake audio file is alleged to have falsely claimed that U.S. President Joe Biden conspired with Pakistan's Army Chief, General Syed Asim Munir, to remove former Prime Minister Imran Khan in 2022. Widely shared online, the audio is reported to have exploited generative AI to spread political misinformation.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The AI system was used to generate false and misleading information that fabricated a statement by a world leader.
The AI-generated audio was deployed as part of a coordinated effort to influence public opinion and political processes at scale.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by a human actor who intentionally generated and distributed the deepfake audio to mislead the public.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
An AI-generated deepfake audio of US President Joe Biden falsely claiming a conspiracy with the Pakistani military was circulated on social media. The incident represents a substantial information warfare concern, exploiting generative AI to impact international relations and domestic political narratives in multiple nations, though it caused no physical or critical infrastructure damage.
The AI system was used as a tool to perform a specific, well-defined function of generating a synthetic voice based on direct inputs, without exhibiting autonomous planning or context retention.
Limited Risk: The report describes an AI-generated deepfake. Under the EU AI Act, AI systems that generate or manipulate image, audio or video content (deepfakes) pose a moderate risk and are subject to specific transparency obligations.
The AI system's primary function was to generate a synthetic clone of Joe Biden's voice.
The report provides detailed forensic and linguistic analyses from multiple independent experts confirming the audio is AI-generated. The dissemination platforms and view counts are well-documented.
None. The consensus among experts is highly consistent that the audio is synthetic.
Technology
Generative AI Voice Cloning
AI algorithms cloned Biden's voice to create realistic but fake statements.
Splicing and Crossfading
Combining audio snippets and adding music to mask synthetic artifacts.
Absence of Natural Speech Pauses
AI failed to replicate natural breathing spaces and human speech cadences.
Data Inputs
Use of Public Voice Samples
Abundant public recordings of Joe Biden were used to train the voice model.
Poor Grammatical Training Data
Linguistic errors in the script resulted in ungrammatical synthetic speech.
Background Music Integration
Adding background music to the input audio file masked AI-generation traces.
Human Factors
Political Polarization
Supporters eagerly shared the fake audio to validate conspiracy narratives.
Lack of Public Verification
Users forwarded the audio without verifying its authenticity or source.
Intentional Misinformation
Creators timed the release to influence the 2024 US and Pakistani politics.
Process and Methods
Inadequate Platform Moderation
Social media platforms failed to flag or block the viral deepfake audio.
Lack of Proactive AI Detection
No automated tools were used by networks to detect synthetic voice clones.
Sidelined Fact Checking
Fact-checking occurred only after the deepfake had achieved viral reach.
Regulatory Environment
Lack of AI Laws in Pakistan
No existing legal framework or penalties governed the malicious use of AI.
Delayed Regulatory Legislation
The draft AI regulation bill was only recently introduced in the Senate.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
