This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
During the 2024 Pakistani general elections, multiple AI-generated deepfake videos and audio clips were circulated on social media to spread political disinformation. These deepfakes falsely depicted political figures, including the PTI founder and various candidates, making statements they never actually made, such as announcing election boycotts or switching party allegiances. The incidents were identified as attempts to manipulate voter perceptions and influence the democratic process.
During Pakistan's 2024 general elections, purportedly AI-generated deepfakes of a politically motivated nature were circulated. These deepfakes reportedly portrayed political figures in misleading contexts, spreading misinformation and aiming to influence voter perceptions and election outcomes.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The AI systems generated highly realistic but entirely false audio and video representations of political figures to mislead voters.
The incident involved coordinated, politically motivated deployment of AI-generated deepfakes to influence public opinion and manipulate the Pakistani general elections.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was caused by human actors intentionally using deployed AI deepfake generation tools to create and spread political disinformation during an election.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
During the 2024 Pakistani general elections, multiple AI-generated deepfakes of political candidates were circulated to spread disinformation, including false announcements of boycotts and defections. This incident represents a substantial threat to information security and sovereignty by targeting democratic processes, demonstrating an evolved capability of human-controlled AI tools to manipulate public discourse at scale.
The AI systems used were deepfake generation tools that performed the specific function of synthesizing video and audio based on direct human inputs and training data of the target individuals' voices and faces.
Risk Level 3: Limited Risk. The report describes the creation and dissemination of 'AI-generated completely new videos or audio', which are explicitly defined as deepfakes under the Act and subject to transparency obligations.
The AI systems were used to generate highly realistic fake videos and audio clips (deepfakes) of political candidates to mislead the public.
The report provides specific details about multiple distinct deepfake incidents targeting named political figures during the 2024 Pakistani elections, including their responses and clarifications. The role of AI in generating these deepfakes is explicitly stated, and the political context is well-documented.
None. The events are clearly described as politically motivated disinformation using synthetic media.
Technology
Deep Learning Algorithms
Algorithms learn from data to generate highly realistic fake media.
AI Video and Audio Synthesis
AI-generated media convinces viewers that someone said or did something.
Data Inputs
Target Training Data
Deep learning models require source data to learn and mimic targets.
Human Factors
Political Disinformation Motivations
Actors knowingly spread untrue AI content to influence voters.
Public Deception Vulnerability
Unnatural details in deepfakes are difficult for average users to detect.
Process and Methods
Rapid Social Media Viral Spread
Deepfakes circulate quickly on platforms like X, Facebook, and WhatsApp.
Reactive Clarification Process
Candidates must manually post videos to debunk AI disinformation.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
