This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
In February 2020, the BJP in India utilized deepfake technology to create campaign videos of party leader Manoj Tiwari. These videos used lip-sync algorithms to make Tiwari appear to speak Haryanvi and English, languages he does not know, to target specific voter demographics. The campaign reached millions via WhatsApp, raising significant concerns about the use of synthetic media to manipulate democratic elections and spread disinformation.
In February 2020, India's BJP used purported deepfake videos of party leader Manoj Tiwari to influence voters in the Delhi legislative election. The videos reportedly showed Tiwari delivering scripted campaign messages in Haryanvi and English, which are languages he did not speak, by altering his lip movements and voice. Distributed via thousands of WhatsApp groups, the deepfakes reportedly reached millions of voters without disclosure.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The AI system was used to generate false videos of a politician saying things he never actually said in languages he does not speak.
The BJP used deepfake technology to conduct a coordinated, large-scale campaign reaching 15 million people to influence political processes and voter behavior in the Delhi elections.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human political actors intentionally deploying deepfake technology post-deployment to influence voters.
Added categories peak at 5: Catastrophic (Epistemic Harm, Inferred)
First documented deployment of deepfake technology in an Indian political campaign, where the BJP used synthetic videos of leader Manoj Tiwari to target voters. Reaching 15 million people via WhatsApp, this incident highlights the threat of automated, highly-targeted disinformation to electoral integrity and democratic sovereignty, establishing a concerning precedent for future elections.
The report describes the AI as a 'lip-sync' deepfake algorithm used to translate audio sounds into basic mouth shapes based on direct inputs, which fits the definition of a Tool performing a specific, well-defined function.
Limited Risk: The report describes the creation and distribution of deepfakes, which are explicitly classified as limited risk under the EU AI Act, requiring transparency obligations to inform users of their synthetic nature.
The AI system's primary function was to generate deepfake videos by altering lip movements to match target audio in different languages.
The report provides clear, detailed, and verified accounts of the creation, distribution, and impact of the deepfake videos, including direct quotes from both the political party (BJP) and the technical firm (The Ideaz Factory) involved.
None. The facts of the deployment and its political intent are uncontested by the creators.
Technology
Lip-sync deepfake algorithm
Allowed altering lip movements to match target audio in another language.
Publicly available codebase
Code needed to fabricate videos is publicly available on websites.
Lack of reliable detection tools
Tools to reliably detect deepfakes are currently unavailable to public.
Data Inputs
Original video footage
Used real video of Manoj Tiwari to overlay synthesized mouth shapes.
Target audio translation
Hired dubbing artist to impersonate the voice in Haryanvi language.
Training speeches
Algorithm trained with speeches of Manoj Tiwari to learn mouth shapes.
Human Factors
Nascent digital literacy
Low digital literacy in India makes users highly vulnerable to manipulation.
Belief in video authenticity
Voters assumed the candidate spoke their language, creating false trust.
Process and Methods
Distribution via WhatsApp groups
Mass distribution across 5,800 groups bypassed public scrutiny.
Bypassing fact-checking resources
Doctored content is extremely difficult for fact-checkers to recognize.
Lack of disclosure
No clear labeling or disclosure that the video was synthetically altered.
Management
Campaign scaling prioritization
Prioritized scaling outreach efforts over potential risks of misinformation.
Use of shadow firms
Political parties use external agencies to maintain deniability.
Regulatory Environment
Absence of deepfake regulations
No clear legal framework in India governs or restricts political deepfakes.
Difficulty in legal enforcement
Outlawing deepfakes is doomed to fail as they operate via shadow firms.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
