This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The 2024 Indian Lok Sabha elections have seen a significant rise in the use of AI-generated deepfakes for political campaigning. Political consultants and firms are deploying synthetic audio and video clones of candidates to conduct personalized voter outreach and negative campaigns against opponents. While some politicians view this as an efficient tool for engagement, the practice has raised serious concerns regarding voter deception, lack of consent, and the potential to undermine democratic norms through the spread of misinformation.
Digital manipulators in India are using deepfake technology to influence the 2024 Lok Sabha elections. These AI-generated videos and audio clips are designed to tarnish the reputations of political candidates, challenging the integrity of electoral processes.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The report highlights the creation of non-consensual sexualized deepfakes of actress Rashmika Mandanna and negative campaigns designed to degrade opponents.
The campaign involved generating fabricated speeches and voice clones that misled voters into believing they were interacting directly with real candidates.
The widespread, coordinated deployment of millions of AI-generated voice clones and deepfakes to influence the 2024 Indian general election represents a large-scale campaign to manipulate political processes and public opinion.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was driven by human political actors and digital marketing firms intentionally deploying AI deepfake tools to manipulate public opinion and conduct voter outreach.
Added categories peak at 4: Severe (Epistemic Harm, Direct)
The widespread deployment of over 50 million AI-generated voice clones and deepfakes during the 2024 Indian elections represents a significant evolution in digital influence operations. While primarily domestic, the scale of this synthetic media campaign poses substantial threats to electoral integrity, information security, and societal trust in democratic institutions.
The AI systems, such as DeepFaceLab, are described as tools operated by humans who issue commands and directives to generate specific synthetic media outputs, with the software executing these deterministic functions.
Limited Risk: The report describes the widespread deployment of AI-generated deepfakes and voice clones in political campaigns, which falls under the transparency obligations of Limited Risk systems.
The primary use of the AI systems described is to generate deepfake videos and clone voices for political campaigns.
The reports provide detailed, first-hand investigative accounts of both unsanctioned negative deepfake campaigns and sanctioned voter outreach efforts, including specific business figures, candidate names, and operational methods.
Some might view the sanctioned outreach as a benign technological innovation for language translation rather than a deceptive practice, though the reports highlight the inherent deception.
Technology
Realistic Voice Cloning
AI matches voice frequency perfectly, making clones difficult to detect.
Low-cost AI subscription tools
Cheap apps costing 25 to 30 dollars allow anyone to generate deepfakes.
Face-swapping software
Software like DeepFaceLab allows easy grafting of faces onto target videos.
Data Inputs
Available public speech data
Abundant online speeches of politicians are cloned without authorization.
Lack of original audio reference
No original audio exists to verify or debunk manipulated voice clones.
High-definition facial data
High-quality photos and videos are easily harvested to build digital avatars.
Human Factors
Low digital literacy among voters
Voters, especially in rural areas, cannot distinguish AI from real people.
Overconfidence in AI outreach
Politicians trust AI to build voter rapport without considering deception.
Malicious intent by campaign managers
Operators deliberately create fake videos and calls to discredit opponents.
Process and Methods
IP spoofing to avoid detection
Manipulators use external and US IPs to hide their actual location.
Lack of disclosure in outreach
Campaigns make millions of calls without disclosing that the voice is AI.
Unsanctioned negative publicity
Firms use deepfakes for political hammering and character assassination.
Management
Prioritizing outreach over ethics
Campaign managers prioritize scale and voter impact over ethical concerns.
Resurrecting deceased politicians
Campaigns use dead leaders' voices without family consent to gain votes.
Lack of internal oversight
Political parties approve AI campaigns without verifying source authenticity.
Regulatory Environment
Absence of clear AI regulations
Lack of strict rules allows widespread proliferation of political deepfakes.
Difficult legal enforcement
Anonymity and external IPs make catching digital manipulators very hard.
Inadequate platform moderation
Social media and messaging apps fail to block or flag deepfake content.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
