This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
In the lead-up to the 2024 South Korean legislative elections, authorities detected 129 instances of AI-generated deepfakes used for illegal campaigning. The National Election Commission has implemented new laws to combat this, but experts warn that the speed of deepfake production currently outpaces verification capabilities. The incident highlights broader concerns regarding the use of synthetic media to manipulate public opinion and undermine democratic processes.
In the lead-up to Korea's parliamentary elections, at least 129 deepfake videos and images were reported to have been detected, violating new election laws. These AI-generated deepfakes were used to mislead and manipulate public opinion, prompting a crackdown by the National Election Commission.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The report highlights that the rapid speed of deepfake production outpaces the verification capabilities of election regulatory bodies.
The incident involves the coordinated use of AI-generated deepfakes to influence political processes and public opinion during democratic elections.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk is caused by human actors intentionally deploying trained AI deepfake generation models to create misleading media for political campaigning.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
During the 2024 South Korean legislative election cycle, authorities detected 129 illegal AI-generated deepfakes used for campaigning. The incident highlights the growing threat of synthetic media to electoral integrity and democratic sovereignty, as the rapid generation of deceptive content outpaces the verification capabilities of regulatory bodies.
The report describes the creation of deepfakes and robocalls using AI technologies. These systems act as tools that generate synthetic media based on human prompts and inputs, without exhibiting independent goal-directed behavior or autonomy.
Limited Risk: The report describes the use of AI-generated deepfake videos and images, which fall under the category of AI-generated content requiring transparency obligations.
The report explicitly focuses on the proliferation of AI-generated deepfake videos and images used for illegal campaigning.
The report clearly outlines the context of the South Korean elections, the specific number of detected deepfakes (129), and the regulatory response. The role of AI as a tool for generating deceptive media is explicit, and the primary risk to democratic norms is well-documented.
None. The incident is clearly a case of malicious use of synthetic media to influence elections.
Technology
Rapid Deepfake Production
Deepfake creation speed outpaces election authority verification efforts.
Evolving Deepfake Sophistication
Constantly evolving AI technology poses ongoing hurdles for detection.
Human Factors
Creation of Misleading Media
Actors generate fake videos to manipulate voters and mislead public opinion.
Process and Methods
Lagging Verification Methods
Verification protocols cannot match the speed of deepfake dissemination.
Regulatory Environment
Recent Law Enforcement Gap
The ban on deepfakes was only recently implemented, leaving legacy issues.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
