This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
During the 2022 South Korean presidential election, multiple candidates, most notably Yoon Suk-yeol, utilized AI-generated deepfake avatars to conduct digital campaigning. These avatars were designed to interact with voters, answer questions, and project a more approachable image than the candidates themselves. The practice sparked significant debate regarding political ethics, the potential for voter deception, and the risk that such technology could be used to spread misinformation or manipulate public opinion.
A South Korean political candidate created a deepfake avatar which political opponents alleged to be fraudulent and a threat to democracy.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The widespread use of deepfake candidates raised concerns about eroding public trust in authentic media and polluting the political information ecosystem.
The incident involves using AI deepfakes to conduct coordinated, population-level political campaigning and influence public opinion during a presidential election.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The deployment of AI avatars was an intentional decision by human campaign teams post-deployment to influence voters.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
During the 2022 South Korean presidential election, domestic campaigns officially deployed AI-generated deepfake avatars of candidates to engage voters. While the incident raised significant ethical concerns regarding voter deception, epistemic trust, and political decorum, it was human-controlled and did not involve foreign interference or critical infrastructure threats, representing a minor national security impact.
The AI avatars function as assistants to the political campaigns, augmenting the candidates' reach by delivering prepared scripts and interacting with voters under human oversight.
Limited Risk: The reports describe the use of AI-generated content such as deepfakes and chatbots ('AI Yoon', 'AiDY') for political campaigning, which require transparency obligations under the EU AI Act.
The primary function of the AI system was to synthesize realistic video and voice to clone the candidates, creating deepfake avatars for campaigning.
The reports provide consistent, detailed accounts of how the AI avatars were created, deployed, and received by the public during the 2022 South Korean election. Multiple sources confirm the technical process (DeepBrain AI cloning candidates from recorded data) and the ethical debates surrounding their use.
The deployment could be viewed purely as a novel marketing campaign rather than an AI safety risk, though the ethical concerns raised about democratic norms justify the safety analysis.
Technology
Deceptive Deepfake Realism
High realism leads some voters to mistake the avatar for the real candidate.
Lack of Emotional Expression
Synthesized voice is flat and cannot replicate natural laughter or sadness.
Inability to Replicate Movements
Software cannot replicate natural physical habits like head-bobbing.
Data Inputs
Pre-recorded Training Data
Avatar is trained on hours of specific audio and video to clone the candidate.
Scripted Non-Candidate Outputs
Scripts are written by campaign staff rather than the candidate himself.
Filtered Behavioral Data
Training data was filtered to hide candidate's habits and speaking flaws.
Human Factors
Voter Deception Risk
Voters may mistake synthetic deepfakes for actual live recordings.
Reduced Trust in Genuine Media
Prevalence of deepfakes lowers public trust in authentic media broadcasts.
Overreliance on Likability
Voters prefer witty digital avatars over complex, dull policy debates.
Process and Methods
Rapid Script Generation
Scripts are written in brainstorms as short as 30 minutes without deep review.
Disguising Candidate Flaws
Campaigns use avatars to mask bad postures and poor speaking habits.
Use of Gamified Language
Script writers use gaming slang and insults to maximize online engagement.
Management
Prioritizing Image over Policy
Campaign management prioritized humorous digital image over serious policy debate.
Ethical Decorum Downgrade
Management approved insulting scripts targeting rivals to gain younger votes.
Regulatory Environment
Unclear Election Laws
Election commission struggled to determine the legitimacy of AI campaign tools.
Inadequate Deepfake Penalties
Absence of strong legal punishments for malicious deepfakes during campaigns.
