This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A deepfake video impersonating a Chicago mayoral candidate was posted to Twitter on the eve of the election. The video featured fabricated audio of the candidate making inflammatory remarks about police conduct. While the platform removed the content and the account, experts noted the difficulty of mitigating the impact of such misinformation during a critical election window.
On the eve of Chicago's mayoral election, a deepfake video impersonating candidate Paul Vallas was posted to Twitter, showing a fake audio of him making inflammatory statements. The video was viewed thousands of times before being taken down. The Vallas campaign condemned the video, calling it a deceptive impersonation.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The AI system was used to impersonate a specific candidate to defame his character and manipulate voter beliefs.
The deepfake was deployed as part of a coordinated attempt to influence public opinion and manipulate a political process on the eve of an election.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by a human actor intentionally creating and deploying a deceptive deepfake to influence an election.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
A deepfake video featuring a realistic cloned voice of a Chicago mayoral candidate was deployed on the eve of an election to manipulate public opinion. While the impact was localized to a municipal election and the content was quickly removed, the incident highlights the evolving threat of AI-generated disinformation to democratic processes.
The AI system performed a specific, well-defined function of voice cloning based on direct inputs provided by the user, operating as a tool without autonomous decision-making or planning.
Limited Risk: The report describes an AI-generated deepfake, which falls under Risk Level 3 due to transparency obligations requiring users to be informed of its synthetic nature.
The AI system was used to clone the candidate's voice to make it say fabricated statements, which was then overlaid on a photo to create a deepfake video.
The report clearly details the nature of the deepfake, its content, the platform used, and the immediate response. While the specific AI tool used is not named, the mechanism (voice cloning/deepfake) is well-understood.
None.
Technology
Rapid Deepfake Generation
AI models allow generating realistic deepfakes in minutes.
High Realism of Synthetic Audio
The deepfake audio was well made and resembled the candidate's voice.
Data Inputs
Candidate Voice Data
Availability of candidate's public voice recordings for model training.
Human Factors
Malicious Intent to Deceive
The creator aimed to deceive voters and damage the candidate's campaign.
User Vulnerability to Misinformation
Normal people struggle to verify if the audio is real or fake.
Process and Methods
Immature Detection Research
Research and tools to identify deepfakes are still in their infancy.
Lack of Proactive Filtering
The video was uploaded and viewed thousands of times before removal.
Management
Reactive Content Moderation
Platform acted after the video was reported and viewed thousands of times.
