This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Researchers at the University of Washington and other entities developed AI-based video synthesis and face-swapping tools (such as FakeApp) capable of creating photorealistic deepfakes. These tools allow for the manipulation of video footage to make public figures appear to say things they never uttered. While researchers highlight potential benign applications like improving video conferencing, the technology has been widely recognized as a significant risk for spreading misinformation, political manipulation, and eroding public trust in media.
University of Washington researchers made a deepfake of Obama, followed by Jordan Peele
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The reports highlight the severe potential for deepfakes to be weaponized in coordinated political disinformation campaigns to manipulate public opinion and elections.
The primary risk demonstrated is the generation of highly realistic, fabricated video and audio content that can mislead audiences and undermine the credibility of visual evidence.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The creation of the deepfake videos by Jordan Peele and the development of the synthesis software by researchers were intentional human actions utilizing deployed AI tools.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
The public emergence of realistic AI-generated deepfakes, demonstrated by the University of Washington's lip-syncing research and the Jordan Peele Obama PSA, established a significant new capability for information warfare and societal manipulation, prompting immediate defense research responses like DARPA's media forensics program.
The AI systems involved, such as the University of Washington's recurrent neural network and FakeApp, operate as tools. They perform specific, well-defined functions of face-swapping and lip-syncing based on direct inputs of video and audio data, without exhibiting autonomous planning or goal-directed behavior.
Limited Risk: The technology described is used to generate deepfakes and synthetic media. Under the EU AI Act, AI-generated content such as deepfakes is classified as Limited Risk (Level 3), requiring transparency obligations so that users are aware the content is AI-generated.
The primary function of the AI systems described (both the UW model and FakeApp) is to generate realistic synthetic videos of people speaking words they did not actually say, often combined with voice cloning technologies.
The reports provide consistent, detailed accounts of the development of lip-syncing AI by the University of Washington and the creation of the Obama deepfake PSA by Jordan Peele and BuzzFeed. The technical mechanisms (RNNs, GANs, FakeApp) and the societal implications are clearly articulated across multiple reputable sources.
The incident could be interpreted purely as a benign scientific breakthrough in video editing rather than an AI safety incident, though the widespread concern over misinformation and non-consensual porn supports the safety risk classification.
Technology
Open-Source AI Algorithms
Public release of FakeApp and open-source deep learning code on Reddit.
Advanced Neural Networks
Recurrent neural networks and GANs enable highly realistic lip-syncing.
Voice Cloning Tech
Software like Lyrebird replicates voices from short audio samples.
Data Inputs
Abundant Public Video Footage
Hours of high-definition footage of public figures are freely available.
Lack of Input Verification
Neural nets are trained on unverified internet data without restrictions.
Social Media Image Uploads
Users upload thousands of tagged images, increasing exposure to fakes.
Human Factors
Confirmation Bias
People tend to believe fake videos that align with their political views.
Low Media Literacy
Unsuspecting viewers struggle to identify subtle digital manipulations.
Malicious Intent of Actors
Pornographers and fraudsters exploit the tech for revenge porn and scams.
Process and Methods
Inadequate Detection Tools
Reverse engineering AI to detect fakes lags behind generation methods.
Lack of Watermarking Protocols
Absence of standard digital signatures or watermarks on edited media.
Tedious Manual Verification
Rigorous newsroom verification processes are slow to counter viral fakes.
Management
Academic Tech Release
Researchers publish powerful tools without fully assessing societal risks.
Corporate Funding of AI Research
Tech giants fund AI research prioritizing capabilities over safety tools.
Lack of Platform Accountability
Social networks fail to implement real-time fake detection systems.
Regulatory Environment
Absence of Global Protocols
No international standards exist to authenticate digital media identity.
Delayed Government Policies
Technological evolution rapidly outpaces the development of policy safeguards.
Inadequate Platform Bans
Banning users on Reddit or other sites fails to stop technology spread.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
