This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
In March 2023, AI-generated photorealistic images depicting the arrest of former U.S. President Donald Trump were widely circulated on social media. While originally created as parody by an investigative journalist using Midjourney, the images were shared by others as factual news, leading to significant public confusion and debate regarding the impact of synthetic media on democratic discourse and information integrity.
AI-generated photorealistic images depicting Donald Trump being detained by the police which were originally posted on Twitter as parody were unintentionally shared across social media platforms as factual news, lacking the intended context.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The synthetic images were shared widely across multiple platforms, demonstrating the potential for AI-generated media to be used for large-scale political influence and public opinion manipulation.
The AI system generated highly realistic but entirely fabricated images of a public figure's arrest, which were subsequently spread online and misled some users into believing a false event had occurred.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was initiated by a human user intentionally prompting the AI to generate the images, and further propelled by other humans intentionally sharing them across social media platforms.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
The viral spread of AI-generated images depicting the fictional arrest of Donald Trump highlighted the evolving threat of photorealistic deepfakes. While the incident was initiated as a parody by a journalist and did not cause physical or systemic harm, it demonstrated how easily synthetic media can pollute the information ecosystem and confuse the public, representing a minor national security concern.
The AI system, Midjourney, functioned as a tool that generated synthetic images based on direct text prompts provided by the user without independent planning or autonomous decision-making.
Limited Risk: The incident involves AI-generated deepfakes, which under the EU AI Act are subject to specific transparency and disclosure obligations to prevent user deception.
The AI system Midjourney was used specifically to generate highly realistic, fabricated images of a public figure to simulate a real-world event.
The reports provide consistent, detailed accounts of the creation, spread, and impact of the Trump arrest deepfakes, including statements from the creator, AI experts, and platform responses.
The incident could be interpreted as a benign artistic parody rather than a safety risk, but the documented public confusion and platform policy violations support the misinformation classification.
Technology
Rapid Generative AI Sophistication
Midjourney v5 released with advanced photorealism mimicking news style.
Imperfect Rendering of Details
AI still struggles with hands, text, and extra limbs, causing some odd details.
Ease of Generating Synthetic Media
Low cost and simple text prompts allow rapid creation of realistic images.
Data Inputs
Scraped Celebrity Training Data
Vast public training data makes famous figures render highly realistically.
Lack of Image Provenance Data
Generated images lack embedded metadata or watermarks to verify origin.
Human Factors
User Sharing Without Context
Users reposted parody fakes as real news to validate political biases.
Lack of Critical Thinking
Public struggled to spot visual errors like extra limbs or garbled text.
Creator Experimentation
Users generated provocative political scenarios to test tool capabilities.
Process and Methods
Inconsistent Platform Policies
Social media sites had varied and weak rules on synthetic media.
Inadequate Detection Tools
Platforms lacked automated systems to detect and label AI fakes.
Lack of Contextual Labeling
Fakes quickly lost their original labels when shared across platforms.
Management
Tech Industry Layoffs
Massive staff cuts reduced trust and safety teams at major platforms.
Inadequate Safety Safeguards
Midjourney released advanced v5 without robust prompt filters or warnings.
Opaque Enforcement Actions
Midjourney banned the creator without explaining which rules were violated.
Regulatory Environment
Absence of AI Regulations
No government policies or standards exist to govern synthetic media creation.
Lack of Developer Liability
Developers face no legal consequences for foreseeable harms of their tools.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
