This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Multiple individuals in China were administratively penalized for using AI software to generate and disseminate false news reports. These incidents included fabricating a landslide death toll in Yunnan, misrepresenting the epicenter and severity of a Sichuan earthquake, and creating a false story about a woman's death following a traffic enforcement incident. The perpetrators utilized AI to generate text and images to attract traffic and platform rewards, resulting in significant social disruption.
In China, purported AI tools were reportedly used to fabricate and disseminate false reports of disasters, including a landslide in Yunnan, an earthquake in Sichuan, and a sudden death after a traffic incident. On May 27, 2024, a real 5.0-magnitude earthquake occurred in Muli County, Sichuan, with no casualties and limited property damage. However, a social media post later falsely claimed the epicenter was in Xide County and exaggerated the event's severity, adding fabricated images of extensive destruction. The deployers of these false reports have since received administrative penalties from Chinese authorities for their actions.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The AI tools were used to generate highly realistic but entirely fabricated news articles and images regarding disasters and public safety incidents.
The perpetrators intentionally used AI tools to fabricate false news stories and images to manipulate public beliefs, attract traffic, and gain financial platform rewards.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was caused by human actors who intentionally used deployed AI tools to generate and spread false news stories for personal gain.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
Domestic individuals in China used AI text and image generation tools to fabricate and spread false news reports about natural disasters and police misconduct to gain social media traffic and financial rewards. The incident was managed via standard local administrative penalties and had minor national security implications, primarily affecting local social order and public trust in official communications.
The report describes the perpetrators using AI software as a tool to generate specific text and images based on their inputs to fabricate news stories. The AI did not act autonomously or as an assistant with ongoing interaction, but rather as a deterministic generation tool.
Risk Level 3: The report describes the use of AI to generate synthetic text and images (deepfakes/synthetic media) to fabricate news, which falls under Limited Risk due to transparency and disclosure obligations for AI-generated content.
The AI systems were used to generate text articles and synthetic images specifically for publication on social media and video platforms to attract traffic.
The report clearly details three specific cases, the names of the perpetrators, the dates, the specific lies fabricated, and the legal outcomes. The details provided are sufficient to confidently classify the risk domains and harms.
None
Technology
AI Generation Tools Accessibility
AI tools make it easy to synthesize realistic fake text and images.
Realistic Image Synthesis
AI software generates highly convincing images of non-existent disasters.
Data Inputs
Unverified Internet Sources
Users feed unverified online rumors into AI software to expand them.
Grafting AI Text with Real Images
Combining AI-generated text with actual photos to mislead readers.
Human Factors
Desire for Traffic and Profit
Perpetrators fabricate rumors to gain attention and platform rewards.
Deliberate Malicious Intent
Users intentionally use AI to generate sensationalized fake news.
Process and Methods
Automated Content Creation
Using AI to automatically write full articles without manual verification.
Inadequate Platform Moderation
Social media platforms fail to block AI-generated rumors immediately.
Management
Incentivizing Traffic over Truth
Platforms reward high-traffic posts, encouraging sensational AI rumors.
Lack of Source Verification
Platforms host unverified AI stories without checking their authenticity.
Regulatory Environment
Reliance on Post-Hoc Punishment
Law enforcement acts after the rumors have already spread and caused harm.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
