This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Chinese law enforcement has reported a significant rise in crimes involving the misuse of AI technologies. Criminals are leveraging generative AI for deepfake fraud, identity theft, unauthorized personality rights usage, and the creation of malicious game hacking software. Authorities have responded with crackdowns, prosecutions, and the implementation of stricter safety evaluations for AI-based verification systems.
Chinese law enforcement has targeted a rise in AI-driven crimes. The crimes include deepfake and voice synthesis used for fraud, identity theft, and unauthorized personality rights usage. In particular, "AI undressing" scams, fake relationships using synthesized voices, and game hacking software make up many of these cases. In response, authorities have prosecuted multiple cases and implemented stricter regulations to control AI misuse.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
A suspect used AI-synthesized dynamic face models to bypass face-recognition systems and illegally download a company's electronic business license.
Criminals used generative AI to automatically generate scripts and synthesize fake videos, such as a fabricated industrial park fire, to gain traffic.
The incident report describes multiple instances of criminals using generative AI for fraud, scams, game cheating, and creating non-consensual sexual imagery to gain financial advantage.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The harms were caused by human criminals intentionally misusing deployed generative AI systems to commit fraud, identity theft, and generate non-consensual pornographic images.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
The incident details a surge in domestic cyber-crime in China leveraging generative AI for identity theft, deepfake fraud, and bypassing government face-recognition systems. While these activities represent an evolved and scaled threat to individual privacy and administrative security, their impact is currently manageable within standard law enforcement procedures and poses minor national security risk.
The AI systems in these cases are used as tools to perform specific, well-defined functions such as generating deepfake images, synthesizing voices, or creating dynamic face models based on direct inputs provided by the users.
Limited Risk: The report describes the widespread use of generative AI to create deepfakes, synthetic voices, and manipulated videos, which fall under AI-generated content requiring transparency obligations.
The primary misuses highlighted involve generating deepfake images ('one-click undressing', face-swaps) and synthetic voices for fraud and personality rights violations.
The reports are official Chinese state media and police releases detailing specific criminal cases, arrests, and court rulings with precise figures (e.g., 7,000 deepfakes, 6.29 million RMB, 50+ victims, 250,000 RMB compensation). This provides a highly reliable factual basis for classification.
None. The cases are clearly defined as criminal misuses of AI technology.
Technology
Low AI Standardization
Low standardization and high customization make AI easy to abuse.
Flawed Liveness Detection
Vulnerabilities in face recognition allowed dynamic AI face-swaps to bypass.
Simple AI Video Synthesis
Generative AI tools make it too easy to synthesize realistic fake videos.
Data Inputs
Leaked Identity Photos
Stolen ID photos and personal info used to generate deepfake face matches.
Unauthorised Voice Data
Unauthorised use of voice recordings to train text-to-speech AI models.
Scraped Video Fragments
Scraped online videos used as raw inputs to synthesize fake disaster clips.
Human Factors
Financial Motivation of Criminals
Criminals are driven by easy profits from selling deepfakes and AI cheats.
Weak Personal Privacy Awareness
Users exposing sensitive biometric data online, easing identity theft.
Desire for Web Traffic
Content creators fabricating fake disaster videos to gain online traction.
Process and Methods
Weak App Verification Processes
Government and payment apps failed to detect synthetic dynamic face inputs.
Inadequate Content Filtering
Platforms lack robust mechanisms to filter AI-synthesized fake news.
Easy AI Tool Accessibility
Lack of strict access controls on powerful AI deepfake and cheating tools.
Management
Inadequate Risk Assessments
App operators failed to assess risks of AI-driven authentication bypass.
Weak Data Security Management
Failure of entities to secure ID photos, leading to massive leaks.
Platform Self-Regulation Failures
Service providers failed to monitor and block malicious AI tool generation.
Regulatory Environment
Lagging Legal Frameworks
Existing laws struggle to keep pace with rapid AI crime evolution.
Lack of Strict Platform Oversight
Insufficient oversight on platforms hosting illegal AI services and tools.
Complex Evidence Collection
High anonymity and complexity of AI crimes hinder law enforcement efforts.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
