This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A criminal group in China utilized an AI multimodal model to perform 'face-swapping' attacks, bypassing facial recognition systems on major platforms. By converting static photos into dynamic videos, the group successfully breached user accounts to steal personal information, which was then sold to fraud syndicates. The Hangzhou Public Security Bureau investigated the case and arrested four suspects in August 2024, noting that the group had profited approximately 200,000 yuan from these activities.
A criminal group in China used AI face-swapping technology to bypass face recognition systems on major platforms, steal personal data, and sell it to fraud syndicates. The group generated convincing video simulations from static photos to breach accounts, reportedly earning 200,000 yuan. After an investigation by the Hangzhou Public Security Bureau, four suspects were arrested across the provinces of Anhui, Guizhou, and Zhejiang.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The AI-generated videos were directly used to breach user accounts and illegally obtain sensitive personal information.
The criminals used AI face-swapping technology to impersonate users, bypass security, and steal personal data for financial gain, which fits targeted exploitation and fraud.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk and subsequent harm were caused by the intentional actions of human criminals who utilized a deployed AI model to bypass security systems.
Added categories peak at 2: Minor (Psychological Harm, Inferred)
A Chinese criminal group utilized an offshore multimodal AI model to generate deepfake videos, successfully bypassing facial recognition authentication on major platforms to steal and sell personal data. While the incident highlights evolving vulnerabilities in biometric security, the impact remains a localized cybercrime with minor national security implications, successfully resolved by local law enforcement.
The report describes the AI system as a tool used by criminals to perform specific, well-defined functions (generating face-swapping videos from static photos and text prompts) without autonomous decision-making or planning.
Limited Risk: The report describes the use of an AI model to generate face-swapping videos (deepfakes), which falls under transparency obligations for AI-generated content.
The AI system was explicitly used to convert static face photos into dynamic videos (blinking, turning head) to bypass facial recognition, which is a form of deepfake video generation.
The police report provides clear, factual details about the crime, the technology used (multimodal large model for face-swapping), the suspects arrested, and the financial scale of the operation (200,000 yuan). There is little ambiguity about the events.
None. The incident is clearly a cybercrime leveraging generative AI for identity theft and fraud.
Technology
Multimodal AI Generation
Overseas AI models generated realistic face-liveness videos from photos.
Vulnerable Face Authentication
Platform login systems failed to distinguish real faces from AI videos.
Data Inputs
Stolen Static Photos
Static photos were stolen and used as input data for the AI model.
Human Factors
Poor Device Security
Users clicked unsafe links, compromising their personal information.
Insider Misconduct
Employees leaked personal data, giving attackers targets for AI bypass.
Process and Methods
Weak Verification Process
Platforms lacked optical liveness checks to block deepfake videos.
Management
Slow Security Upgrades
Platform management delayed upgrading authentication against AI threats.
Regulatory Environment
Unregulated Overseas AI
Lack of controls on overseas AI models enabled malicious deepfakes.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
