This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Weibo's automated keyword identification model is facing challenges in effectively moderating content due to the rapid evolution of Chinese internet slang and coded language used by users to bypass censorship. Despite the company's efforts to refine its filtering tools, experts suggest that the dynamic nature of online resistance makes it difficult for automated systems to maintain comprehensive control over banned topics.
Weibo's user moderation model is having difficulty keeping up with shifting user slang in defiance of Chinese state censors.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The AI system fails to perform reliably under the pressure of rapidly evolving, creative human slang, exposing limitations in its capability to accurately moderate content.
The Weibo keyword identification model is utilized to conduct large-scale automated censorship of political and social discourse, directly restricting public opinion and freedom of expression.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The deployment of the keyword identification model to censor user content is an intentional decision made by Weibo's management to comply with government regulatory directives.
Weibo's deployment of an updated keyword identification model to comply with Chinese regulatory directives represents a substantial domestic information control effort. While the system struggles to adapt to creative human slang, it successfully restricts the civil liberties and expression of millions of users, highlighting the use of AI as an instrument of state-level societal control.
The Weibo keyword identification model acts as a tool designed to perform the specific, well-defined function of filtering and blocking designated keywords, homophones, and coded language based on direct text inputs.
High Risk: The system is used for large-scale automated content moderation and censorship, which has significant implications for fundamental rights, specifically the freedom of expression and access to information.
The primary function of the Weibo keyword identification model is to filter, moderate, and block specific words, homophones, and coded language to comply with censorship demands.
The report clearly outlines Weibo's deployment of a keyword identification model to censor user content, and details the various linguistic strategies users employ to bypass it. The role of the AI system as an automated filtering tool is explicitly stated, and the resulting harm to freedom of expression is well-documented.
The incident could be viewed purely as a political/regulatory issue rather than an AI safety issue, but the central role of the automated keyword identification model in executing the censorship justifies its classification as an AI safety risk.
Technology
Flawed Keyword Models
Models fail to detect homophones and coded language like Helan.
Inflexible NLP Algorithms
Algorithms struggle with visual scripts like chrysanthemum text.
Data Inputs
Static Keyword Lists
Databases lack real-time updates for rapidly evolving slang.
Lack of Contextual Data
Training data does not capture double meanings of common words.
Human Factors
Adversarial User Behavior
Users actively exploit homophones and emojis to bypass filters.
Cultural Use of Puns
Deep tradition of wordplay allows users to easily invent slang.
Process and Methods
Slow Model Updates
Process to update models cannot match the speed of new slang.
Poor Context Evaluation
Methods fail to differentiate literal meanings from coded insults.
Management
Overreliance on Automation
Management relies on automated models to solve complex social issues.
Unrealistic Cleanup Targets
Weibo set unattainable goals to fully eliminate homophones.
Regulatory Environment
Strict Censorship Demands
State regulators force platforms to ban massive lists of terms.
Conflicting Directives
Platforms cannot ban words that honor state entities like military.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
