This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
During widespread protests against COVID-19 restrictions in China, a coordinated campaign of Chinese-language accounts flooded Twitter search results with adult content. This spam technique successfully obscured news and images of the protests for several hours. The incident highlighted concerns regarding Twitter's reduced capacity to manage deceptive foreign influence operations following significant staff layoffs.
Twitter Feed was flooded by content from Chinese-language accounts which allegedly aimed to manipulate and reduce social media coverage about widespread protests against coronavirus restrictions in China.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The spam campaign flooded search results with links to escort services and adult offerings, exposing users to inappropriate and pornographic content.
The incident involved a coordinated campaign using automated accounts to conduct large-scale censorship and propaganda, aiming to manipulate public opinion and suppress news of political protests.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was driven by human actors who intentionally deployed automated spam accounts to suppress protest news, exploiting Twitter's post-deployment platform vulnerabilities.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
A coordinated campaign used automated accounts to flood Twitter search results with adult content, successfully obscuring news of widespread anti-lockdown protests in China. This incident highlights how adversaries can exploit vulnerabilities in private social media platform moderation to conduct effective information suppression and restrict global visibility of civil rights movements.
The report does not characterize the AI system's mode of operation. It mentions automated accounts and Twitter's automated filters, but does not describe their specific behavioral architecture.
Risk Level 4: Minimal or No Risk. The incident involves automated spam accounts and Twitter's spam filters, which are classified as posing low or negligible risks to users and society under the Act.
The automated accounts generated social media content (spam links and adult offerings) to flood the platform, while Twitter's automated spam filtering failed to mitigate the campaign.
The report clearly describes the nature of the coordinated spam campaign, its timing, and its impact on Twitter searches during the Chinese protests. The role of automated accounts and the failure of Twitter's moderation systems are well-documented.
The incident could be viewed purely as a human-driven political censorship campaign rather than an AI safety failure, though it directly exploited automated systems and failures in automated content moderation.
Technology
Inadequate Automated Spam Filtering
Automated systems failed to flag and block coordinated Chinese spam campaigns.
Vulnerable Search Algorithms
Search algorithms prioritized high-volume spam over legitimate protest news.
Data Inputs
Dormant Accounts Reactivation
Lack of flags on suddenly active accounts spreading spam links.
Spam Keyword Manipulation
Spammers manipulated location tags and keywords to drown out protest news.
Human Factors
Mass Resignations of Analysts
All China-specific influence analysts resigned, leaving a human review gap.
Staff Burnout and Reductions
Remaining staff were overwhelmed by the sudden high volume of spam.
Process and Methods
Reliance on Manual Moderation
The mitigation process relied too heavily on manual intervention.
Delayed Incident Response
Lack of proactive monitoring delayed the mitigation of the spam attack.
Management
Severe Staff Downsizing
Drastic headcount reduction weakened the platform's anti-propaganda defenses.
Underestimation of Geopolitical Risk
Management did not adequately staff critical regional safety teams.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
