This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Social media platforms including TikTok, YouTube, Instagram, and Twitch utilize automated content moderation algorithms that flag specific words without context. This has forced content creators to adopt 'algospeak'—a system of euphemisms and misspellings—to avoid demonetization or content removal. This phenomenon disproportionately affects marginalized groups, such as LGBTQ and BIPOC creators, by silencing their ability to discuss sensitive topics and effectively altering human language in real-time.
TikTok's, YouTube's, Instagram's, and Twitch's use of algorithms to flag certain words devoid of context changed content creators' use of everyday language or discussion about certain topics in fear of their content getting flagged or auto-demonetized by mistake.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
Arbitrary demonetization and down-ranking by algorithms reduce the quality of employment and income stability for independent creators.
The moderation algorithms lack the capability to understand context, humor, or slang, leading to false positives and the censorship of benign speech.
The automated moderation systems perform unequally across demographic groups, disproportionately targeting and silencing the legitimate speech of marginalized communities such as LGBTQ and BIPOC creators.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The suppression of legitimate speech and the resulting rise of algospeak are unexpected and unintentional outcomes of deploying automated content moderation systems.
Added categories peak at 3: Substantial (Psychological Harm, Indirect)
Automated content moderation algorithms on major social media platforms have driven the rise of 'algospeak' as users attempt to bypass contextual blindspots. While this disproportionately impacts marginalized creators and allows harmful groups to evade filters, it presents minor national security concerns, primarily limited to long-term digital human rights and minor information monitoring challenges.
The content moderation systems are described as automated tools that perform the specific function of filtering and flagging content based on a static list of problematic words without contextual understanding.
Minimal or No Risk: The automated content moderation filters and recommendation algorithms described are similar to spam filters, which pose low or negligible risks under the EU AI Act and have no additional regulatory obligations.
The primary function of the AI systems is content moderation to filter out unsafe words, while recommendation algorithms down-rank flagged content on user feeds.
The reports provide clear, consistent, and detailed accounts of the 'algospeak' phenomenon across multiple major platforms. The role of automated content moderation algorithms is explicitly described, and the resulting harms are well-documented with quotes from creators, researchers, and linguists.
None. The phenomenon of algospeak as a reaction to algorithmic moderation is widely accepted and clearly demonstrated.
Technology
Context-blind word filtering
Algorithms flag words without understanding context, slang, or nuance.
Algorithmic distribution model
TikTok's FYP algorithm forces users to optimize content to avoid down-ranking.
Automated demonetization systems
Systems automatically demonetize or shadowban creators based on flagged terms.
Data Inputs
Static keyword databases
AI moderation relies on rigid, static lists of banned words.
Lack of dialect and slang data
Algorithms lack training on evolving internet dialects and marginalized slang.
Human Factors
User evasion tactics
Users actively invent and adopt 'algospeak' to bypass algorithmic filters.
Fear of algorithmic punishment
Creators feel forced to self-censor to protect their reach and income.
Public pressure for fast removal
Public demand for quick content removal drives aggressive, automated filtering.
Process and Methods
Opaque moderation policies
Platforms lack transparency, forcing users to reverse-engineer filter rules.
Inflexible review mechanisms
Automated systems lack human-in-the-loop validation for flagged terms.
Whack-a-mole filtering approach
Platforms reactively ban new replacement terms, perpetuating the cycle.
Management
Prioritizing brand safety over speech
Platforms prioritize advertiser demands and brand safety over user expression.
Over-reliance on automated tools
Management deploys cheap automated filters at scale instead of human reviewers.
Lack of transparency
Management fails to provide clear guidelines on how algorithms operate.
Regulatory Environment
Lack of policy guidelines
No clear regulatory frameworks exist to govern automated content moderation.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
