This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Facebook and Twitter's automated content moderation systems have been reported to fail in effectively enforcing community standards for small language groups, particularly in the Balkans. These systems struggle with non-English languages and non-Romanic scripts, leading to both the failure to remove hate speech and the wrongful censorship of legitimate content. The lack of transparency and investment in human moderation for these regions exacerbates the issue.
Facebook's and Twitter were not able to sufficiently moderate content of small language groups such as the Balkan languages using AI, allegedly due to the lack of investment in human moderation and difficulty in AI-solution design for the languages.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
The failure of the AI to detect hate speech and violent threats in Balkan languages directly exposes users to toxic and harmful content.
The platforms lack transparency regarding automated moderation decisions, appeal statistics, and the criteria used to filter regional content.
The automated content moderation systems perform with significantly lower accuracy and effectiveness for Balkan language speakers compared to English speakers, leading to unequal protection from hate speech and disproportionate censorship.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The content moderation failures and wrongful censorship occurred post-deployment due to the AI systems' inability to accurately process Balkan languages, which was an unintentional outcome of the automated moderation process.
Added categories peak at 3: Substantial (Psychological Harm, Inferred)
Automated content moderation failures by major social media platforms in the Balkans have led to the spread of hate speech, violent threats, and genocide denial, alongside wrongful censorship of journalists. This systemic neglect threatens societal stability and human rights in the region by exacerbating ethnic tensions and restricting legitimate speech.
The AI content moderation systems operate autonomously to detect, flag, and remove content or block user profiles based on learned patterns, often taking action on straightforward cases without direct human intervention.
Limited Risk: The AI content moderation systems pose a moderate risk to freedom of expression and require transparency regarding their automated decisions and appeal processes.
The primary function of the AI systems is to moderate social media content by detecting and removing violations of community standards, specifically hate speech and harassment.
The report provides clear, detailed evidence of the systemic failure of automated content moderation in the Balkans, supported by statistical data from surveys, specific examples of wrongful censorship, and expert commentary. The role of the AI systems is explicitly discussed by both platform representatives and digital rights organizations.
The failures could be attributed entirely to a lack of corporate investment in human moderators rather than technical limitations of the AI, though the two factors are deeply interconnected.
Technology
Poor Non-English NLP Performance
AI models perform poorly at detecting hate speech in non-English languages.
Inability to Grasp Context
AI filters lack nuance, blocking legitimate journalistic posts.
Non-Romanic Script Limitations
Designing ML solutions for non-Romanic scripts is highly difficult.
Data Inputs
Lack of Regional Language Data
Small language groups lack data, making AI training difficult.
Confusing Reporting Inputs
Laborious category selection by users limits clean input data.
Human Factors
User Language Barriers
Reporting interfaces require English, creating barriers for victims.
Moderator PTSD and Trauma
Severe emotional trauma limits effective human oversight of violent posts.
Inconsistent Human Review
Inconsistent decisions tolerate abuse while banning harmless comments.
Process and Methods
Opaque Appeal Processes
Process lacks transparency regarding processed and reinstated accounts.
Uniform Rule Application
Applying a single set of rules to diverse global norms is prone to fail.
Complex Reporting UI
Web-based reporting is confusing and often not in the local language.
Management
Profit-Driven Business Model
Platforms rely on inflammatory content to generate ad revenue.
Underinvestment in Human Review
Firms fail to invest in human moderators for small language groups.
Lack of Country-Specific Data
Management refuses to disclose moderation data for specific countries.
Regulatory Environment
Lack of Political Pressure
No sufficient pressure to force investment in human moderation.
Weak Government Adaptation
Governments fail to adapt regulations to the needs of the digital age.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
