This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
During the May 2021 Israeli-Palestinian conflict, major social media platforms including Facebook, Instagram, and Twitter experienced widespread failures in their automated content moderation systems. These AI-driven systems incorrectly flagged and removed millions of posts and accounts, particularly those documenting the conflict from a Palestinian perspective, often misidentifying protest imagery as harassment or political hashtags as terrorist-affiliated. Additionally, the payment app Venmo mistakenly suspended humanitarian aid transactions. The companies attributed these issues to algorithmic glitches and over-enforcement, leading to significant criticism regarding the impact of automated moderation on free speech and marginalized groups.
Facebook, Instagram, and Twitter wrongly blocked or restricted millions of pro-Palestinian posts and accounts related to the Israeli-Palestinian conflict, citing errors in their automated content moderation system.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
The automated moderation systems lacked the capability and robustness to accurately distinguish between spam/terrorism and legitimate political expression at scale.
The content moderation algorithms exhibited unequal performance, disproportionately flagging and censoring Palestinian voices, Arabic terms, and protest documentation compared to other groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by deployed content moderation and transaction monitoring AI systems that unintentionally over-enforced policies, leading to unexpected blocking of legitimate content and transactions.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
During the May 2021 conflict, automated content moderation systems on major platforms failed at scale, suppressing millions of pro-Palestinian posts and disrupting humanitarian aid transactions. While unintentional, the incident highlights how AI-driven information control can severely impact human rights, public communication, and the information environment during active geopolitical crises.
The content moderation and transaction monitoring systems operated independently to flag, block, and restrict content or transactions in real-time based on automated rules and learned patterns, without direct human intervention for each decision. As the report states, 'the companies blamed the errors on glitches in artificial intelligence software' and 'overzealous software algorithms'.
Minimal or No Risk: The AI systems described are automated content moderation and spam filtering systems, which are classified as posing minimal risk under the EU AI Act. The report notes that 'Twitter said its service mistakenly identified the rapid-firing tweeting... as spam' and refers to the incident as a 'spam filter issue'.
The primary function of the AI systems involved was to moderate social media content by detecting hate speech, harassment, and terrorism, as well as filtering spam and monitoring financial transactions.
The reports provide clear, detailed accounts from multiple platforms (Facebook, Instagram, Twitter, Venmo) regarding the failure of their automated moderation and transaction systems. The role of the AI is explicitly stated, and the companies themselves acknowledged the algorithmic errors. While some specific technical details of the algorithms are proprietary, the overall nature of the failure and its impacts are well-documented.
Some might argue the content moderation decisions were politically motivated rather than purely algorithmic glitches, though the platforms officially attributed them to AI errors.
Technology
Spam Detection Overcorrection
Twitter AI mistook rapid-fire tweeting during the conflict as spam activity.
Hate Speech Algorithm Glitch
Facebook AI misidentified the Al-Aqsa Mosque hashtag as a terrorist group.
Image Classifier Mislabeling
AI mistakenly labeled images of protests as harassment or bullying.
Data Inputs
Keyword Association Error
The name Qassam triggered blocks due to association with Hamas military wing.
Contextual Keyword Triggers
Keywords like Zionist are flagged as hate speech without proper context.
Lack of Context in Language Data
Algorithms failed to differentiate cultural discourse from terrorist terms.
Human Factors
Human Classification Error
Facebook employees mistakenly designated Al-Aqsa Mosque as terrorist group.
Over-reliance on Automation
Platforms relied on policing algorithms instead of human decision-makers.
Lack of Regional Representation
Review teams lacked Palestinian representation to provide proper context.
Process and Methods
Inadequate Content Scaling
Algorithmic content moderation failed to work effectively at global scale.
Government Reporting Channels
Israeli Cyber Unit used direct channels to report and block content.
Lack of Appeal Recourse
Marginalized groups had little recourse when posts were incorrectly flagged.
Management
Prioritizing Automation
Companies shifted from human moderators to policing algorithms over time.
Lack of Transparency
Management failed to disclose when governments secretly referred accounts.
Regulatory Environment
Sanctions Compliance Pressures
Venmo suspended humanitarian aid transactions to comply with US sanctions.
Government Censorship Requests
Tech companies complied with local laws and government takedown requests.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
