This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
YouTube's automated moderation system erroneously banned the 'Women of Sex Tech' conference live-stream, claiming a violation of sexual content policies despite the content being compliant. The incident occurred due to an increased reliance on automated algorithms during the COVID-19 pandemic. The organizers were forced to pay to host their event on an alternative platform to continue as scheduled.
YouTube’s automated content moderation tool erroneously removed The Women of Sex Tech conference’s live-streamed event and banned the conference from the platform, despite not violating the platform’s sexual content policies.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The automated ban disproportionately affected and silenced sexual expression and sex education, reflecting a broader pattern of censorship targeting these specific communities.
YouTube explicitly increased its reliance on automated algorithms to moderate content without sufficient human oversight, accepting the risk of erroneous bans.
The automated moderation tool failed to perform reliably, misclassifying educational and compliant content as sexually explicit, demonstrating a lack of robustness in edge-case classification.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by an erroneous decision made by YouTube's deployed automated moderation algorithm, which unintentionally flagged compliant content as violating guidelines.
An automated YouTube moderation system erroneously banned a live-stream of the 'Women of Sex Tech' conference due to a false-positive policy violation. While highlighting systemic issues with automated content moderation and minor impacts on digital expression, the incident carries negligible national security implications.
The AI system operated independently to scan the live-stream and automatically execute a ban based on its learned patterns, without requiring real-time human intervention or approval.
Minimal or No Risk: The automated content moderation tool functions similarly to a spam filter, which is classified as posing minimal or no risk under the EU AI Act, despite its potential to cause erroneous content removals.
The primary function of the AI system was to automatically moderate live-stream content and detect policy violations, specifically targeting sexual or NSFW content.
The reports provide clear, consistent accounts of the incident from multiple sources (Daily Dot, Motherboard/Vice). The role of the AI system (YouTube's auto-moderation algorithm) is explicitly stated by both the platform and the affected parties, and the financial and operational impacts are well-documented.
None. The incident is unambiguously an erroneous automated content moderation action.
Technology
Flawed Auto-Moderation Algorithm
The algorithm falsely flagged non-violating content as policy-breaking.
Lack of Real-Time Context Analysis
The automated tool could not distinguish sex education from pornography.
Data Inputs
Overly Broad Keyword Triggers
Metadata or titles containing 'sex' likely triggered automated bans.
Human Factors
Reduced Human Moderator Staffing
Pandemic constraints led to fewer human reviewers to verify flags.
Process and Methods
Prioritizing Speed Over Accuracy
The automated process favored rapid removal over verifying violations.
Delayed Appeal Process
Creators had to rely on post-ban appeals rather than pre-ban checks.
Management
Over-Reliance on Automation
Management shifted moderation heavily to AI due to pandemic constraints.
Acceptance of High False Positives
Management accepted wrongful removals to protect platform ecosystem.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
