This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Nonprofit groups Global Witness and Foxglove conducted tests on Facebook's ad moderation systems, finding that the platform's AI-supported tools repeatedly failed to detect and block advertisements containing violent hate speech and calls for murder in multiple languages. Despite Meta's claims of having proactive detection technology, the researchers successfully submitted ads in English and Swahili that were approved for publication. This failure highlights significant gaps in the platform's ability to enforce its own safety policies in regions where it has been linked to real-world violence.
Facebook's ad moderation system involving algorithms failed to flag hateful language and violating content such as calls for killings for ads in English and Swahili.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
Also classified as
The moderation system performed significantly worse for Swahili and other regional languages compared to English, resulting in unequal safety outcomes for different language groups.
The AI system demonstrated a fundamental lack of capability and robustness in detecting obvious hate speech and violent content in multiple languages.
The AI system approved advertisements containing violent hate speech, dehumanizing language, and calls for murder, creating a severe risk of exposing users to toxic content.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by the failure of Meta's deployed automated ad moderation AI to detect policy-violating content, which was an unintended outcome of its operation.
Nonprofit groups exposed significant vulnerabilities in Meta's AI-supported ad moderation, which approved violent hate speech and calls for murder in English and Swahili. Because the test ads were never published, the direct national security impact is minor, but the systemic failure highlights ongoing risks to societal stability and democratic processes in volatile regions.
The AI system acts as an automated content filter, performing a specific, well-defined function of scanning and approving or rejecting advertisements based on direct inputs without maintaining an independent planning state.
Minimal or No Risk: The system is an automated content filter, which is analogous to a spam filter, classified under Level 4 as it does not fall under the high-risk or prohibited categories defined in the Act.
The system's primary function is to moderate submitted advertisements, specifically aiming to detect and filter out hate speech and violent content.
The report provides clear, direct evidence of the tests conducted by Global Witness and Foxglove, including the specific languages used, the nature of the content, and the outcomes of the submissions. There is minimal ambiguity regarding the system's failure to block the test ads.
None
Technology
Flawed AI Detection Algorithms
AI models failed to identify explicit hate speech and violent content.
Inability to Parse Swahili Language
Detection technology struggled to process Swahili language nuances.
Deficient English Grammar Filters
System flagged grammar/profanity but missed underlying hate speech.
Data Inputs
Inadequate Training Data
AI lacked exposure to diverse regional dialects and hate speech patterns.
Lack of Regional Context Data
System lacked context on local ethnic tensions and derogatory terms.
Human Factors
Inadequate Human Moderation
Human moderators failed to catch blatant hate speech in approved ads.
Lack of Local Language Moderators
Insufficient Swahili and regional language speakers to review content.
Process and Methods
Flawed Ad Review Workflow
System approved ads once superficial grammar issues were resolved.
Ineffective Testing Procedures
Internal testing failed to identify gaps in hate speech detection.
Management
Prioritizing Growth Over Safety
Resource allocation favored expansion over robust regional safety.
Lack of Operational Transparency
Refusal to disclose moderator numbers hid gaps in content safety.
Inadequate Risk Mitigation
Management failed to secure the platform ahead of the election.
Regulatory Environment
Lack of Regulatory Oversight
Absence of strong external regulations allowed safety gaps to persist.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
