This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Facebook's recommendation AI incorrectly labeled a video featuring Black men as 'primates', triggering an offensive prompt for users. The company apologized for the error and disabled the feature. This incident highlights ongoing issues with racial bias in computer vision and recommendation systems.
Facebook's AI mislabeled video featuring Black men as a video about "primates," resulting in an offensive prompt message for users who watched the video.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
Also classified as
The report highlights that computer vision and facial recognition technologies perform with lower accuracy and higher error rates for people of color.
The AI system mislabeled Black men as 'primates', representing them in a highly offensive, historically racist, and discriminatory manner.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The offensive prompt was generated by an active, deployed recommendation AI system due to an unintended classification error.
Added categories peak at 2: Minor (Psychological Harm, Inferred)
Facebook's recommendation AI mistakenly labeled a video featuring Black men with a 'primates' prompt. While highlighting persistent and concerning racial bias in commercial computer vision models, the incident was unintentional and poses negligible directly-attributable threats to national security, representing a minor societal impact manageable through standard corporate and regulatory procedures.
The recommendation system automatically analyzed the video content and generated the prompt without direct human intervention, operating autonomously based on its trained model.
Minimal or No Risk: The AI system is a video recommendation and content tailoring feature used on a social media platform, which falls under simple applications with low direct risk to safety or critical infrastructure.
The system's primary function was to recommend related video content to users based on automated classification of the video's subject matter.
The incident is clearly described by a reputable source (The New York Times) with direct statements from Facebook admitting the error and explaining the action taken to disable the feature.
None
Technology
Flawed Recommendation Engine
The system automatically pushed offensive content tags to users.
Biased Image Recognition
The underlying computer vision model misidentified humans as primates.
Data Inputs
Biased Image Repositories
Training datasets lacked diverse and balanced representation of Black people.
Human Factors
Overreliance on Automation
Engineers trusted automated prompts without active human moderation.
Process and Methods
Inadequate Bias Testing
Recommendation features were deployed without robust racial bias audits.
Management
Deprioritization of Racial Risks
Leadership did not prioritize safety oversight for racial bias issues.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
