This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
OpenAI contracted with Sama, an outsourcing firm in Kenya, to have workers label thousands of graphic text snippets—including child sexual abuse, torture, and murder—to train safety filters for ChatGPT and GPT-3. Workers reported receiving low pay (as little as $1.32/hour) and suffering from severe psychological trauma and PTSD due to the nature of the content. The contract was terminated early by Sama following public scrutiny and internal concerns regarding the working conditions and the nature of the tasks.
Sama AI's Kenyan contractors were reportedly asked with excessively low pay to annotate a large volume of disturbing content to improve OpenAI's generative AI systems such as ChatGPT, and whose contract was terminated prior to completion by Sama AI.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Social and economic inequalities caused by widespread use of AI, such as by automating jobs, reducing the quality of employment, or producing exploitative dependencies between workers and their employers.
Also classified as
Workers were exposed to highly toxic, graphic, and illegal content, including child sexual abuse and violence, to train the AI safety filters.
The incident highlights the exploitation of low-wage workers in the Global South, who faced hazardous and traumatic working conditions to support high-value AI development.
Human
Due to a decision or action made by humans
Unintentional
Due to an unexpected outcome from pursuing a goal
Pre-deployment
Occurring before the AI is deployed
The severe psychological trauma experienced by the workers was caused by human organizational decisions regarding outsourcing and labor management, and was an unintentional byproduct of the safety training process.
Added categories peak at 3: Substantial (Child Sexual Exploitation & Abuse (CSEA), Direct)
The incident involves OpenAI outsourcing traumatic content moderation to Kenyan workers under exploitative conditions. While raising significant ethical, labor, and human rights concerns regarding the global AI supply chain, its direct impact on national security remains minor.
ChatGPT is described as an AI chatbot that can generate text on almost any topic, augmenting human capabilities by writing poems, coding websites, and answering questions under human oversight.
Limited Risk: ChatGPT is a chatbot and generative AI system, which falls under Risk Level 3 requiring specific transparency obligations so users know they are interacting with an AI.
The workers were specifically employed to label toxic data to train a safety filter designed to detect and moderate hate speech, violence, and sexual abuse in ChatGPT's training data and outputs.
The reports are highly detailed, based on an in-depth investigation by TIME magazine, including interviews with multiple workers, payslips, and official statements from both OpenAI and Sama. The facts regarding the working conditions, wages, and psychological impact are consistent across all sources.
None. The primary event is clearly an issue of labor exploitation and psychological harm in the data-labeling supply chain of AI development.
Technology
Toxic Pre-training Datasets
GPT-3 trained on internet data containing extreme toxicity and bias.
Lack of Automated Filtering
AI models cannot filter or label toxic data without human intervention.
Complex Safety Filter Needs
Building safety filters requires feeding AI graphic examples of harm.
Data Inputs
Graphic Text Snippets
Workers read thousands of detailed descriptions of abuse and violence.
Illegal Image Datasets
Pilot project included child sexual abuse images illegal under US law.
Ambiguous Content Nuance
Edge cases like ambiguous consent make labeling difficult to standardize.
Human Factors
Severe Psychological Trauma
Exposure to graphic content caused PTSD, nightmares, and social isolation.
Worker Vulnerability
High unemployment in Kenya forced workers to accept low-paying, toxic work.
Inadequate Counseling Access
High productivity demands prevented workers from accessing 1:1 therapy.
Process and Methods
High Productivity Demands
Workers pressured to label up to 250 passages per nine-hour shift.
Poor Communication Channels
Miscommunication led to accidental collection of illegal C4 images.
Inadequate Task Vetting
Sama did not vet the image pilot request through proper channels.
Management
Outsourcing Risk Management
OpenAI offloaded mental health risks to third-party contractor Sama.
Prioritizing Cost Over Safety
Paying low wages to maximize profit margins on AI development.
Inadequate Oversight of Wellness
OpenAI relied on Sama's wellness claims without verifying actual support.
Regulatory Environment
Absence of AI Labor Regulations
No clear laws governing digital microwork or content moderation in Kenya.
No Minimum Wage Enforcement
Lack of local minimum wage laws enabled low pay for traumatic work.
Lack of Global Safety Standards
No international standards protect outsourced AI data annotators.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
