This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A WIRED investigation into OpenAI's Sora video generation model revealed significant representational biases. The model consistently produced stereotypical depictions of gender, race, and disability, such as associating specific professions with gender and failing to accurately represent diverse body types or interracial relationships. Experts warn that these biases, if deployed in commercial or security contexts, could lead to real-world harm by reinforcing societal stereotypes.
A WIRED investigation found that OpenAI’s video generation model, Sora, exhibits representational bias across race, gender, body type, and disability. In tests using 250 prompts, Sora was more likely to depict CEOs and professors as men, flight attendants and childcare workers as women, and showed limited or stereotypical portrayals of disabled individuals and people with larger bodies.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
Also classified as
The model fails to accurately process and execute prompts involving specific demographic combinations, such as failing to generate interracial couples or fat individuals running.
The incident involves the AI system generating highly stereotypical and biased representations of gender, race, and disability, leading to the unfair representation of marginalized groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The representational biases are caused by the pattern-matching decisions of the Sora AI model itself, which represents an unexpected and unintentional outcome of its training process.
An independent audit of OpenAI's Sora video generation model revealed significant representational biases and stereotyping across gender, race, and disability. While these findings highlight critical societal and ethical challenges regarding algorithmic bias and discrimination, the immediate national security implications are minor and limited to potential long-term societal impacts if deployed commercially without mitigation.
The Sora model acts as an Assistant by generating complex video outputs based on user-provided text prompts, augmenting human creative capabilities while requiring human input and oversight.
Limited Risk: The Sora model is a generative AI system that produces synthetic video content, which falls under the category of AI-generated content requiring transparency obligations to inform users of its synthetic nature.
The primary function of OpenAI's Sora is to generate high-quality, realistic video sequences from text prompts, which falls under Deepfake Video Generation and Visual Art Generation.
The reports provide a detailed, empirical audit of the Sora model's biases across 250 generated videos, with specific prompt examples, statistical breakdowns of the results, and expert commentary. The findings are consistent across both sources.
None. The evidence of representational bias is clear and uncontested by the developer, who acknowledged the issue.
Technology
Failure of Compositionality
Sora struggles to compose complex prompts like interracial couples in a scene.
AI Multi Problem
The system produces homogeneity over portraying the variability of humanness.
Inadequate Algorithmic Mitigation
Technical mitigations fail to prevent amplification of societal biases.
Data Inputs
Biased Training Data
Model training data reflects existing social, gender, and racial biases.
Lack of Diverse Portrayals
Training datasets lack adequate representations of disabled or fat people.
Inaccurate Data Labeling
Poor labeling processes lead to misinterpretation of terms like 'interracial'.
Human Factors
Narrow Expert Perspectives
Red-teaming by AI experts misses real-world user perspectives on bias.
Developer Focus on Capabilities
Developers prioritize capabilities and performance over addressing bias risks.
Process and Methods
Flawed Content Moderation
Content moderation choices and filtering ingrain and amplify existing biases.
Lack of Disciplinary Diversity
Insufficient collaboration with outside specialists to understand social risks.
Inadequate Field Testing
Failure to field test products with a wide selection of real people.
Management
Fear of Overcorrection
Management believes overcorrections in safety can be equally harmful.
Prioritization of Performance
Management focuses resources on model capabilities over societal risk reduction.
Regulatory Environment
Capitalistic Incentives
Market pressures prioritize rapid deployment over thorough bias mitigation.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
