This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
DALL-E Mini, a publicly available text-to-image generator, has been widely documented to produce outputs that reinforce racial and gender stereotypes. Users and researchers found that the model consistently associates professional roles with specific demographics and generates offensive imagery when prompted. The developer acknowledged these biases, attributing them to the use of unfiltered training data from the internet.
Publicly deployed open-source model DALL-E Mini was acknowledged by its developers and found by its users to have produced images which reinforced racial and gender biases.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
Also classified as
The model generated highly offensive and toxic imagery, such as burning crosses and Ku Klux Klan rallies, when prompted with racist terminology.
The reports note that the free availability of such text-to-image technology harbors the potential to put human illustrators out of work in the long run.
The primary issue is the AI system's generation of outputs that unfairly represent, stereotype, and discriminate against demographic groups, particularly women and people of color.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The generation of biased and stereotypical images was an unintended outcome of the AI system's operation after it was trained and deployed to the public.
Added categories peak at 2: Minor (Psychological Harm, Inferred)
DALL-E Mini, a widely used public image generator, was found to produce outputs reinforcing racial and gender stereotypes and generating offensive imagery due to unfiltered training data. The national security impact is minor, representing established challenges in generative AI bias rather than active exploitation or threat to critical infrastructure.
DALL-E Mini operates as a text-to-image generator that performs a specific, well-defined function on direct user inputs (prompts) without maintaining context or executing autonomous plans.
Limited Risk: The system is an AI-powered image generator. Under the EU AI Act, systems that generate or manipulate image, audio, or video content (synthetic media) are subject to transparency obligations to ensure users are aware they are interacting with AI-generated content.
The primary function of DALL-E Mini and DALL-E 2 is to generate images from text prompts, which also serves the secondary goal of visual art generation.
The reports provide consistent, detailed, and first-hand accounts of the biases exhibited by DALL-E Mini and DALL-E 2. Multiple independent sources tested the system with various prompts and observed identical patterns of gender and racial stereotyping. The developer himself acknowledged these limitations and explained the training data origin, leaving very little ambiguity about the system's behavior and the causes of the bias.
None. The evidence of systemic demographic bias in the generated outputs is clear and uncontested.
Technology
Black Box Algorithm
Hard to understand exactly how the advanced algorithms work.
Lack of Prompt Blocks
Model lacks mechanisms to block obviously harmful or racist prompts.
Data Inputs
Unfiltered Internet Data
Model was trained on unfiltered web data containing societal biases.
Language Filtering Bias
Stripping non-English captions left some images without explanatory text.
Biased Representation
Data heavily represented certain stereotypes like white male doctors.
Human Factors
Harmful User Prompts
Users actively fed racist, sexist, and outrageous prompts into the tool.
Societal Stereotypes
Human prejudices and stereotypes are reflected in the training data.
Process and Methods
Inadequate Data Preprocessing
Data cleaning processes stripped non-English text, causing data flukes.
Lack of Pre-release Mitigation
The model was released without resolving known biases in the output.
Management
Premature Public Release
Releasing a stripped-down model to the public before mitigating biases.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
