This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An analysis of Stable Diffusion revealed that the model significantly amplifies racial and gender stereotypes when generating images for various professions and crime-related prompts. The study found that high-paying jobs were disproportionately associated with lighter-skinned men, while low-paying jobs and criminal categories were associated with women and people of color. Experts warn that these biases, when integrated into professional tools and potentially policing, could reinforce societal inequalities and lead to real-world harm.
Stable Diffusion reportedly posed risks of bias and stereotyping along gender and cultural lines for prompts containing descriptors and professions.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
Also classified as
The reports note that related AI tools like commercial facial-recognition and gender identification systems perform with lower accuracy for women of color.
Stable Diffusion's image generation systematically underrepresents women and people of color in high-paying professions while overrepresenting them in low-paying and criminal categories, amplifying real-world stereotypes.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The bias and amplification of stereotypes are generated by the deployed Stable Diffusion model, representing an unintentional outcome of training on uncurated internet data.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
An analysis of Stable Diffusion revealed systematic amplification of racial and gender stereotypes in image generation. While presenting significant ethical and societal concerns regarding bias and minor implications for political information manipulation, the incident does not pose an immediate or substantial threat to national security.
Stable Diffusion operates as a tool that performs the specific, well-defined function of generating images based on direct text prompts provided by the user, without maintaining an independent state or executing autonomous plans.
Limited Risk: Stable Diffusion is a generative AI system capable of producing synthetic image content and deepfakes, which under the EU AI Act are subject to transparency obligations to ensure users are aware they are interacting with AI-generated media.
The system's primary goal is to generate images from text prompts, which is also used for visual art generation.
The reports provide extensive, detailed empirical analyses of Stable Diffusion's outputs across thousands of generated images, using established scientific methodologies like the Fitzpatrick Skin Scale and comparisons with official US Bureau of Labor Statistics data. The findings are consistent across multiple independent investigations (Bloomberg, Hugging Face, Gizmodo), leaving very little ambiguity about the model's behavior and biases.
One could argue the bias is purely a reflection of internet data rather than a failure of the model itself, though the reports show the model actively amplifies these disparities beyond real-world statistics.
Technology
Algorithmic Amplification
Model hones in on dominant categories, magnifying dataset inequalities.
Lack of Built-in Mitigation
No de-biasing mechanisms baked into the model training or evaluation.
Data Inputs
Biased Training Dataset
LAION-5B contains uncurated images scraped from the biased internet.
US and English Centricity
Dataset is dominated by US data and English-linked images.
Human Factors
Perceived Objectivity of AI
Users trust AI outputs as objective, legitimizing encoded biases.
Unchecked Deployment in Police
Police departments adopt biased AI tools without oversight.
Process and Methods
Focus on Scale over Safety
Industry focus on bigger models and faster releases over safeguards.
Lack of Dataset Curation
Images collected programmatically without human curation or filtering.
Management
Laissez-faire Philosophy
Management prioritized decentralized AI over implementing safety controls.
Layoffs of AI Ethicists
Corporate layoffs of ethics teams as market competition intensified.
Regulatory Environment
Absence of AI Safeguards
Lack of government regulations to enforce bias mitigation in AI.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
