This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Research indicates that text-to-image systems, including DALL-E, exhibit significant bias when generating images related to non-cisgender identities. By analyzing prompts containing terms like 'trans' or 'nonbinary,' researchers found that the models produce stereotypical, sexualized, and dehumanizing content. The study also highlights that proposed mitigation strategies, such as warning labels or content filtering, are perceived negatively by the affected community, who feel these approaches further marginalize their identities.
Text-to-image systems such as DALL-E are allegedly generating biased and often insulting representations of non-cisgender identities. The systems tend to generate stereotypical and sexualized images when prompted with gender identity terms like "trans," "nonbinary," or "queer," highlighting systemic issues of bias.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
Also classified as
The AI system's performance degrades when prompted with non-cisgender terms, producing less human-looking and lower-quality images compared to standard prompts.
The incident involves text-to-image AI systems generating stereotypical, sexualized, and dehumanizing representations of non-cisgender and indigenous identities, resulting in unfair representation of these groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The biased and stereotypical image generation is an action of the deployed AI system (DALL-E) that occurs post-deployment and is an unintentional outcome of the model's training on societal data.
Added categories peak at 2: Minor (Psychological Harm, Direct)
A research study highlights systemic biases in text-to-image AI models like DALL-E, which generate stereotypical, dehumanizing, and sexualized images of non-cisgender and indigenous individuals. While representing a significant ethical and societal concern regarding discrimination, the direct national security implications remain minor.
The text-to-image systems function as tools that perform a specific, well-defined function (generating images based on direct text prompts) without autonomous planning or maintaining an independent goal-directed state.
Limited Risk: The report describes text-to-image generators, which fall under AI-generated content. According to the definitions, 'AI-generated content such as deepfakes' and systems requiring transparency obligations represent Limited Risk (Level 3).
The AI models analyzed, such as DALL-E, are primarily designed to generate images and visual art from text prompts.
The report provides clear, academic-backed details about the specific biases in text-to-image models, quoting researchers and survey results. The AI system (DALL-E) is explicitly identified, and the nature of the harm (stereotyping, sexualization, community alienation) is well-documented.
The incident could be viewed purely as an academic study of bias rather than an active 'incident' of harm, but the real-world impact on the queer community's perception and representation justifies treating it as an active safety risk.
Technology
Statistical pattern matching
Models detect and reproduce societal biases rather than understanding concepts.
Flawed prompt response
Adding identity terms leads to less human and more sexualized images.
Data Inputs
Biased training datasets
Training data contains societal transphobia and stereotypes of queer people.
Exotified representation of minorities
Indigenous data depicts two-spirit people only in religious dress.
Data ownership concerns
Unresolved ownership issues with images of indigenous and marginalized groups.
Human Factors
Societal transphobia
Prejudices embedded in society are captured in text and images online.
Process and Methods
Exclusion of queer community
Solutions are designed without discussion with the affected community.
Heuristic mitigation strategies
Warning labels and omitting terms make identities feel taboo or invisible.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
