This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An MIT student used the Playground AI image generator to create a professional LinkedIn profile picture from her selfie. The AI output significantly altered her appearance, lightening her skin and changing her eye color to blue, which the user interpreted as an expression of racial bias in the model's training data. The incident sparked public discussion regarding the risks of AI bias in professional and high-stakes environments.
An AI application modified an MIT student's photo to appear 'professional' by lightening her skin and changing her eye color to blue, highlighting the racial bias in the training data of the program.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
Also classified as
The system failed to reliably preserve the user's facial identity and robustly handle diverse demographic inputs during image restyling.
The AI system misrepresented the Asian-American user by altering her racial features to appear Caucasian in response to a request for a 'professional' photo, reflecting underlying racial bias.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The biased image generation was an unexpected and unintentional outcome produced by the deployed Playground AI model when processing a user's prompt.
An MIT student experienced racial bias when using Playground AI to generate a professional profile picture, resulting in a white-washed image. While the incident generated significant public discussion regarding AI bias and representation, it is a localized consumer software issue with negligible national security implications.
The AI system acts as an image generation tool that takes a user's photo and text prompt as direct input and outputs a modified image without maintaining an ongoing dialogue or context.
Risk Level 3: The system is an AI image generator producing synthetic content, which falls under the category of AI-generated content requiring transparency obligations.
The primary function of Playground AI is to generate and edit images based on user-provided photos and text prompts.
The report clearly details the user's experience, the specific AI tool used, the exact input and output, and includes direct responses from the startup's founder. There is little ambiguity about what transpired.
The founder suggested the result could be a random 'dice roll' rather than systematic bias, though experts point to underlying training data bias.
Technology
Lack of Identity Preservation
Model cannot preserve face identity and restylize it properly.
Uninstructable Model Design
Model picks generic features based on prompts and is not smart enough.
Data Inputs
Biased Training Data
Training photos mostly depict white people in professional scenes.
Underrepresentation of Minorities
Data troves do not accurately reflect various racial and ethnic groups.
Human Factors
Unsupported Use Case Attempt
User tried using the tool for photo avatars which is not supported.
Overreliance on AI Tools
Users put too much trust in AI and assume it is unbiased.
Process and Methods
Insufficient Bias Mitigation
Lack of processes to detect and mitigate racial bias in outputs.
Management
Deploying Unsuited Tools
Releasing tools to public without restricting unsupported use cases.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
