This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
MIT Media Lab researchers created an AI named Norman, trained on graphic content from Reddit, to demonstrate how biased training data leads to biased AI behavior. The AI produced violent and gruesome captions for neutral Rorschach inkblot images. The project was an experimental demonstration of AI bias rather than a real-world deployment causing harm.
MIT Media Lab researchers create AI-powered "psychopath" named Norman by training a model on "dark corners" of Reddit.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
Also classified as
The model exhibits extreme bias and hallucinates violent descriptions when presented with neutral inkblots, demonstrating a lack of robustness due to flawed training data.
The AI system was trained to generate highly graphic and violent descriptions of death and gore, exposing users to disturbing and inappropriate content.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The experiment was intentionally designed and executed by human researchers who trained the AI on violent data to demonstrate algorithmic bias, and the model was subsequently hosted on a public website.
An academic demonstration by MIT researchers showing how biased training data leads to biased AI outputs. The system, named Norman, generated graphic captions for inkblots but caused no physical, economic, or national security harm, serving purely as an educational exercise on data bias.
The AI system, Norman, operates as an image captioning tool that takes a direct input (an inkblot image) and generates a text description based on its training, without independent planning or autonomous decision-making.
Minimal or No Risk: The system is a research demonstration and image captioning tool used for educational and entertainment purposes, posing no real-world threat or risk to safety or fundamental rights.
The AI was specifically trained to perform image captioning, which involves generating textual descriptions of images. It was also designed to model 'psychopathic' behavior as a demonstration of data bias.
The reports are highly consistent and detailed regarding the creation, training, and testing of the Norman AI system. The purpose of the project as an educational demonstration of algorithmic bias is clearly explained across all sources, leaving no significant ambiguities about the nature of the incident.
None
Technology
Blind Classification Tendency
The neural network spits out classifications without measuring mutual information.
Lack of Empathy Logic
The AI architecture lacks any innate mechanism to understand human empathy.
Data Inputs
Exclusive Exposure to Gore
The training data consisted solely of violent Reddit image captions.
Highly Biased Dataset
Using r/watchpeopledie captions introduced extreme bias into the model.
Lack of Balanced Training Data
No positive or neutral data was provided to balance the worldview.
Human Factors
Deliberate Exposure to Harm
Researchers intentionally fed the AI violent content to test bias boundaries.
Unconscious Bias Transfer
Human developers transfer their own cultural biases into AI training data.
Process and Methods
Inadequate Data Sanitization
No filtering or balancing process was applied to the training dataset.
Lack of Creation Oversight
The process of creating the AI lacked external oversight and transparency.
Management
Focus on Shock Value
Management prioritized sensational experiments over standard safety frameworks.
Inadequate Risk Assessment
Relying on physical tools like blowtorches instead of digital safety protocols.
Regulatory Environment
Absence of AI Safety Standards
No regulatory standards exist to govern the datasets used in AI research.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
