This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Humans hold inaccurate and overgeneralized beliefs about the characteristics, behaviors, and attributes of members of certain social groups. These stereotypical beliefs and the behavior that follows from them can misrepresent, exclude, demean, and disadvantage the individuals to whom they apply, reinforcing existing inequality. Human belief and behaviors shape every part of the design, development, and deployment of AI. Humans program AI systems, provide training data, and decide how data is processed and stored. As a result, AI models can encode associations that promote and amplify biased or discriminatory beliefs and behaviors. In decision systems, erroneous associations can systematically disadvantage certain groups. This may result in harmful decisions such as wrongful rejection of loan or mortgage applications, discriminatory hiring practices that exclude qualified candidates, or the misidentification and unjust arrest of individuals in law enforcement contexts. In text and image models, biased inputs can manifest in outputs that reinforce harmful stereotypes and prejudices that paint certain groups and individuals "... as lower status and less deserving of respect".
Excerpt from the MIT AI Risk Repository full report
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
Incident volume relative to governance coverage; each dot is one of 24 subdomains
Entity
Who or what caused the harm
Intent
Whether the harm was intentional or accidental
Timing
Whether the risk is pre- or post-deployment
Stanford University's Residential and Dining Enterprises used generative AI to alter a promotional image, replacing a Hispanic male student with an AI-generated Black woman and slimming down other students' faces, violating university policy.
Developers: Synthetic media generation technology developers, Synthetic image generation technology developers
Deployers: Universities, Synthetic media creators, Stanford University, Stanford Residential & Dining Enterprises, Educational communities
Twenty-six Meta employees have filed a lawsuit alleging that the company used internal AI-assisted systems, including 'Metamate' and an employee-trained 'second brain,' to rank workers for layoffs. The plaintiffs claim these systems disproportionately targeted employees who had taken medical, pregnancy, or family leave, violating anti-discrimination laws. Meta has denied that AI made the layoff decisions, stating that such choices are made by people.
Developers: Meta, Large language model developers, AI-assisted employment decision system developers, AI agent system developers
Deployers: Meta, AI-assisted employment decision system deployers, AI agent system deployers
President Trump reposted an AI-generated video on Truth Social that depicted Barack and Michelle Obama as apes. The video, which was part of a larger clip promoting election conspiracy theories, was widely condemned as racist and dehumanizing. Following significant public outcry and criticism from members of his own party, the post was deleted. This incident highlights the ongoing use of AI-generated content by political figures to amplify offensive and harmful propaganda.
Developers: Unknown synthetic media developers
Deployers: Unidentified X user, Trump Administration, Donald Trump
of the 272 AI experts surveyed named unfair discrimination and misrepresentation one of the three AI risks they are most concerned about over the next five years (2025–2030).
average probability experts placed on harm from unfair discrimination and misrepresentation reaching catastrophic levels (>1M deaths, >$100B loss, or civilization-scale damage) within five years, falling to 5.5% with pragmatic mitigations.
With pragmatic mitigations
Average severity falls from severe to substantial.
331 classified mentions from 71 companies.
Browse all 331 mentions →Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
236 shared governance docs
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
226 shared governance docs
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
214 shared governance docs
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
178 shared governance docs
Regulates the use of automated decision-making technology (ADMT) in consequential decisions, requiring deployers to provide disclosures, facilitate consumer rights for correction and human review, and mandates record-keeping by developers. Grants the Colorado Attorney General exclusive enforcement authority, effective January 1, 2027.
Establishes the Texas Responsible Artificial Intelligence Governance Act to regulate AI systems. Prohibits certain manipulative behaviors and uses of biometric data by AI systems. Requires consumer disclosure when interacting with AI. Creates the Texas Artificial Intelligence Council for oversight and recommendations. Establishes a regulatory sandbox program for innovative AI systems. Enforces compliance through the Attorney General.
Amends laws to require New York government agencies to disclose, assess, and submit reports on their use of AI decision-making tools. Ensures human oversight, prohibits biased outcomes, and protects employee rights regarding AI's impact on employment and duties.
