This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Many AI models, especially those based on deep learning, involve complex mathematical structures that can be difficult to interpret, even for experts. AI systems are also often trained on vast datasets that they use to learn patterns and make predictions. The complexity and volume of this data mean that the learning process – how data points influence the AI's development and final decisions – can be opaque. Furthermore, in many cases, the algorithms, data, and specific methodologies used in developing AI are considered proprietary, and companies may be reluctant to share them openly. Because of these factors, obtaining understandable information about the decision-making process for AI can be challenging.
For users, an inability to interrogate how an output was obtained may lead to a lack of trust and confidence in the system's results and to resistance to adopting the technology. Users may also misinterpret or struggle to find and amend errors in the model's results.
For regulators, AI opacity can frustrate auditing or other compliance standards. Where an AI system's compliance cannot be assessed, a "responsibility gap" may be created, and it may become difficult or impossible to hold systems or relevant actors accountable for their actions. In certain sectors such as healthcare and the military, decisions made by AI systems can have profound consequences, making transparency and accountability particularly pressing issues.
Excerpt from the MIT AI Risk Repository full report
Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
Incident volume relative to governance coverage; each dot is one of 24 subdomains
Entity
Who or what caused the harm
Intent
Whether the harm was intentional or accidental
Timing
Whether the risk is pre- or post-deployment
Anthropic embedded hidden steganographic tracking codes in its Claude Code developer tool to detect unauthorized API resellers and model distillation, sparking privacy concerns and leading Alibaba to ban the tool.
Developers: Anthropic, AI software developers
Deployers: Anthropic, AI service providers
A proposed class action lawsuit has been filed against Eightfold AI, alleging that its AI-driven hiring platform screens and scores job applicants without providing the legally required notice or dispute rights. Plaintiffs claim the system compiles personal data and personality profiles to rank candidates, which they argue violates the Fair Credit Reporting Act and California consumer protection laws. The lawsuit highlights concerns regarding the lack of transparency and accountability in AI-based employment screening tools used by major corporations.
Developers: Eightfold AI, Algorithmic employment screening system developers
Deployers: PayPal, Microsoft, Employers using algorithmic employment screening systems, Eightfold AI
Three make-up artists at MAC (an Estée Lauder subsidiary) lost their jobs after an automated video interview process conducted by HireVue. The employees were unable to receive a clear explanation for the algorithm's assessment of their performance. The case resulted in an out-of-court settlement, and HireVue subsequently discontinued the visual analysis component of its software.
Developers: HireVue
Deployers: Estée Lauder
of the 272 AI experts surveyed named lack of transparency or interpretability one of the three AI risks they are most concerned about over the next five years (2025–2030).
average probability experts placed on harm from lack of transparency or interpretability reaching catastrophic levels (>1M deaths, >$100B loss, or civilization-scale damage) within five years, falling to 4.8% with pragmatic mitigations.
With pragmatic mitigations
Average severity stays substantial.
86 classified mentions from 35 companies.
Browse all 86 mentions →AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
334 shared governance docs
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
305 shared governance docs
Inadequate regulatory frameworks and oversight mechanisms that fail to keep pace with AI development, leading to ineffective governance and the inability to manage AI risks appropriately.
298 shared governance docs
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
236 shared governance docs
Requires operators to disclose AI interactions, especially to minors. Prohibits manipulative tactics, sexually explicit content, or misleading mental health claims. Mandates crisis intervention protocols. Subjects violations to penalties and limits enforcement to the attorney general or consumer protection director.
Requires operators of conversational AI services to disclose AI status to all users, apply added safeguards for minors, such as preventing sexually explicit content and simulated emotional dependence, and implement suicide and self-harm crisis-referral protocols. Prohibits presenting outputs as provided by or equivalent to licensed professionals, and requires annual reporting to the Attorney General on crisis referrals and related protocols.
Requires large frontier AI developers in Illinois to establish and publish a frontier AI framework addressing catastrophic risk assessment by 2028, undergo annual independent third-party audits, file an annual disclosure/registration statement, and report critical safety incidents within 72 hours; prohibits false statements about AI risk or framework compliance; establishes escalating civil penalties up to $3,000,000 per violation; and protects whistleblowers who report safety violations.
