This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
LLMs can sometimes generate content that is factually incorrect, misleading, poorly researched, or unintelligible. Risks in this category occur accidentally and not as a result of humans intentionally trying to cause harm, as is the case with disinformation.
Common sources of AI misinformation include noisy training data, sampling strategies that introduce randomness, outdated knowledge bases, and fine-tuning processes that encourage sycophantic behavior. Incorrect and misleading information generated by LLMs can result in a range of actual and anticipated negative outcomes. Individuals exposed to false information may form inaccurate beliefs and perceptions. This undermines their autonomy and ability to make free and informed choices.
Where inaccuracies in LLM predictions influence an individual's decisions and actions, the individual may experience indirect physical, emotional, or material harms especially but not exclusively in high-stakes domains such as mental health, physical health, law, and finance. For example, an LLM that offers misleading information about medical drug use may cause a consumer to harm themselves or others.
Excerpt from the MIT AI Risk Repository full report
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Incident volume relative to governance coverage; each dot is one of 24 subdomains
Entity
Who or what caused the harm
Intent
Whether the harm was intentional or accidental
Timing
Whether the risk is pre- or post-deployment
Multiple instances of AI-based upscaling tools being used to 'enhance' blurry video footage have resulted in the fabrication of details and the misidentification of individuals. In France, this led to the public misgendering of a 14-year-old girl. In Serbia, law enforcement disseminated AI-enhanced images of a suspect that did not match the actual individual, potentially hindering investigations and misleading the public. Experts warn that upscaling tools invent synthetic details rather than recovering original information, making them unsuitable for identification purposes.
Developers: Synthetic media generation technology developers, Synthetic image generation technology developers, AI image enhancement system developers
Deployers: Synthetic media creators, Social media users
During a child custody and divorce appeal in Nebraska, a self-represented litigant submitted a legal brief containing fictitious case citations and inaccurate legal propositions generated by artificial intelligence, leading the Nebraska Supreme Court to strike the brief.
Developers: Large language model developers
Deployers: Pro se litigants, Jarrod Shannon
A law firm submitted a court filing containing fabricated case citations and quotations generated by an unchecked legal AI tool, leading to a federal judge issuing a warning about the risks of AI in legal practice.
Developers: Large language model developers, Harvey, AI-assisted legal drafting system developers
Deployers: Shapiro Arato Bach LLP, Lawyers, Law firms
of the 272 AI experts surveyed named false or misleading information one of the three AI risks they are most concerned about over the next five years (2025–2030).
average probability experts placed on harm from false or misleading information reaching catastrophic levels (>1M deaths, >$100B loss, or civilization-scale damage) within five years, falling to 6.8% with pragmatic mitigations.
With pragmatic mitigations
Average severity falls from severe to substantial.
139 classified mentions from 35 companies.
Browse all 139 mentions →Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
235 shared governance docs
Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
219 shared governance docs
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
206 shared governance docs
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
185 shared governance docs
Requires operators to disclose AI interactions, especially to minors. Prohibits manipulative tactics, sexually explicit content, or misleading mental health claims. Mandates crisis intervention protocols. Subjects violations to penalties and limits enforcement to the attorney general or consumer protection director.
Regulates the use of automated decision-making technology (ADMT) in consequential decisions, requiring deployers to provide disclosures, facilitate consumer rights for correction and human review, and mandates record-keeping by developers. Grants the Colorado Attorney General exclusive enforcement authority, effective January 1, 2027.
Regulates human-like interactive AI services in China, requiring compliance with security, data protection, and ethical standards. Assigns oversight responsibilities to cybersecurity departments. Encourages indigenous technological innovation while imposing strict content controls, especially regarding minors and national security.
