This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Multiple instances of legal professionals submitting court filings containing AI-hallucinated citations have occurred in the U.S. and Canada. In one case, attorneys were sanctioned $31,100 for submitting briefs with non-existent legal authorities. In another, Anthropic's legal team acknowledged using an AI-generated citation that contained incorrect titles and authors in a copyright lawsuit.
In a legal filing in Universal Music Group et al. v. Anthropic, lawyers for Anthropic acknowledged that expert witness testimony submitted in the case contained erroneous citations generated by the company's Claude AI system. The filing stated that the inaccuracies, which included incorrect article titles and author names, were not caught during manual review. Anthropic characterized the issue as an honest mistake and apologized in court.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
Lawyers over-relied on AI systems to perform critical legal research without verifying the accuracy of the generated citations.
The AI systems failed to perform reliably, generating plausible-sounding but completely fabricated legal citations.
The AI systems generated fabricated legal citations and case details (hallucinations) which were then submitted to courts as real authorities, misleading legal professionals and judges.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by AI systems generating hallucinated, non-existent citations during active use by legal professionals.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Multiple lawyers in the US and Canada submitted court filings containing AI-hallucinated citations, leading to judicial sanctions. While these incidents highlight risks of professional overreliance on AI and temporary disruptions to court proceedings, they pose negligible threats to national security and are managed through standard judicial processes.
The AI systems acted as assistants, designed to augment human capabilities by generating outlines, citations, and text. The reports note that the tools 'assist lawyers' and are used to 'outline and generate' briefs, but ultimately require human oversight and verification, which the lawyers failed to perform.
Limited Risk: The systems involved are chatbots and generative AI assistants used for text generation. According to the definitions, 'Chatbots' fall under Limited Risk, requiring transparency so users know they are interacting with AI.
The AI systems were used by legal professionals to assist in drafting briefs, outlining arguments, and generating citations.
The reports provide clear, consistent, and detailed accounts of multiple legal cases where AI-generated hallucinations were submitted to courts. The specific AI tools, law firms, judges, and consequences (such as the $31,100 sanction) are explicitly documented across multiple reliable sources.
None. The facts clearly point to AI hallucination combined with a lack of human verification.
Technology
AI Hallucination
Large language models generate plausible-sounding but fabricated information.
Lack of Fact-Checking Features
Chatbots lack built-in real-time validation of generated references.
Data Inputs
Training Data Limitations
Models are trained on text patterns rather than verified databases.
Human Factors
Overreliance on AI Outputs
Lawyers assumed the chatbot outputs were accurate without verification.
Lack of AI Literacy
Users did not understand that LLMs routinely fabricate citations.
Process and Methods
Inadequate Citation Verification
Manual review processes failed to check the validity of generated citations.
Poor Internal Communication
Lawyers shared drafts without disclosing that AI was used to generate them.
Management
Lack of AI Use Policies
Law firms failed to establish strict protocols for verifying AI-generated work.
Regulatory Environment
Unclear Professional Guidelines
Lack of clear, enforced standards on AI use in legal document preparation.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
