This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A federal appeals court sanctioned an attorney $2,500 for submitting a legal brief containing 21 instances of fabricated case citations and factual misrepresentations generated by AI. The court noted that the attorney failed to verify the AI-generated content and subsequently provided a misleading response when confronted. This incident highlights the ongoing issue of AI-hallucinated material in legal filings, with the court emphasizing that ignorance of AI risks is no longer an acceptable excuse for legal professionals.
In Fletcher v. Experian Information Solutions, Inc., the Fifth Circuit found that attorney Heather Hersh had used generative AI to draft a substantial portion of a reply brief that contained fabricated legal support and factual representations. The court first issued a show-cause order and later imposed a $2,500 sanction, linking the filing errors to unverified AI-generated output.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The attorney overrelied on the AI system by failing to verify the accuracy of the generated legal citations before submitting them to the court.
The AI system generated 21 instances of fabricated case citations and factual misrepresentations, which constitutes false or misleading information that misled the user and was submitted to a court.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was initiated by the AI system generating fabricated citations (hallucinations) during its post-deployment use, which was an unintentional outcome of the generation process.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
A federal appeals court sanctioned an attorney $2,500 for submitting a legal brief containing 21 AI-generated fabricated case citations. This incident represents a domestic professional conduct and AI-reliance issue with negligible implications for national security.
The AI system was used as a writing assistant to draft a legal brief, augmenting the attorney's capabilities but requiring human verification and oversight which was neglected.
Limited Risk: The system used was a generative AI writing assistant or chatbot, which falls under transparency obligations to ensure users are aware of the AI's nature and verify its outputs.
The AI system was used by the attorney to draft a legal brief, which is a writing assistant function involving technical text generation.
The report provides clear, factual details about the court proceedings, the specific attorney sanctioned, the exact amount of the fine ($2,500), and the number of fabricated citations (21). The role of the AI in generating the hallucinations is explicitly stated.
None. The facts of the AI hallucination and subsequent court sanction are clear and uncontested.
Technology
AI Hallucination of Case Citations
Generative AI drafted a brief with 21 fabricated quotations and cases.
Human Factors
Failure to Verify AI Output
The attorney did not check the accuracy of the AI-generated citations.
Misleading the Court
The attorney falsely blamed databases and evaded admitting she used AI.
Process and Methods
Lack of Verification Process
No process was in place to verify the authenticity of legal citations.
Management
Inadequate Ethical Oversight
Lack of firm oversight led to the submission of unverified legal briefs.
Regulatory Environment
Absence of AI-Specific Rules
The court opted against adopting rules regulating generative AI use.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
