This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A Kansas federal judge sanctioned five lawyers for submitting court filings that contained non-existent case citations and quotations generated by ChatGPT. The lawyers failed to verify the AI-generated content before signing the documents, leading to a combined $12,000 in fines. The judge highlighted the ethical obligations of attorneys to vet AI-generated research to avoid submitting 'hallucinated' legal authority.
In Lexos Media IP LLC v. Overstock.com Inc., a Kansas federal judge sanctioned five lawyers after finding that filings included AI-generated defective legal authority that had not been verified before submission. The lawyers were fined $12,000 combined.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
Also classified as
The AI system generated false and misleading information in the form of non-existent legal citations and quotations.
The lawyers overrelied on ChatGPT by using it for legal research and submitting its unverified, hallucinated outputs directly to a federal court without checking them for accuracy.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was initiated by the AI system generating hallucinated, non-existent legal citations, which is an unintentional outcome of using the model for legal research.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Five lawyers were sanctioned by a Kansas federal judge for submitting court filings containing non-existent case citations generated by ChatGPT. This incident represents a localized failure of professional ethics and AI verification with negligible national security implications, as it posed no threat to physical infrastructure, intelligence, sovereignty, the economy, or societal stability.
The AI system was used as a writing and research assistant to help draft legal documents, augmenting the lawyer's capabilities but requiring human oversight which was neglected.
Limited Risk: ChatGPT is a general-purpose chatbot and generative AI system, which falls under Risk Level 3 due to transparency obligations for conversational systems and synthetic content.
The lawyer used ChatGPT as a writing and research assistant to generate legal arguments, citations, and text for official court filings.
The report provides clear, unambiguous details regarding the specific AI tool used, the nature of the failure (hallucinations), the parties involved, and the exact financial penalties imposed by the court.
None
Technology
AI Hallucination
The generative AI tool fabricated non-existent legal citations and quotes.
Unreliable AI Research Tools
Generative AI can produce plausible but entirely fictional legal authorities.
Data Inputs
Unverified AI Outputs
AI-generated text was directly input into court filings without verification.
Human Factors
Failure to Verify Citations
Lawyers signed and filed documents without checking citations for accuracy.
Personal Pressures
The primary attorney used ChatGPT under personal pressure, bypassing checks.
Negligent Document Signing
Co-counsel and local counsel signed documents without reviewing the content.
Process and Methods
Inadequate Review Process
The firm lacked a process to cross-reference and verify legal citations.
Management
Absence of Firm AI Policies
The law firm operated without strict policies governing the use of AI tools.
Inadequate Partner Supervision
Management failed to ensure all signing attorneys vetted the submitted filings.
Regulatory Environment
Lack of Standardized AI Rules
Courts rely on retroactive sanctions rather than proactive AI usage standards.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
