This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An attorney representing State Farm in a California lawsuit submitted court filings containing seven fabricated legal citations generated by the AI tool Irys.
In Fa'alagilagi Meni-Siliga v. A's Contractor, Inc., et al., attorneys representing State Farm acknowledged that AI-assisted motions contained fabricated and otherwise inaccurate legal authorities. Attorney Jacquelene Robinson said she used the legal AI tool Irys and failed to verify some citations. Opposing counsel identified the errors, after which the firm apologized and filed corrected versions.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The attorney relied on the AI tool without verifying the citations, mistakenly believing it performed automatic cite-checking.
The AI system generated fabricated legal citations, quotes, and holdings, which constitutes false and misleading information that was subsequently submitted to a court.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by the AI system Irys generating hallucinated, nonexistent legal citations, which was an unintended outcome of its use.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
An attorney in a civil lawsuit unintentionally submitted AI-generated fabricated legal citations to a California court. The incident represents a professional negligence and AI hallucination issue with negligible national security implications.
The AI tool Irys was used to assist the attorney in preparing motions, acting as a writing and research assistant that required human oversight and verification before filing.
Limited Risk: The system is a generative AI tool used for drafting text, which falls under transparency obligations for chatbots and generative content.
The AI system Irys was used by an attorney to assist in drafting and preparing legal motions in limine.
The reports provide clear, consistent, and detailed accounts of the attorney using the AI tool Irys, the specific fabricated citations generated, and the subsequent court filings and apologies. There is no significant disagreement on the facts of the AI's failure.
None
Technology
AI Hallucinations in Irys
The AI tool Irys generated fabricated case citations and quotes.
Lack of Westlaw Integration
Irys did not connect to Westlaw to perform internal cite checks.
Human Factors
Mistaken Belief in Tool Checks
Attorney incorrectly assumed Irys automatically verified citations.
Neglecting Manual Cite Checks
Attorney failed to manually verify the cited authorities before filing.
Lack of Lead Counsel Awareness
Lead counsel was unaware that AI was used to prepare the filings.
Process and Methods
Inadequate Filing Review
The firm lacked a formal process to verify AI-generated citations.
Lack of Clear AI Policy
The firm had not implemented specific guidelines for generative AI use.
Management
Inadequate Counsel Oversight
State Farm did not monitor external counsel's AI policies and tools.
Lack of Supervisory Control
Lead attorney did not review the second-chair's reliance on AI.
Regulatory Environment
Lack of External AI Audits
No regulatory body or court mechanism pre-screened AI legal filings.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
