This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A law firm, Boies Schiller Flexner, submitted a legal brief containing fabricated case citations and errors, which the partner responsible attributed to the improper use of AI during the drafting process. The court denied the firm's request to replace the brief and considered monetary sanctions, marking a significant instance of AI-related error in legal practice.
Boies Schiller Flexner filed a respondents' brief in Bixler v. Church of Scientology International that was reportedly prepared with AI assistance and allegedly contained material citation errors. Opposing counsel reportedly identified mischaracterized and mistitled authorities, including one nonexistent case. Partner John Kucera accepted responsibility for failing to verify the citations and sought to replace the brief; the court denied that request and later considered monetary sanctions.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The lawyer overrelied on the AI system to draft the brief and failed to scrupulously proofread or verify the citations before filing.
The AI system inadvertently generated fabricated legal citations and false case holdings, which were then submitted to a court of law as factual.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by an AI system generating hallucinated and fabricated legal citations during the drafting of a brief, which was an unexpected and unintentional outcome of its use post-deployment.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
A US law firm partner submitted a legal brief containing fabricated AI-generated citations in a civil lawsuit. While representing a significant professional error and highlighting risks of overreliance on AI in the legal sector, the incident has negligible national security implications.
The report states that artificial intelligence was used during the preparation of the brief to assist the lawyers, which is characteristic of an Assistant that requires human oversight and proofreading.
Minimal or No Risk: The AI system used was a writing or drafting assistant, which generally poses minimal risk and has no additional regulatory obligations under the Act.
The AI system was used during the preparation and drafting of a legal brief, acting as a writing assistant for the law firm.
The report clearly outlines the incident where a lawyer submitted a brief with fabricated citations generated by AI. The roles of the AI, the lawyer, and the opposing party are well-defined, leaving little ambiguity about the nature of the failure.
None
While the specific AI model name is missing, it is not strictly necessary to classify the incident's domain, causal factors, or harms.
Technology
AI Hallucinations
AI generated a completely made-up case and incorrect citations.
Flawed AI Text Generation
AI mistitled cases and mischaracterized legal holdings.
Human Factors
Lack of Personal Proofreading
The partner failed to scrupulously proofread and cite check the brief.
Overreliance on AI Outputs
Lawyers trusted the AI-generated brief without verifying the citations.
Process and Methods
Inadequate Citation Verification
The process for verifying citations failed to catch the AI-generated errors.
Management
Ineffective Oversight
Responsible partner failed to properly oversee the brief preparation process.
Inadequate Policy Enforcement
Existing AI policies and trainings were not successfully enforced.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
