This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An attorney representing a Deutsche Bank subsidiary submitted an appellate brief containing four AI-hallucinated, nonexistent case citations, leading the D.C. Court of Appeals to strike the brief and refer the attorney for disciplinary investigation.
In Douglas v. Deutsche Bank National Trust Co., attorney Loishirl W. Hall acknowledged using Google's generative AI search tool to locate case authority and failing to verify citations before an appellate brief was filed. The D.C. Court of Appeals found four cited cases did not exist, said the filing misdirected the court and burdened judicial resources, struck Deutsche Bank's brief, and referred the matter for disciplinary review.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The attorney overrelied on the AI tool for legal research and failed to verify the citations before submitting them to the court.
The AI system failed to perform reliably by hallucinating nonexistent legal cases.
The AI system generated fabricated legal citations (hallucinations) which were presented as real authority in a court brief, leading to epistemic harm and legal penalties.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was initiated by the AI system generating fabricated legal citations (hallucinations) during post-deployment use, which was an unexpected and unintentional outcome of the legal research query.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
An attorney used a Google AI tool that generated fabricated case citations in a court filing. While it disrupted local judicial proceedings and led to professional discipline, it has negligible national security implications.
The report describes the AI as a search tool used to assist the attorney in locating case authority. It acted as an assistant, providing information that required human verification and oversight before being submitted to the court.
Limited Risk: The Google AI program used is a generative AI tool capable of generating text and citations, which falls under the category of AI-generated content requiring transparency and verification.
The AI system was used as a search and drafting assistant to help the attorney locate legal authorities and prepare an appellate brief.
The court order provides highly detailed, authoritative, and verified factual accounts of the incident, including the specific fake citations, the attorney's admission, and the court's response.
None. The facts of the AI hallucination and subsequent court filing are clear and uncontested.
The specific name of the Google AI tool used would be helpful but is not strictly necessary for classification.
Technology
AI Propensity to Hallucinate
Generative AI tools fabricate nonexistent legal citations as real authority.
AI Susceptibility to Inaccuracies
AI models make factual errors and mischaracterize holdings of real cases.
Human Factors
Blind Reliance on AI Outputs
Attorney failed to independently verify the AI-generated case citations.
Inexperience of Submitting Attorney
The attorney had never drafted or filed an appellate brief prior to this case.
Lack of Understanding of AI Risks
Failure to appreciate the technology's propensity to invent legal authority.
Process and Methods
Inadequate Brief Review Process
Other signing attorneys failed to vet the brief's citations before filing.
Lack of Pre-filing Verification
No process was in place to verify citations obtained via AI search tools.
Management
Ineffective Policy Enforcement
Firm policy banning AI drafting was not actively enforced or monitored.
Deficient Supervisory Oversight
The firm failed to actively supervise the work of a novice associate.
Regulatory Environment
Ambiguous Appellate Sanction Rules
Local rules lack clear provisions for directly sanctioning negligent attorneys.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
