This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An Oklahoma district judge admitted to using OpenAI's ChatGPT for legal research, which resulted in him issuing a court order containing two fictitious case citations.
Stephens County, Oklahoma, Associate District Judge Lawrence Wheeler reportedly used ChatGPT for legal research while preparing a family-law order that cited at least two nonexistent cases. The order denied a request for a psychological evaluation and reprimanded the requesting attorney; Wheeler later vacated it after the attorney challenged it before the Oklahoma Supreme Court.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The AI system failed to perform reliably by fabricating non-existent legal cases when queried for legal research.
The AI system generated fictitious legal citations (hallucinations) that were presented as factual in an official court order, leading to inaccurate beliefs and legal errors.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by ChatGPT generating hallucinated, non-existent legal citations which were then unintentionally included in a court order.
An Oklahoma district judge used ChatGPT for legal research, leading to the inclusion of two fictitious case citations in an official court order. While this compromised a local judicial process, it was quickly identified and corrected through standard legal appeals, resulting in minor, localized impacts with negligible national security implications.
The judge used ChatGPT as a research assistant to find legal precedents, and he wrote the order himself based on the information provided by the AI, which fits the definition of augmenting human capabilities with human oversight.
Limited Risk: The system used was ChatGPT, which is a general-purpose chatbot. Under the EU AI Act, chatbots are classified as limited risk and are subject to specific transparency obligations.
The judge used ChatGPT to assist in researching and drafting a court order, relying on it to answer questions about legal precedents and generate text.
The reports provide consistent, detailed, and clear factual accounts of the incident, including the specific judge, the court, the case type, the specific fabricated citations, and the subsequent investigations by the OSBI and the Attorney General's office.
None
Technology
AI Hallucination
ChatGPT generated plausible but fictitious case citations during legal research.
Plausibility Bias
The AI is programmed to generate answers that look correct and expected.
Human Factors
Blind Trust in AI
The judge accepted the generated legal citations without verification.
Lack of AI Literacy
The judge was unaware of the risks of generative AI hallucinations.
Process and Methods
Lack of Verification
No standard process was used to verify the citations before issuing the order.
Improper Use of AI
Using a general-purpose chatbot for authoritative legal research.
Management
Inadequate Oversight
Lack of internal checks in the judge's chambers to catch erroneous rulings.
Regulatory Environment
Evolving Ethical Standards
Rules and warnings on AI use in courts were only recently being established.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
