This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Three attorneys from the law firm Butler Snow were disqualified from a federal case in Alabama after they submitted court filings containing five fabricated legal citations generated by ChatGPT. The presiding judge, Anna Manasco, reprimanded the lawyers and referred them to the state bar, noting that the conduct was tantamount to bad faith. While the firm itself was not sanctioned due to its internal policies and subsequent remedial actions, the incident highlights the ongoing issue of legal professionals failing to verify AI-generated research.
Three Butler Snow attorneys defending a former Alabama corrections commissioner were reportedly disqualified after two motions cited five non-existent cases generated via ChatGPT. Judge Anna Manasco is reported to have publicly reprimanded the lawyers, referred them to the Alabama State Bar, and ordered broad disclosure; she declined to sanction the firm. An external review of 2,400 citations across 330 filings reported no further errors.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The attorneys over-relied on ChatGPT's output, failing to verify the citations before submitting them to the court.
The AI system generated fabricated legal citations, which constitutes discrete false and misleading information that was subsequently submitted to a federal court.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was initiated by ChatGPT generating fabricated citations (hallucinations), which was an unexpected outcome of the user's query.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Attorneys at a US law firm submitted court filings containing fabricated citations generated by ChatGPT. The incident is a matter of professional misconduct and judicial administration with negligible national security implications.
The AI system, ChatGPT, acted as an Assistant by generating suggested case citations to support the lawyer's legal arguments, which required human verification and oversight before being submitted to the court.
Risk Level 3: Limited Risk. The system involved is ChatGPT, a general-purpose generative AI chatbot. According to the definitions, chatbots and AI-generated content pose a moderate risk and are classified as Limited Risk.
The AI system was used by the attorney to assist in drafting legal motions by generating supporting case citations.
The reports provide clear, consistent, and detailed accounts of the incident, including the specific law firm, the lawyers involved, the judge's rulings, the nature of the AI-generated fabrications, and the resulting sanctions. There is no conflicting information regarding the core facts of the AI's role or the outcomes.
None
Technology
AI Hallucinations
ChatGPT fabricated non-existent legal citations and case details.
Human Factors
Failure to Verify
Reeves failed to verify ChatGPT citations via Westlaw or Pacer due to haste.
Policy Non-Compliance
Reeves knew the AI policy limitations but chose not to comply with them.
Blind Trust by Co-Signers
Cranford and Lunsford signed filings without checking the citations.
Process and Methods
Inadequate Review Process
The firm's filing process allowed unverified AI citations to be submitted.
Management
Lack of Pre-Filing Controls
Management lacked active technical controls to block unauthorized AI tools.
Regulatory Environment
Insufficient Deterrents
Prior modest court sanctions failed to deter systemic misuse of generative AI.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
