This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Sullivan & Cromwell, a major law firm, submitted an emergency bankruptcy motion containing fabricated legal citations generated by an AI tool. The firm acknowledged that its internal AI-use policies were not followed and that a secondary review process failed to catch the errors, which were subsequently identified by opposing counsel.
Sullivan & Cromwell reportedly filed an emergency motion in the Chapter 15 bankruptcy case of Prince Global Holdings that contained numerous citation and legal inaccuracies. The firm later acknowledged that some errors were AI hallucinations and that its AI-use and citation-review policies had not been followed. After opposing counsel identified the problems, the firm reportedly undertook a formal remedial response.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The lawyers overrelied on the AI tool, failing to follow internal policies or perform a proper secondary review of the generated citations.
The AI system generated discrete false information in the form of fabricated legal citations and non-existent legal sources within a court filing.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was caused by the AI system generating unintended hallucinations and fabricated citations during post-deployment use by the law firm.
A prominent law firm submitted a federal bankruptcy court filing containing fabricated legal citations generated by an AI tool. While the incident briefly introduced false information into a government proceeding, it was quickly identified and corrected through standard judicial processes, resulting in negligible national security impact.
The AI system was used to assist lawyers with legal research and drafting, augmenting their capabilities but requiring human oversight and verification before submission.
Limited Risk: The AI system was used to generate text content (court filing drafts), which falls under the category of AI-generated content requiring transparency, though it was used here as a drafting assistant.
The AI system was used by lawyers to assist in drafting a court filing and conducting legal research, which directly aligns with the functions of a Writing Assistant and Technical Text Generation.
The report clearly outlines the sequence of events: an unnamed AI tool was used for legal drafting, it generated fabricated citations, the firm failed to review it, and opposing counsel caught the error. There is no ambiguity about what occurred, though the specific AI model is not named.
None. The firm explicitly acknowledged the AI hallucinations and apologized.
Technology
AI Hallucinations
The AI tool generated non-existent legal citations and misquoted law.
Human Factors
Failure to Follow AI Policies
Attorneys used AI tools without following the firm's established AI policies.
Inadequate Vetting of Output
Lawyers did not fully vet and verify the AI-generated research results.
Process and Methods
Flawed Secondary Review
The secondary review process failed to catch the inaccurate citations.
Management
Inadequate Oversight of AI Use
Management failed to ensure compliance with existing AI policies.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
