This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Deloitte Australia produced a $439,000 report for the Australian Department of Employment and Workplace Relations that contained numerous fabricated academic citations and a falsified legal quote. Following an investigation by academics, Deloitte admitted to using an Azure OpenAI GPT-4o tool chain in the report's preparation. The firm subsequently issued a corrected version of the report and agreed to a partial refund of the contract fee.
Australian academics reportedly identified fabricated references and a misattributed legal quote in a $439,000 Deloitte report for the Department of Employment and Workplace Relations on welfare compliance. Deleted citations allegedly referred to non-existent works, and a Federal Court judgment was misstated. Deloitte later acknowledged limited use of generative AI in preparing the report and issued a corrected version with a partial refund.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
Deloitte employees over-relied on the AI system's outputs, failing to verify the fabricated references before submitting the report to the government.
The AI system generated false and misleading information in the form of non-existent academic citations and a fabricated court quote, which were then included in a government advisory report.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The fabrication of references was an unintentional hallucination produced by the GPT-4o model during its post-deployment use by Deloitte.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Deloitte Australia submitted a government report containing AI-generated fabricated academic citations and a false court quote. While representing a minor compromise to government advisory processes, the incident was quickly resolved through correction and a partial refund, posing negligible threat to national security.
The report describes the AI system being used as a generative AI large language model tool chain to assist in the creation, traceability, and documentation of the report, acting as an assistant that required human review.
Limited Risk: The system used was a general-purpose generative AI large language model (Azure OpenAI GPT-4o) used to generate text content, which falls under transparency obligations for AI-generated content.
The AI system was used by Deloitte to assist in writing and documenting a technical assurance report for a government department.
The reports provide clear, consistent, and detailed accounts of the incident, including the specific AI model used (Azure OpenAI GPT-4o), the nature of the errors (hallucinated citations and court quotes), the financial consequences (partial refund of a $439,000 contract), and statements from both Deloitte and the affected academics.
None. The use of generative AI and the resulting hallucinations are explicitly admitted in the updated report methodology.
Technology
Generative AI Hallucinations
GPT-4o model generated nonexistent academic references and court quotes.
Unreliable AI Tool Chain
The Azure OpenAI tool chain produced plausible-sounding but fake data.
Data Inputs
Unverified Training or Prompt Data
The AI model lacked grounding in verified legal and academic databases.
Human Factors
Inadequate Human Review
Deloitte personnel failed to verify AI outputs before publication.
Over-reliance on Generative AI
Consultants relied on AI to perform core analytical and drafting tasks.
Carelessness by Report Authors
Authors showed unfamiliarity with key legal facts of the Amato case.
Process and Methods
Flawed Verification Processes
No robust process existed to cross-check cited academic and legal sources.
Undisclosed AI Methodology
The use of generative AI was hidden in the initial version of the report.
Flawed Error Correction Method
Deloitte initially replaced fake citations with other AI-generated fakes.
Management
Poor Quality Assurance
Deloitte management approved a substandard, unverified report for delivery.
Initial Lack of Transparency
Management avoided disclosing AI use when errors were first exposed.
Regulatory Environment
Weak Procurement Oversight
Government agencies lacked mechanisms to verify AI use by contractors.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
