This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Education Scotland published curriculum reform evidence papers that contained approximately 20 fabricated references and significant statistical errors. The agency later confirmed that AI tools were used to edit these documents, though it remains unconfirmed whether the AI directly generated the fabrications. The incident has raised concerns regarding the reliability of government policy documents and the oversight of AI usage in public administration.
Education Scotland, the national education agency, published evidence papers for Scotland's school curriculum review. A University of Edinburgh lecturer identified about 20 references he believed did not exist. The agency corrected errors and disclosed that AI was used to edit the papers. Whether AI introduced any of the disputed references remains unconfirmed.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The AI system failed to perform reliably, leading to factual errors and hallucinations in a critical public administration application.
The AI system inadvertently generated and spread incorrect information, including fabricated academic citations and false statistics, in official government policy papers.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The fabrication of references and statistical errors was an unexpected and unintentional outcome of using an AI system to edit official reports post-deployment.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Education Scotland published curriculum reform papers containing fabricated references and statistical errors due to unverified AI editing. This administrative failure represents a minor impact on government functions, highlighting the need for robust human oversight when integrating AI tools into public policy drafting, but carries negligible national security risk.
The report states that AI was used to edit the reports, acting as an assistant to the human authors who ultimately published the documents.
Limited Risk: The system was used to generate and edit text content, which falls under AI-generated content requiring transparency about its AI origin.
The AI system was used to edit official curriculum reports, functioning as a writing assistant and copy-editing tool.
The report clearly details the nature of the AI's failure (fabricating references and statistics) and the context of its use (editing curriculum papers). While the specific AI model is not named, the impact and the reactions of the involved parties are well-documented.
The fabrications could have been due to human error or plagiarism rather than AI hallucination, though Education Scotland confirmed AI was used.
Technology
AI Hallucination in Text Generation
The AI generated non-existent academic references and fabricated sources.
Inaccurate Statistical Generation
AI produced impossible enrollment numbers contradicting actual exam data.
Lack of Fact-Checking Mechanisms
No automated verification tools were integrated to check generated claims.
Data Inputs
Unverified Training Data
AI models trained on unverified web data generated plausible-sounding lies.
Lack of Grounded Reference Data
The AI system lacked access to real-time Scottish education databases.
Human Factors
Overreliance on AI Outputs
Authors trusted AI-generated text without verifying the cited sources.
Lack of AI Literacy
Staff lacked understanding of AI hallucination risks in policy drafting.
Process and Methods
Inadequate Review of Drafts
Official policy documents were published without rigorous academic vetting.
Delayed Response to Alerts
Officials brushed aside early warnings about errors for seven months.
Management
Lack of Operational Transparency
Management hid AI usage until external researchers flagged errors.
Poor Risk Management
Leadership did not assess risks of using generative AI for evidence papers.
Regulatory Environment
Lack of AI Guidelines in Education
No clear regulatory frameworks governed AI use in public policy documents.
