This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The South African government's Draft National AI Policy was found to contain at least six fictitious academic citations, which experts attribute to AI hallucinations. While the Department of Communications and Digital Technologies is reviewing the discrepancies, they have downplayed the impact on the policy's substance. The incident highlights concerns regarding the irresponsible use of AI tools in government research and the lack of human oversight in verifying AI-generated content.
South Africa's Draft National AI Policy, gazetted for public comment, reportedly contained at least six fictitious academic references. Several cited articles or journals reportedly did not exist or were disclaimed by journal editors, and experts said the errors were consistent with AI hallucinations. The Department of Communications and Digital Technologies said it was reviewing the discrepancies but argued they did not affect the draft's substance.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
Government officials relied on the AI tool to generate references for a national policy document without verifying the outputs, demonstrating unsafe use and overreliance.
The AI system generated completely fictitious academic citations that were included in a draft national policy document, spreading false information.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by an AI tool hallucinating fictitious academic citations during the drafting process, which was an unintentional outcome of using the model.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
The South African government's Draft National AI Policy was found to contain fabricated academic citations due to AI hallucinations. This incident represents a minor, self-inflicted blow to government credibility and highlights the risks of overreliance on AI tools in public administration without rigorous human verification, but poses no significant threat to national security.
The AI system acted as an Assistant, used by government officials to help draft or generate references for the policy document. It augmented human writing capabilities but required human verification, which was neglected in this case.
Limited Risk: The system is a general-purpose large language model used to generate text and citations, which falls under AI-generated content requiring transparency and verification to prevent misleading the public.
The AI tool was used to assist in drafting the national policy document and generating its academic reference list.
The report provides clear, verified evidence from multiple academic journal editors confirming that the citations in the draft policy are indeed fabricated. The department's response and the analysis by AI experts strongly support the conclusion of AI hallucination due to lack of human oversight.
It is theoretically possible, though highly unlikely, that the references were fabricated manually by a human writer, but the pattern of generating plausible-sounding but non-existent journals and volumes is highly characteristic of LLM hallucinations.
Technology
AI Model Hallucination
AI models predict plausible-sounding but fictitious academic citations.
Data Inputs
Unverified Training Data
AI model predicts references based on patterns rather than factual databases.
Human Factors
Lack of Human Supervision
Users relied on AI outputs without verifying the existence of cited sources.
Irresponsible AI Use
Drafting policy documents using AI tools without proper care or disclosure.
Process and Methods
Inadequate Citation Review
The draft policy review process failed to verify the academic references.
Management
Lack of Quality Oversight
Management failed to implement rigorous verification steps before publication.
Downplaying of Errors
Management minimized the significance of fabricated scientific references.
Regulatory Environment
Absence of AI Use Standards
No clear regulations governing the use of AI in government policy drafting.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
