This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A federal report released by the Trump administration's 'Make America Healthy Again' commission was found to contain numerous fictitious studies and erroneous citations. Investigations by media outlets identified 'oaicite' markers in the report's URLs, indicating the use of OpenAI's ChatGPT to generate the bibliography. While the White House subsequently issued a corrected version of the report, the incident highlights the risks of using generative AI for tasks requiring high-stakes verification and scientific rigor.
The federal "Make America Healthy Again" (MAHA) report, released under HHS Secretary Robert F. Kennedy Jr., included hundreds of citations, some of which were reportedly nonexistent or erroneous. Analysts reportedly identified markers consistent with AI-generated text, such as repeated entries, nonexistent studies, and URLs containing "oaicite," suggesting use of tools like ChatGPT.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
Government officials relied on ChatGPT's outputs to construct a high-stakes federal health report without verifying the accuracy of the generated citations.
The incident involved ChatGPT generating fictitious scientific studies and citations that were published in a federal report, spreading false information to the public.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by OpenAI's ChatGPT generating hallucinated, fictitious citations which were then included in a federal health report.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
An official federal health report from a presidential commission was found to contain fictitious scientific citations generated by OpenAI's ChatGPT. While the report was quickly corrected, the incident highlights the national security risks of government overreliance on generative AI without rigorous human verification, temporarily undermining the credibility of state-level policy outputs.
The report describes ChatGPT being used to assist in developing the report and generating citations, which is characteristic of an AI assistant. The text notes that 'the URLs of several references included oaicite, a marker that OpenAI applies to responses provided by artificial intelligence models like ChatGPT, which strongly suggests it was used in developing the report.'
Risk Level 3: Limited Risk. The system used was ChatGPT, a general-purpose chatbot. Under the EU AI Act, chatbots and generative AI systems that generate text are classified as limited risk, requiring transparency obligations so users know they are interacting with AI-generated content.
The AI system (ChatGPT) was used as a writing assistant to help draft, format, or find citations for a federal commission report, resulting in the generation of technical text (bibliographic citations).
The reports provide clear, consistent, and detailed evidence of the AI's role, specifically identifying 'oaicite' markers in the report's URLs and detailing multiple specific instances of fabricated citations. Multiple independent investigations (NOTUS, Washington Post, NYT) confirmed these findings, and the White House subsequently updated the report, validating the claims.
One could argue the errors were purely human formatting mistakes as claimed by the HHS spokesperson, but the technical evidence of 'oaicite' markers strongly refutes this.
Technology
Generative AI Hallucinations
AI model generated fictitious citations and studies that did not exist.
Unreliable Citation Generation
AI model garbled URLs and fabricated co-authors or journal names.
Data Inputs
Unvalidated AI Outputs
AI-generated text was directly imported as source data for the report.
Human Factors
Overreliance on AI Outputs
Authors trusted ChatGPT's citations without checking original sources.
Lack of Professional Verification
Writers failed to verify the existence of cited studies manually.
Process and Methods
Absence of Verification Review
No independent review process was used to verify the bibliography.
Inadequate Drafting Guidelines
No processes existed to regulate or disclose the use of AI in writing.
Management
Inadequate Quality Control
Management failed to ensure rigorous review before releasing the report.
Dismissal of AI Risks
Management downplayed AI-generated errors as minor formatting issues.
Regulatory Environment
Lack of AI Disclosure Standards
No regulatory standards required disclosing AI use in government reports.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
