This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The author of a nonfiction book titled 'The Future of Truth' acknowledged that he used ChatGPT and Claude to assist in writing the book, which led to the inclusion of several AI-generated and misattributed quotations. The errors were identified by The New York Times, prompting the author to take responsibility and initiate a review process for future editions. The incident highlights the risks of using AI tools for research and verification in professional publishing.
Steven Rosenbaum's nonfiction book The Future of Truth reportedly included AI-generated or misattributed quotations after he used ChatGPT and Claude while working on the book. The New York Times reported that more than half a dozen quotations in reviewed sections were synthetic or wrongly attributed, including passages assigned to named writers and scholars. Rosenbaum later acknowledged the errors as accidental and said affected passages would be reviewed for correction.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The author over-relied on ChatGPT and Claude for research and verification without independently fact-checking the generated quotes.
The AI systems generated false and fabricated quotes that were published as factual, leading to the spread of misleading information in a nonfiction book.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The fabrication of quotes was an unintentional outcome of the AI systems' generation process during post-deployment use by the author.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
An author's unintentional inclusion of AI-generated fabricated quotes in a nonfiction book highlights the risks of overreliance on AI tools for research. The incident represents a minor epistemic issue within the publishing industry with negligible implications for national security.
The author used ChatGPT and Claude as writing and research assistants to augment his writing process, which required human oversight and verification that ultimately failed.
Limited Risk: The incident involves chatbots and AI-generated content, which pose a moderate risk and require transparency and verification to prevent the spread of misinformation.
The AI tools were used by the author to assist in the research, writing, and editing process of his book.
The report provides clear, firsthand accounts from the author admitting to the use of AI and the resulting errors, as well as direct confirmations from the misquoted individuals.
None
Technology
AI Hallucination of Quotes
ChatGPT and Claude generated realistic but entirely fabricated quotes.
Inaccurate Source Attribution
AI models mixed real quotes with incorrect source origins.
Data Inputs
Lack of Real-Time Data Access
AI models trained on static data without live verification links.
Pattern-Based Text Generation
Models generated plausible-sounding text instead of factual quotes.
Human Factors
Overreliance on AI Output
Author trusted AI-generated research without independent verification.
Lack of Fact-Checking Diligence
Author failed to manually verify quotes before publication.
Process and Methods
Inadequate Editorial Review
Publisher lacked rigorous verification processes for AI-assisted text.
Unverified AI Research Workflow
Integration of AI in writing lacked structured validation steps.
Management
Failure in Publisher Oversight
Publishers did not implement strict compliance checks for AI usage.
Lack of AI Disclosure Policies
No formal management policies requiring verification of AI-generated inputs.
Regulatory Environment
Absence of AI Publishing Standards
No industry-wide regulations or guidelines for AI-assisted books.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
