This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An author discovered an AI-generated copycat of his technical book on Amazon, featuring a fake author and likely written using ChatGPT. This incident highlights a broader trend of publishers using generative AI to flood platforms with low-quality, automated content to game search algorithms and product rankings. While some companies defend this as a productivity tool, experts warn of the risks to information integrity and the potential chilling effect on human authorship.
The author Chris Cowell had spent more than a year writing his book "Automating DevOps with GitLab CI/CD Pipelines" when, three weeks before its release, another book appeared bearing the exact title by an author (Marie Karpos) for whom no information could be found. The book appeared to have been written by ChatGPT. While the original Washington Post story does not say so, it is possible the name and description were taken from the Amazon preorder page.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems capable of creating economic or cultural value, including through reproduction of human innovation or creativity (e.g., art, music, writing, coding, invention), destabilizing economic and social systems that rely on human effort. The ubiquity of AI-generated content may lead to reduced appreciation for human skills, disruption of creative and knowledge-based industries, and homogenization of cultural experiences.
Also classified as
AI-generated content is flooding search engines, product reviews, and news sites, polluting the information ecosystem with unvetted and fabricated material.
Publishers used fake author profiles and copycat book titles to siphoning sales and gain an illegitimate financial benefit.
The incident highlights how generative AI is used to copy and devalue human creative and technical writing, leading to a chilling effect on human authors who face automated competition.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk is caused by human publishers intentionally deploying generative AI tools post-deployment to rapidly produce copycat books and low-quality web content for financial gain.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
The incident involves the commercial use of generative AI to rapidly produce copycat books and fake news websites. While it highlights growing challenges in intellectual property protection and digital information integrity, its national security impact remains minor, with no direct threats to critical infrastructure, sovereignty, or national security capabilities.
The report describes AI systems like ChatGPT and GPT models being used as writing assistants to generate text, articles, and books based on human prompts, with some publishers keeping a 'human in the loop' to edit and verify the outputs.
Limited Risk: The report describes the use of chatbots (ChatGPT) and generative AI to produce text and books, which falls under the category of AI-generated content requiring transparency obligations.
The primary goal of the AI models was to act as a writing assistant to generate books and articles, specifically generating technical text for DevOps pipelines.
The report provides clear, detailed, and verified accounts of the copycat book incident on Amazon, backed by interviews with the affected author, industry experts, and references to investigations by NewsGuard.
The incident could be viewed primarily as a copyright/IP infringement issue rather than an AI safety issue, but the central role of generative AI in enabling this scale of copycat publishing makes it an AI safety risk.
Technology
AI Propensity to Hallucinate
Models generate false facts, like claiming a retired pitcher is a partner.
Lack of Grounding in Reality
Generative tools create unvetted prose that lacks factual grounding.
Plagiarism and Copycatting
AI generates books mirroring specific titles and content of human authors.
Data Inputs
Unvetted Online Training Data
Models train on vast, unvetted data pools, losing track of truth.
Inauthentic Platform Reviews
Fake reviews are used to boost visibility of low-quality AI books.
Human Factors
Overreliance on Automated Outputs
Publishers post AI text directly without reviewing or editing it.
Economic Incentives for Cheap Content
Creators prioritize low cost over content quality and accuracy.
Chilling Effect on Human Authors
Human writers are discouraged by cheap, automated competition.
Process and Methods
Absence of Human-in-the-Loop Review
Direct publishing of AI outputs without quality control checks.
Flawed Platform Filtering
Platforms fail to detect copycat titles and fake reviews effectively.
Poor Disclosure Practices
Sites hide AI use or make disclosures obscure to readers.
Management
Prioritizing Volume Over Quality
Companies focus on high-volume clickbait to maximize ad revenue.
Inadequate Risk Assessment
Publishers adopt AI without assessing reputational risks or errors.
Workforce Restructuring
Media companies lay off staff while shifting to AI workflows.
Regulatory Environment
Lack of AI Liability Frameworks
Uncertainty around AI liability and intellectual property rules.
Permissive Platform Search Rules
Search engines allow AI content if it does not gamify rankings.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
