This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Researchers have identified that AI models, including GPT-3, GPT-4o, and Claude 3.5, have ingested and now reproduce a nonsensical phrase, 'vegetative electron microscopy,' originating from digitized errors in scientific literature. This 'digital fossil' has proliferated into academic papers, complicating peer review and polluting scientific knowledge bases. The incident highlights the risks of training AI on massive, uncurated datasets like CommonCrawl and the difficulty of removing embedded errors from commercial models.
Researchers reportedly traced the appearance of the nonsensical phrase "vegetative electron microscopy" in scientific papers to contamination in AI training data. Testing indicated that large language models such as GPT-3, GPT-4, and Claude 3.5 may reproduce the term. The error allegedly originated from a digitization mistake that merged unrelated words during scanning, and a later translation error between Farsi and English.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The models lack the robustness to identify and filter out obvious nonsense, reliably reproducing errors ingested from their training data.
The AI systems inadvertently generate and spread a nonsensical, incorrect scientific term, leading to its proliferation in academic literature and undermining scientific consensus.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The propagation of the nonsense term is driven by the post-deployment outputs of LLMs that unintentionally memorized training data errors.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
AI models ingested and reproduced a historical translation error, 'vegetative electron microscopy', polluting academic literature. While it highlights vulnerabilities in AI training datasets and scientific research integrity, it poses negligible direct threats to national security.
The AI models (GPT-3, GPT-4o, Claude 3.5) function as writing assistants that generate text and complete sequences based on user prompts, augmenting human writing but requiring human oversight.
Limited Risk: The systems involved are general-purpose chatbots and generative AI models (GPT-3, GPT-4o, Claude 3.5) which fall under the category of AI systems that require transparency obligations so users know they are interacting with AI-generated content.
The primary function of the LLMs involved (GPT-3, GPT-4o, Claude 3.5) is to act as writing assistants and generate technical text based on user prompts.
The reports provide detailed, empirical, and consistent accounts of how the nonsense term originated, how it was ingested by specific AI models, and how it has propagated into academic literature. The findings are backed by scientific sleuths and automated tools like the Problematic Paper Screener.
The incident could be viewed purely as a human translation and academic fraud issue (paper mills), but the empirical evidence showing LLMs consistently generating the term supports the AI-centric classification.
Technology
LLM Error Reinforcement
Models like GPT-4o permanently embed and reproduce the nonsense term.
Faulty OCR Digitization
Digitization process merged text across columns to create a phantom term.
Data Inputs
Contaminated Training Data
CommonCrawl dataset contains scraped pages containing the error.
Farsi Translation Error
Translation error in Iranian papers introduced the term to the web.
Human Factors
Paper Mill Fraud
Fraudulent networks use AI to quickly generate and publish papers.
Uncritical AI Reliance
Authors submit AI-generated text without verifying technical terms.
Process and Methods
Inadequate Peer Review
Reviewers and editors let obvious nonsense terms pass into journals.
Ineffective Screening Tools
Automated tools only detect known errors, missing new variations.
Management
Publisher Inaction
Publishers responded inconsistently, sometimes defending the errors.
Proprietary Data Secrecy
Tech companies hide training datasets, making errors hard to correct.
Regulatory Environment
Lack of AI Training Standards
Absence of regulations for auditing and cleaning massive training data.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
