This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The CopyCop network, a Russian-linked influence operation, utilized generative AI models including ChatGPT and DALL-E 3 to automate the creation and dissemination of over 19,000 deceptive news articles. By plagiarizing and rewriting content from legitimate media outlets with specific partisan prompts, the network aimed to influence political discourse in the US, France, and the UK. While the operation successfully scaled its content production, researchers and government agencies identified and debunked many of its narratives, noting that the AI-generated content was often of low quality and failed to significantly alter election outcomes.
In early March 2024, a network named CopyCop began publishing modified news stories using AI, altering content to spread partisan biases and disinformation. These articles, initially from legitimate sources, were manipulated by AI models, possibly developed by OpenAI, to disseminate Russian propaganda. Over 19,000 articles were published, targeting divisive political issues and creating false narratives.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The automated generation of over 19,000 articles across multiple fake websites directly contributes to the pollution of the online information ecosystem.
The incident involves state-aligned actors using generative AI to conduct a coordinated, large-scale disinformation campaign targeting democratic elections and public opinion.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The disinformation campaign was intentionally designed and executed by human operators using deployed generative AI systems to influence public opinion.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
The CopyCop campaign represents an evolved information warfare capability where Russian-linked actors leveraged commercial LLMs to automate and scale the production of over 19,000 deceptive articles. While the immediate impact on elections and public opinion was limited due to low content quality and swift detection, it demonstrates the growing national security threat of AI-driven foreign influence operations targeting democratic processes.
The report describes the threat actors using commercial AI tools like ChatGPT and DALL-E 3 to plagiarize, translate, and edit content. The AI systems operated on direct inputs (prompts) to produce specific text and images without exhibiting independent planning or autonomous decision-making.
Limited Risk: The report describes the use of AI to generate text and images for fake news websites. Under the EU AI Act, AI systems that generate or manipulate image, audio, or video content (deepfakes) or generate text are classified as Limited Risk, requiring transparency so users are aware they are interacting with AI-generated content.
The AI models were primarily used to generate and translate articles at scale for dissemination on websites and social media platforms.
The reports from Recorded Future, Clemson University, and the U.S. government provide highly detailed and consistent accounts of the CopyCop network's operations, including specific numbers of articles (19,000), websites (11), and the exact AI models used (ChatGPT, DALL-E 3). The inclusion of leaked prompts directly confirms the role of generative AI.
The campaign could be viewed purely as a traditional geopolitical influence operation, but the central role of LLMs in automating and scaling the content generation makes it a clear AI safety/misuse incident.
Technology
AI Voice Cloning
Adversaries cloned the FBI director's voice to fabricate voter fraud claims.
Generative LLM Plagiarism
LLMs were used to easily plagiarize, translate, and rewrite news content.
Automated Content Scaling
AI tools enabled the rapid, high-volume generation of biased election stories.
Data Inputs
Biased Prompt Engineering
System inputs included prompts instructing LLMs to inject political bias.
Unverified Source Material
Adversaries fed legitimate news into LLMs to modify and weaponize them.
Lack of Input Safeguards
Commercial AI models lacked filters to detect malicious political manipulation.
Human Factors
Prompt Leakage Oversight
Operators left system prompts visible in published AI-generated articles.
Targeting Vulnerable Readers
Operations targeted specific voter demographics to deepen political division.
Process and Methods
Inadequate Content Moderation
Social media platforms struggled to proactively block automated fake accounts.
Automated Article Generation
Propagandists automated the rewriting and posting of thousands of articles.
Management
State-Sponsored Orchestration
Foreign governments funded and directed networks to deploy AI disinformation.
Regulatory Environment
Weak Hosting KYC Controls
Weak KYC allowed adversaries to easily buy web hosting infrastructure.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
