This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
OpenAI identified and banned two clusters of China-linked ChatGPT accounts that were used to conduct covert influence operations targeting U.S. AI policy debates. The accounts generated social media comments, images, and false claims about data centers and U.S. tech competition to amplify domestic divisions. While the operations sought to manipulate public opinion, OpenAI found no evidence of significant breakout beyond the accounts' own activity.
OpenAI reported that two clusters of ChatGPT accounts likely originating from China used its models to support apparent covert influence operations about U.S. AI and technology policy. The accounts reportedly generated a variety of media and social media comments about AI data centers, electricity costs, tariffs, and U.S. tech competition before OpenAI banned them.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The campaigns generated false claims, such as alleging ChatGPT user data had been compromised, to mislead the public.
The incident involved foreign state-linked actors using ChatGPT to generate coordinated, covert influence campaigns aimed at manipulating public opinion and political debates regarding U.S. AI infrastructure.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was caused by human threat actors intentionally using deployed ChatGPT models to generate and spread covert influence campaign content.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
China-linked actors used OpenAI's ChatGPT to conduct covert influence operations targeting U.S. AI policy debates. The operations generated social media content to exploit domestic controversies but were detected and banned by OpenAI, achieving little to no authentic engagement and presenting minor national security impact.
The report describes ChatGPT being used to generate social media comments, images, and comic strips based on specific prompts provided by the operators. This fits the Assistant behavior type as the AI is augmenting human capabilities to generate content under direct human instruction and oversight.
Limited Risk: The incident involves the use of 'ChatGPT' to generate 'social media comments and images', which falls under the category of chatbots and AI-generated content requiring transparency obligations.
The primary function of the AI in this incident was to generate social media comments, images, and comic strips to be posted on platforms like X.
The reports from OpenAI and Alethea provide clear, detailed, and consistent accounts of the threat actors, their tactics, the specific campaigns, and the lack of real-world impact. There is high consensus on the facts of the ChatGPT misuse.
None
Technology
Generative AI Abuse
ChatGPT used to easily generate covert propaganda comments and comic strips.
AI-Narrated Content Farms
Machine voiceovers and automated templates scale up fake news delivery.
Algorithmic Amplification
Social media algorithms push AI slop to targeted local audiences.
Data Inputs
Inauthentic Social Data
Networks of fake profiles seed AI-generated imagery into local groups.
Fabricated Tech Articles
Fake articles hosted on domains imitating real tech news outlets.
Exploited Satellite Imagery
Real satellite images used to anchor false claims about infrastructure.
Human Factors
Pre-existing Public Anxiety
Existing fears about AI, grid stability, and jobs are easily exploited.
Low Digital Literacy
Users fail to distinguish AI-generated propaganda from authentic news.
Hyperlocal Grievances
Local opposition to data centers makes residents receptive to fake claims.
Process and Methods
Delayed Threat Detection
Inability to detect coordinated AI operations before they spread widely.
Dismantled Tracking Teams
Reduction of government teams tracking foreign influence operations.
Replicable Propaganda Aesthetics
Domestic actors easily copy adversarial AI Lego-style video templates.
Management
Reactive Safety Measures
AI developers ban accounts only after influence campaigns have run.
Siloed Threat Intelligence
Security, PR, and government teams fail to share a unified threat picture.
Engagement-Driven Business Model
Social platforms prioritize user engagement over narrative integrity.
Regulatory Environment
Weak AI Governance
Absence of clear international rules governing AI-driven influence.
Geopolitical Friction
Adversarial states use AI to covertly weaken democratic institutions.
Lack of Platform Regulation
No regulatory penalties for hosting coordinated inauthentic AI content.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
