This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
OpenAI identified multiple malicious campaigns leveraging its AI tools, including a Chinese state-linked surveillance operation ('Peer Review') that used ChatGPT to debug code for monitoring anti-state speech. Other campaigns involved disinformation targeting dissidents and US politics, as well as financial fraud schemes like 'pig butchering' and task scams originating from Cambodia and North Korea.
OpenAI reportedly uncovered evidence of a Chinese state-linked AI-powered surveillance campaign, dubbed "Peer Review," designed to monitor and report anti-state speech on Western social media in real time. The system, believed to be built on Meta’s open-source Llama model, was detected when a developer allegedly used OpenAI’s technology to debug its code. OpenAI also reportedly identified disinformation efforts targeting Chinese dissidents and spreading propaganda in Latin America.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The reports also detail Cambodia-based romance and task scams, as well as North Korean employment fraud, which target specific individuals for financial gain.
The incident involves coordinated, state-linked campaigns using AI to develop surveillance tools for monitoring public protests and to generate large-scale political disinformation.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human threat actors who intentionally leveraged deployed AI models to facilitate surveillance, disinformation, and financial scams.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
State-linked and criminal actors leveraged AI tools to enhance disinformation, develop social media surveillance capabilities, and conduct financial scams. While the AI acted primarily as a human-controlled assistant, the integration of LLMs into state-sponsored intelligence and influence operations represents a substantial, evolving threat to information security and democratic stability.
The reports describe threat actors using ChatGPT to debug and modify source code, translate documents, and generate social media posts and articles. The AI acted as an assistant to augment the actors' capabilities, requiring manual prompting and human oversight rather than operating autonomously.
Limited Risk: The report describes the misuse of ChatGPT, which is a chatbot, and the generation of AI-generated content for disinformation and scams, both of which fall under the Limited Risk category requiring transparency obligations.
The AI system (ChatGPT) was primarily used by the threat actors as a writing assistant to draft articles, social media posts, and translation, as well as to debug and edit code for their surveillance tools.
The reports from multiple reputable sources consistently describe the same set of campaigns, their origins, and how OpenAI detected and mitigated them. The details regarding the 'Peer Review' and 'Sponsored Discontent' campaigns are clear and corroborated across all sources.
The activities could be viewed purely as traditional cyber espionage and influence operations that incidentally used AI, rather than failures of AI safety itself.
Technology
Open-Source Model Accessibility
Llama models are open-source, allowing local deployment without oversight.
Code Debugging Capabilities
ChatGPT's coding abilities allowed actors to debug surveillance spyware.
Language Translation Power
LLMs easily translated disinformation articles into Spanish to spread influence.
Data Inputs
Social Media Data Ingestion
Surveillance tool ingested real-time posts from X, Facebook, and Instagram.
Document Screenshots Analysis
ChatGPT analyzed screenshots of English documents showing protest announcements.
Public Information Scraping
ChatGPT researched public data on Western politicians and think tanks.
Human Factors
Manual Prompting Techniques
Actors used manual prompting to mimic normal users and evade detection.
Malicious Intent of State Actors
Chinese operators actively sought to monitor dissidents and anti-China protests.
Deceptive Social Engineering
Fraudsters used AI to craft highly convincing romance scams to trick victims.
Process and Methods
Model Misuse for Spyware
Threat actors bypassed standard use cases to debug surveillance tool code.
Local Deployment of Models
Running open-source models locally prevents external monitoring of activities.
Fragmented AI Tool Usage
Using multiple models (Llama, DeepSeek, ChatGPT) obscured the threat signal.
Management
Policy Enforcement Gaps
OpenAI's policy bans surveillance, but enforcement relied on post-hoc detection.
Regulatory Environment
Lack of Open-Source Controls
No regulations restrict the distribution or local hosting of powerful LLMs.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
