This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
OpenAI reported the disruption of five covert influence operations originating from Russia, China, Iran, and Israel that utilized its generative AI models. These actors employed the technology to generate social media content, translate articles, debug code, and create fake personas to manipulate public opinion and influence geopolitical narratives. While the operations were active, OpenAI noted that they struggled to gain meaningful audience engagement or reach.
In a report released by OpenAI, the company described how its generative AI tools were misused by state actors and private companies in Russia, China, Iran, and Israel to conduct covert influence campaigns aimed at manipulating public opinion and geopolitical narratives.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The threat actors used AI to generate false or misleading articles, headlines, and social media comments to deceive the public.
The incident involved multiple state-sponsored and commercial actors using generative AI to run coordinated, large-scale covert influence operations targeting political processes and public opinion globally.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was caused by human threat actors who intentionally used deployed AI models to run covert influence operations.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Disruption of five state-sponsored covert influence operations from Russia, China, Iran, and Israel utilizing generative AI to manipulate public opinion and target global elections. While the campaigns achieved low engagement and were successfully disrupted, they demonstrate an evolving information warfare threat where adversaries leverage AI to scale deceptive content generation.
The threat actors used the AI models to generate text, translate articles, and debug code, acting as an assistant to augment human productivity. As OpenAI noted, 'AI can change the toolkit that human operators use, but it does not change the operators themselves.'
Limited Risk: The report describes chatbots and AI-generated content used in influence campaigns. According to the definitions, Risk Level 3 includes 'Chatbots' and 'AI-generated content such as deepfakes' which require transparency obligations.
The AI models were primarily used by the threat actors to generate social media posts, comments, and articles, as well as to translate and edit content across multiple languages.
The reports from both the New York Times and OpenAI provide highly detailed, consistent, and first-hand accounts of the disrupted operations, including their names, origins, methods, and lack of audience engagement.
None
Technology
Abuse of Generative AI Tools
Threat actors exploited LLMs to generate deceptive content at scale.
AI-Assisted Code Debugging
Actors used models to debug bot code and automate distribution.
Model Vulnerability to Exploitation
Models generated persuasive text and translations for foreign campaigns.
Data Inputs
Social Media Query Data
Actors queried models with social media data to optimize campaigns.
Research of Current Events
Threat actors used AI to research current events for propaganda.
Human Factors
Malicious Actor Intent
State-sponsored actors actively sought to manipulate public opinion.
Operator Human Error
Actors accidentally published model refusal messages on social media.
Fictional Persona Creation
Humans used AI to generate fake biographies to mimic real users.
Process and Methods
Multi-Platform Distribution
Lack of coordination between generation tools and distribution platforms.
Mixing AI and Manual Content
Combining AI text with memes made campaigns harder to detect initially.
Deceptive Article Translation
Using AI to translate and edit articles to bypass language barriers.
Management
Difficulties in Distribution Tracking
AI developers struggled to track how generated content was distributed.
Incomplete Threat Sharing
Historic gaps in industry-wide sharing of threat indicators.
Regulatory Environment
Lack of Global AI Governance
Absence of international laws allowed state actors to exploit AI tools.
Sanction Evasion by State Actors
Sanctioned groups like Iranian Guards accessed US-based AI technologies.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
