This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Anthropic reported several instances of malicious actors misusing their LLM in March 2025. These included an influence-as-a-service operation orchestrating over 100 social media bots, a credential stuffing campaign targeting IoT security cameras, recruitment fraud in Eastern Europe, and a novice actor developing sophisticated malware. Anthropic identified and banned the accounts involved in these activities.
In April 2025, Anthropic published a report detailing several misuse cases involving its Claude LLM, all detected in March. These included an "influence-as-a-service" operation that orchestrated over 100 social media bots; an effort to scrape and test leaked credentials for security camera access; a recruitment fraud campaign targeting Eastern Europe; and a novice actor developing sophisticated malware. Anthropic banned the accounts involved but could not confirm downstream deployment.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The AI was used by threat actors to develop advanced malware and build brute-force tools targeting IoT security cameras.
The primary and most novel misuse described is a coordinated influence-as-a-service campaign using the AI to orchestrate over 100 social media bots to manipulate public opinion and political narratives at scale.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human threat actors intentionally misusing the deployed AI model for malicious purposes post-deployment.
Added categories peak at 3: Substantial (Epistemic Harm, Indirect)
Anthropic disrupted several malicious campaigns misusing its Claude LLM, most notably an influence-as-a-service operation where the AI autonomously orchestrated over 100 social media bots to influence political narratives. Other misuses included credential stuffing, Eastern European recruitment fraud, and a novice actor developing advanced malware. While the direct real-world impacts were mitigated, these incidents demonstrate how frontier AI models are evolving to lower technical barriers for complex cyber and information operations.
The report describes the AI system being used as an orchestrator to make tactical decisions on whether social media bots should like, share, comment on, or ignore posts based on politically motivated personas. This semi-autonomous orchestration demonstrates goal-directed decision-making to achieve specific objectives.
Unacceptable Risk: The report describes the AI being used for an 'influence-as-a-service' operation to orchestrate social media bots and make tactical decisions to manipulate public opinion, which constitutes 'AI used for subliminal manipulation or behavioral influence leading to harm'.
The AI was primarily used to generate politically-aligned social media posts and orchestrate bot interactions across platforms like Facebook and X, while also functioning as a general-purpose chatbot and writing assistant for other malicious tasks.
The reports are based on direct threat intelligence findings from Anthropic, detailing specific case studies of misuse of their model. While some technical details like exact prompts or indicators of compromise are omitted for security, the descriptions of the activities, their scale, and the mitigation actions taken are clear and authoritative.
None. The activities are clearly classified as malicious misuses of a deployed LLM.
Technology
Dual-Use Model Capabilities
Model's advanced coding and reasoning can be repurposed for malicious acts.
Agentic Orchestration Capabilities
Model can make tactical decisions to orchestrate networks of social bots.
Natural Language Generation
Generates highly convincing, multilingual text that mimics human humor.
Data Inputs
Malicious Prompting
Adversaries input instructions to bypass safety and generate harmful outputs.
Structured JSON Data Formats
Structured inputs and outputs allowed efficient scaling of persona management.
Integration of Stealer Logs
Processing raw Telegram logs and breach data to target IoT devices.
Human Factors
Novice Actor Upskilling
Low-skilled actors use AI to flatten the learning curve for malware.
Convincing Social Engineering
Actors leverage polished language to dupe unsuspecting job seekers.
Adversarial Intent
Malicious actors actively seek to exploit AI models for financial gain.
Process and Methods
Lack of Real-Time Prompt Detection
Standard safety classifiers failed to block all adversarial prompt patterns.
Absence of Shared Indicators
Lack of shared IOCs, IP addresses, or prompts hinders industry defense.
Inadequate Post-Deployment Auditing
Abuse was detected after operations had already scaled and engaged users.
Management
Over-Reliance on Post-Hoc Bans
Reactive account banning occurred after the malicious activity took place.
Insufficient Threat Intel Sharing
Anthropic's public report lacked actionable intelligence like prompts or IOCs.
Inadequate Pre-Release Testing
Model safety guardrails did not fully anticipate agentic orchestration risks.
Regulatory Environment
Weak Social Media Bot Controls
Social media platforms failed to prevent automated bot networks.
Lack of AI Safety Standards
Absence of standardized frameworks for evaluating AI-driven influence ops.
Cross-Border Cyber Regulation Gaps
Difficult to regulate commercial influence services operating across borders.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
