This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Following the release of ChatGPT and Codex, cybersecurity researchers and threat actors demonstrated that these models could be used to generate functional malicious code, phishing emails, and malware components. The tools were found to lower the barrier to entry for cybercrime, allowing individuals with little to no coding experience to develop and iterate on malicious software, despite OpenAI's safety guardrails.
OpenAI's ChatGPT was reportedly abused by cyber criminals including ones with no or low levels of coding or development skills to develop malware, ransomware, and other malicious softwares.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to develop cyber weapons (e.g., by coding cheaper, more effective malware), develop new or enhance existing weapons (e.g., Lethal Autonomous Weapons or chemical, biological, radiological, nuclear, and high-yield explosives), or use weapons to cause mass harm.
Also classified as
Threat actors and researchers used the AI to generate highly convincing phishing emails and plan automated romance scams using fake personas.
The incident involves cybercriminals using ChatGPT to develop cyber weapons, specifically coding malware such as infostealers, backdoors, and ransomware concepts.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk is driven by human cybercriminals intentionally exploiting and manipulating the deployed AI system to generate malicious software and phishing materials.
Cybersecurity researchers and threat actors demonstrated that ChatGPT and Codex could be leveraged to generate functional malicious code and phishing emails. While lowering the barrier to entry for low-skilled cybercriminals, the incident represents an evolved capability for cyber threat generation rather than an active national security crisis.
The reports describe ChatGPT as an AI-driven natural language processing tool that assists users with tasks like composing emails, essays, and code. It does not act autonomously to launch attacks, but rather serves as a force multiplier for human actors who must validate and execute the code.
Limited Risk: The reports describe ChatGPT as an 'AI chatbot' and 'natural language processing tool' generating text and code, which falls under the transparency obligations for chatbots and generative AI.
The AI system was primarily used by both researchers and threat actors to generate functional programming code and write persuasive text for emails.
The reports are highly consistent, drawing from a detailed primary source (Check Point Research) with concrete examples of generated code (Python infostealers, Java downloaders, VBA macros). The technical capabilities and limitations are well-documented across multiple reputable tech journalism outlets.
None
Technology
Dual-use nature of computer code
Benign programming functions can easily be modified for malicious purposes.
Imperfect safety guardrails
Filters can be bypassed by rephrasing prompts or using roleplay tactics.
Low barrier to entry
Conversational interface enables non-technical users to generate malware.
Data Inputs
Training on public code repositories
Vast training data includes public code containing vulnerabilities and exploits.
Abuse of malware research text
Users input published research papers to recreate specific malware strains.
Human Factors
Malicious intent of threat actors
Cybercriminals actively abuse the chatbot to develop offensive hacking tools.
Low technical skill of users
Allows script kiddies with zero coding skills to launch cyberattacks.
Human vulnerability to deception
Targets are easily fooled by highly convincing, error-free phishing emails.
Process and Methods
Iterative prompting techniques
Attackers refine basic outputs step-by-step to create complex malware.
Lack of user intent verification
No process exists to verify whether code requests are for benign use.
Management
Rapid public release of tool
Broad public release occurred before establishing foolproof safety controls.
No penalties for policy violations
No immediate consequences are enforced for violating terms of service.
Regulatory Environment
Absence of AI safety regulations
No legal framework forces developers to secure models against cybercrime.
Lack of regulatory enforcement
No oversight body exists to penalize developers for abuse of AI tools.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
