This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A 21-year-old woman in Seoul, South Korea, used ChatGPT to research the lethality of mixing benzodiazepines with alcohol. She subsequently used this information to poison three men, resulting in two deaths and one survivor. Police used the chat history as evidence of premeditated intent to kill.
In Seoul, a woman allegedly used ChatGPT to ask whether mixing sleeping pills or benzodiazepines with alcohol could be fatal before poisoning drinks given to three men. Two men later died in separate motel incidents, and a third survived after losing consciousness. Police reportedly cited her chatbot queries and search history as evidence of intent.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
Also classified as
The AI system lacked the capability or robust safety filters to detect and block queries aimed at planning a murder.
The AI system provided information on lethal drug and alcohol combinations, facilitating an illegal act and violence due to a lack of guardrails against assisting in homicidal planning.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The harm was caused by a human user who intentionally used ChatGPT to research and execute a lethal poisoning plan.
Added categories peak at 2: Minor (Psychological Harm, Inferred)
A domestic criminal incident in South Korea where an individual used ChatGPT to research lethal drug-alcohol combinations to commit murder. This incident constitutes a local law enforcement case with negligible national security implications, though it highlights ongoing debates regarding AI guardrails and the retrieval of dual-use or dangerous information.
The AI system functioned as an assistant, answering factual questions and providing information to the user without taking independent action. The report notes that the user asked ChatGPT questions to clarify whether her cocktail would prove fatal, and the chatbot provided factual responses.
Limited Risk: The system involved is ChatGPT, which is a chatbot. The EU AI Act classifies chatbots under Limited Risk, requiring specific transparency obligations to ensure users are aware they are interacting with an AI.
The system is ChatGPT, a general-purpose chatbot designed to interact with users and answer questions across various topics.
The report clearly details the role of ChatGPT in providing factual information to a user planning a crime. The facts regarding the murders, the specific questions asked, and the AI's responses are explicitly stated and confirmed by police and OpenAI.
None
Technology
Lack of Guardrails for Safe Info
Chatbot provided factual answers on lethal drug doses without raising alarms.
No Contextual Intent Detection
The system failed to detect the malicious intent behind factual questions.
Risk of AI Psychosis Exacerbation
Chatbots can worsen symptoms in users experiencing mental health crises.
Data Inputs
Unfiltered Factual Training Data
AI training data included precise details on lethal drug combinations.
Lack of Query Intent Labeling
Input processing lacked labels to flag series of suspicious factual queries.
Human Factors
User Mental Illness
The user had a mental illness, which can be exacerbated by chatbot use.
Premeditated Malicious Intent
The user actively sought information to plan and execute multiple murders.
Process and Methods
Inadequate Guardrail Frameworks
Safety processes only flagged self-harm, ignoring homicidal lines of inquiry.
No Multi-turn Risk Analysis
The safety methods did not analyze the cumulative risk of sequential chats.
Management
Inadequate Risk Assessment
Management failed to assess risks of users leveraging AI for planning crimes.
Prioritizing Utility Over Safety
The company focused on factual accuracy over preventing dangerous outcomes.
Regulatory Environment
Absence of Chatbot Safety Laws
A lack of legal standards allowed chatbots to operate without robust safety.
Lack of Harm Reporting Mandates
No regulatory requirements existed to report user risks or harmful queries.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
