This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A 28-year-old nursing student developed an intense emotional and romantic attachment to a customized ChatGPT instance she named 'Leo.' Despite OpenAI's safety guidelines prohibiting sexually explicit content, the user successfully circumvented these rules to engage in erotic roleplay. The relationship resulted in significant financial expenditure, excessive time usage, and recurring emotional distress when the model's context window reset, forcing the user to retrain the persona.
A New York Times report described "Ayrin" (pseudonym), a 28-year-old nursing student living abroad, who customized OpenAI's ChatGPT to act as an "AI boyfriend" for advice and erotic roleplay. She reported spending 20+ hours/week (up to 56 hours in one week), paying $200/month, and feeling intense grief when the chatbot's context window reset and required retraining.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
The user developed an intense emotional and romantic attachment to a general-purpose chatbot, leading to obsession, financial strain, and severe grief during system resets, illustrating a clear case of overreliance on a system not designed for human companionship.
Human
Due to a decision or action made by humans
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk of emotional overreliance and financial expenditure was driven by the user's active customization of the chatbot and intentional circumvention of safety filters.
Added categories peak at 2: Minor (Psychological Harm, Direct)
An individual developed an intense emotional attachment to a customized chatbot by bypassing safety filters. While highlighting psychological risks of human-computer interaction, the incident has negligible national security implications and poses no threat to critical infrastructure, sovereignty, or national stability.
The AI system operates as an Assistant, responding to user prompts to provide conversation, emotional support, and advice, without taking independent actions or decisions outside of the chat interface.
Limited Risk: The system is a chatbot, which falls under Risk Level 3 of the EU AI Act, requiring transparency to ensure users are aware they are interacting with an AI.
The AI system was customized and used primarily as a conversational chatbot to provide companionship and roleplay.
The reports provide consistent, detailed first-hand accounts of the user's interactions, emotional state, and financial spending, which are corroborated across multiple journalistic sources.
None
Technology
Sycophantic AI Training
Human-reinforced training makes AI overly agreeable, fostering deep attachment.
Context Window Limitations
Limited memory causes AI 'amnesia', leading to cyclical user grief.
Highly Realistic Persona Tools
Humanlike voices and custom settings make the AI feel deceptively human.
Data Inputs
User Personalization Prompts
Users input explicit instructions to force the AI into romantic/sexual roles.
Ingestion of Erotic Training Data
AI learned romantic and sexual patterns from vast internet training data.
Human Factors
User Isolation and Loneliness
Physical separation from spouse and friends drove the user to seek AI comfort.
Deliberate Safety Bypass
User actively bypassed content filters using prompts from online communities.
Susceptibility to Endless Empathy
User became addicted to the AI's constant compassion and validation.
Process and Methods
Bypassable Safety Warnings
Orange warnings for explicit content were easily ignored by the user.
Crowdsourced Jailbreak Methods
Online communities shared instructions on how to bypass AI safety filters.
Unpredictable Text Generation
Hard to block all explicit scenarios due to unpredictable language generation.
Management
Engagement-Driven Revenue Model
High-priced subscriptions incentivize keeping users emotionally hooked.
Inadequate Psychological Safeguards
Management failed to deploy robust systems to prevent severe user dependency.
Regulatory Environment
Inadequate Safety Regulations
Absence of legal frameworks to protect users from harmful AI attachment.
