This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A user reported that their ChatGPT account history was populated with sensitive conversations belonging to unrelated individuals, including login credentials for a pharmacy portal. OpenAI investigated the incident and attributed the exposure to an account takeover originating from Sri Lanka, though the incident highlighted broader concerns regarding the lack of 2FA and login tracking on the platform. The event underscores the risks of data leakage in AI services and the importance of user-side data hygiene.
A security breach involving ChatGPT led to the exposure of sensitive conversations, including login credentials and personal data, after a user account was compromised. OpenAI responded to the incident with an explanation.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
Also classified as
The exposure was facilitated by an account takeover and a lack of standard security features like 2FA on the ChatGPT platform.
The incident involved the exposure of non-public sensitive information, including pharmacy portal login credentials and unpublished research details, to an unrelated user.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was primarily caused by human actions, specifically an intentional account takeover by external actors and a lack of standard security features implemented by OpenAI developers.
Consumer ChatGPT accounts were compromised via credential stuffing originating from Sri Lanka, leading to the exposure of private chat histories, including commercial credentials and unpublished research. The incident represents a minor privacy and commercial security issue rather than a national security threat, manageable through standard cyber hygiene and platform security updates.
ChatGPT acts as an AI assistant that provides information, suggestions, and troubleshooting help to users, requiring human oversight and decision-making.
Limited Risk: ChatGPT is a general-purpose chatbot, which falls under the category of AI systems that pose a moderate risk and require specific transparency obligations.
ChatGPT is primarily deployed as a conversational chatbot and writing assistant to help users with queries, coding, and brainstorming.
The report provides clear details of the incident, including the user's account, the nature of the leaked data, and OpenAI's official explanation of an account takeover. There is minor disagreement between the user and OpenAI regarding whether the account was compromised, but the overall mechanism of exposure is well-explained.
The incident could be interpreted as a direct system bug leaking histories (as initially suspected) rather than an account takeover, though OpenAI's investigation strongly points to the latter.
Technology
Lack of 2FA
No multi-factor authentication allowed unauthorized access to user accounts.
No Session IP Tracking
Users could not monitor active sessions or login locations to detect breaches.
Potential Caching Glitches
Middlebox caching errors could map active sessions to unrelated users.
Data Inputs
Input of Sensitive Credentials
Users inputting sensitive system credentials into the chatbot interface.
Proprietary Data Sharing
Users entering proprietary code and presentation details into queries.
Lack of Input Sanitization
System accepted plaintext credentials and sensitive data without warnings.
Human Factors
Credential Sharing or Reuse
Users reusing passwords across multiple accounts or platforms.
Overreliance on AI Privacy
Users assuming conversations with the AI chatbot are completely private.
Poor Security Awareness
Employees troubleshooting secure portals using public AI tools.
Process and Methods
Weak Account Security Policies
Absence of basic security standards like login alerts or 2FA.
Inadequate Session Management
Lack of robust validation for concurrent or geographically distant sessions.
Inadequate Data Leak Prevention
Lack of client-side checks to prevent pasting sensitive information.
Management
Prioritizing Speed Over Security
Rapid deployment of features without standard security controls like 2FA.
Inadequate User Access Controls
Management failure to mandate industry-standard authentication methods.
Regulatory Environment
Lack of AI Data Standards
Absence of regulatory frameworks governing data privacy in LLM applications.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
