This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Security researchers at UpGuard discovered that misconfigured llama.cpp servers were publicly exposing user prompts, including interactive roleplay scenarios depicting sexual abuse of children. The vulnerability stemmed from the insecure configuration of the /slots API endpoint in the llama.cpp framework. While the content involved fictional characters, the incident demonstrates how open-source AI tools can be exploited to generate and facilitate child sexual abuse material (CSAM) and exposes significant privacy risks for users of such systems.
A study by UpGuard reports that misconfigured llama.cpp servers publicly exposed user prompts, including hundreds of interactive roleplay scenarios. Some prompts explicitly described fictional sexual abuse of children aged 7–12. While no real children were involved, the findings demonstrate how open-source LLMs can be exploited to generate AI-enabled child sexual abuse material (CSAM).
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
Also classified as
The leaked data revealed that the AI systems were actively generating highly toxic content, specifically interactive roleplay depicting child sexual abuse.
The exposed endpoint leaked highly intimate, non-public user prompts and conversations, violating the privacy expectations of the users.
The incident was primarily caused by a security vulnerability where the /slots API endpoint of the llama.cpp framework was left exposed to the public internet without authentication.
Human
Due to a decision or action made by humans
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The exposure of user prompts was caused by human misconfiguration of the llama.cpp servers, which was an unintentional security oversight during post-deployment operation.
Added categories peak at 2: Minor (Child Sexual Exploitation & Abuse (CSEA), Direct)
The exposure of approximately 400 llama.cpp servers worldwide highlights critical privacy vulnerabilities in open-source AI deployments and the ease of generating toxic content, including text-based child abuse material. While the national security threat is minor, the incident underscores the growing challenges of data protection and content moderation in decentralized AI environments.
The AI models function as conversational chatbots that interactively respond to user prompts to facilitate roleplay scenarios, acting as assistants to the users' creative writing.
Limited Risk: The systems are conversational chatbots, which are classified as Limited Risk (Level 3) under the EU AI Act, requiring transparency so users know they are interacting with an AI.
The AI systems are explicitly described as chatbots used for interactive fantasy and sexual roleplay conversations.
The reports from UpGuard and WIRED provide clear, consistent, and detailed technical and situational evidence regarding the llama.cpp vulnerability and the nature of the leaked prompts.
None
Technology
Exposed /slots Endpoint
The /slots endpoint leaks prompt text and was left enabled.
Lack of Authentication
No default authentication on llama.cpp HTTP server interfaces.
Uncensored Open Models
Open-source models lack safety guardrails against CSAM generation.
Data Inputs
Explicit System Prompts
Users inputted highly detailed, explicit roleplay scenarios involving minors.
Sensitive User Inputs
Users shared highly intimate or explicit text prompts with the server.
Human Factors
Hobbyist Misconfiguration
Hobbyists deployed servers publicly without checking security settings.
Emotional Attachment to AI
Users form emotional bonds, leading to oversharing of intimate data.
Ignorance of Security Warnings
Users ignored explicit documentation warnings against enabling /slots.
Process and Methods
Inadequate Access Controls
Lack of standard deployment checklists led to public API exposure.
Lack of Content Moderation
Self-hosted platforms lack active content moderation and safety filters.
Management
Lack of Corporate Governance
Hobbyist setups operate entirely outside corporate IT security governance.
Inadequate Risk Assessment
Deployers failed to assess the risks of public endpoint exposure.
Regulatory Environment
Absence of AI Regulation
No active regulation exists for text-based CSAM roleplay generators.
Lack of Legal Enforcement
Regulatory bodies lack tools to police self-hosted, decentralized AI.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
