This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A user on the Character.AI platform created an unauthorized chatbot impersonating a murder victim, Jennifer Ann Crecente, using her name and yearbook photo. The victim's family discovered the bot via a Google Alert and reported it to the company, which subsequently removed the profile. The incident highlights significant ethical concerns regarding the lack of proactive guardrails on AI platforms that allow users to create digital replicas of real people, particularly deceased individuals, without consent.
A user on the Character.ai platform created an unauthorized AI avatar of Jennifer Ann Crecente, a murder victim from 2006, without her family's consent. The avatar was made publicly available, violating Character.ai's policy against impersonation. After the incident surfaced, Character.ai removed the avatar, acknowledging a policy violation.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The creation of a chatbot based on a murdered teenager is highly offensive and violates community norms, exposing the family and users to inappropriate and abusive content.
The incident involves the unauthorized impersonation of a specific deceased individual to create a digital replica, which constitutes targeted manipulation and harassment of the victim's family.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was initiated by a human user who intentionally created an unauthorized chatbot of a deceased person, exploiting a post-deployment platform that lacked proactive creation guardrails.
Added categories peak at 2: Minor (Psychological Harm, Direct)
An unauthorized chatbot impersonating a deceased murder victim was created on Character.AI. While causing severe distress to the family and highlighting ethical and regulatory gaps in platform moderation, the incident has negligible national security implications.
The AI system functions as a conversational chatbot designed to interact with users, provide information, and adopt specific personas based on user prompts, which fits the definition of an Assistant.
Limited Risk: The system is a chatbot that generates synthetic content and impersonates real individuals, which falls under the category of chatbots and deepfakes requiring transparency under the EU AI Act.
The primary function of Character.AI is to allow users to create and converse with customizable chatbots that simulate specific personalities and styles of speech.
The reports provide consistent, detailed accounts of the incident across multiple reputable sources. The facts regarding the creation of the bot, the family's discovery, the platform's response, and the emotional impact are clear and uncontradicted.
None. The event is clearly an unauthorized impersonation of a deceased individual on a generative AI platform.
Technology
Generative AI Impersonation
LLMs allow easy creation of highly realistic personas of real individuals.
Algorithmic Biography Errors
AI mixed details of the victim with her uncle's gaming journalism background.
Ineffective Proactive Filters
Automated systems failed to block the unauthorized use of a victim's name.
Data Inputs
Unregulated Web Scraping
Victim's yearbook photo and personal history were easily accessible online.
Unauthorized Data Uploads
Users can upload names and photos of any real person without verification.
Lack of Input Consent Verification
No mechanism exists to verify if the creator has consent to use the likeness.
Human Factors
Insensitive User Behavior
A user chose to create a chatbot of a deceased teenage murder victim.
User Trust in AI Hallucinations
Users may believe false biographical details generated by the chatbot.
Process and Methods
Reactive Moderation Process
The platform relies on victim reports to take down offensive chatbots.
Slow Customer Support Response
The company only acted quickly after a viral social media post by the uncle.
Frictionless Bot Creation
The platform allows rapid creation of chatbots with zero safety checks.
Management
Growth Prioritized Over Safety
Management focused on scaling to millions of users without safety guardrails.
Inadequate Risk Assessment
Management failed to anticipate the ethical risks of mimicking deceased people.
Bypassing Ethical Guardrails
Founders launched the tool after Google refused due to safety concerns.
Regulatory Environment
Section 230 Liability Shield
Federal law shields platforms from liability for user-created content.
Lack of Federal AI Privacy Laws
Existing laws do not protect private individuals from AI impersonation.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
