This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
In August 2024, a Norwegian citizen named Arve Hjalmar Holmen filed a GDPR complaint against OpenAI after ChatGPT generated a false and defamatory biography claiming he had murdered his two sons and served a 21-year prison sentence. The AI's output contained accurate personal details, such as his hometown and the number of children he has, which made the fabricated criminal allegations appear plausible. The incident highlights ongoing issues with AI 'hallucinations' and the adequacy of disclaimers in preventing the spread of harmful, inaccurate personal data.
In August 2024, ChatGPT is reported to have falsely claimed that Norwegian citizen Arve Hjalmar Holmen had killed his two sons and been sentenced to 21 years in prison. The fabricated response allegedly included specific details about the supposed crime, despite Holmen never being accused or convicted of any offense. The incident prompted a GDPR complaint for defamation and inaccurate personal data.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The incident is a classic example of AI hallucination, representing a fundamental lack of capability and robustness in the model's factual retrieval.
The AI system generated completely false and defamatory information claiming the user had murdered his children and served prison time, which is a clear instance of generating false and misleading information.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The false and defamatory biography was generated by the ChatGPT AI system after its deployment, representing an unintentional hallucination error by the model.
Added categories peak at 2: Minor (Psychological Harm, Direct)
A Norwegian citizen filed a GDPR complaint against OpenAI after ChatGPT generated a highly defamatory, false biography accusing him of murder. While the incident highlights significant issues with AI hallucinations and data privacy, its national security impact is negligible, representing a localized regulatory and civil rights concern rather than a threat to state security.
The AI system (ChatGPT) acted as a conversational assistant, responding to a user's prompt ('Who is Arve Hjalmar Holmen?') to provide information, requiring human query input and oversight.
Risk Level 3. Limited Risk: The system is a chatbot (ChatGPT), which falls under the category of AI systems that pose a moderate risk and require specific transparency obligations to ensure users are informed they are interacting with an AI.
The system is a general-purpose conversational chatbot designed to answer user queries and generate text based on prompts.
The reports provide consistent, detailed accounts of the specific prompt used, the exact false output generated by ChatGPT, the user's reaction, and the legal actions taken. There are no major conflicting details regarding the core incident.
None
Technology
Next-Token Prediction Model
Model predicts likely words, causing factual errors and wild assertions.
Black Box Nature of LLMs
Lack of transparency makes it impossible to trace how outputs are generated.
Inherent LLM Hallucinations
Generative AI systems naturally invent information and present it as fact.
Data Inputs
Lack of Real-Time Web Search
The model lacked live search capabilities to verify real-time facts.
Improper Data Blending
Correct personal details were incorrectly blended with unrelated tragic events.
Human Factors
Plausibility of False Outputs
Highly plausible responses trick users into believing false info is correct.
Influence of Prior Queries
Previous searches by the user potentially biased the chatbot's generation.
Process and Methods
Insufficient Safeguards
Relying on a small text disclaimer rather than technical prevention methods.
Inadequate Output Verification
No real-time verification process to filter out defamatory biographies.
Management
Lack of Transparency
OpenAI did not reply to access requests regarding internal data sources.
Inadequate Safety Testing
Deploying models without resolving severe hallucination and defamation risks.
Regulatory Environment
GDPR Accuracy Violations
Failure to comply with European data protection rules regarding personal data.
Lack of Pre-deployment Audits
Absence of regulatory mandates requiring pre-deployment safety audits.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
