This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Microsoft Copilot repeatedly generated defamatory content falsely identifying a German court reporter as a perpetrator of crimes he had covered in his professional capacity. The system also exposed the journalist's private contact information. Despite attempts to have the information removed, the false claims persisted, highlighting the unreliability of LLMs as research tools and the challenges of enforcing data accuracy and privacy rights under GDPR.
Microsoft's Copilot is reported to have falsely accused veteran court reporter Martin Bernklau of committing serious crimes, including child abuse and fraud. The tool is described as having generated defamatory content that not only accused Bernklau of multiple crimes he covered as a journalist but also provided his personal contact details. Attempts by Microsoft to remove the false entries were only temporarily successful, as the defamatory information reportedly reappeared.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The AI system exposed the journalist's private contact details, including his full address and phone number, without consent.
The system failed to perform reliably, hallucinating false narratives and failing to maintain permanent deletions of the incorrect data.
The AI system generated false and defamatory accusations against a journalist, confusing him with the criminals he had reported on, leading to severe emotional distress and reputational risk.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was caused by the AI system's generation of false information post-deployment, which was an unintended outcome of its design.
Added categories peak at 2: Minor (Psychological Harm, Direct)
Microsoft Copilot generated defamatory statements and leaked private contact details of a German journalist due to LLM hallucinations. While causing personal and psychological distress, the incident presents negligible direct threat to national security, highlighting instead systemic challenges in generative AI data accuracy and GDPR compliance.
The report describes Microsoft Copilot as an AI chatbot and search assistant that users query to retrieve information. It acts as an assistant by summarizing web search results and providing answers to user queries, but requires human verification due to its tendency to hallucinate.
Limited Risk: The system is a chatbot (Microsoft Copilot), which falls under Risk Level 3 due to transparency obligations requiring users to be informed they are interacting with an AI.
The system is primarily deployed as an interactive conversational chatbot (Copilot) that also performs web search and content summarization.
The reports provide consistent, detailed accounts of the specific false accusations, the journalist's reaction, the technical reasons for the failure, and the legal/regulatory context. There are no major contradictions.
None
Technology
Entity Association Failure
Model confused the court reporter with the criminals in his reports.
Probabilistic Output
LLM generates text based on statistical probability, not factual truth.
Source Hallucination
System fabricated sources and article titles to support false claims.
Data Inputs
Unstructured Public Data
Decades of online court reports linked the journalist to crime terms.
Irrelevant Source Linking
Search results returned unrelated videos and articles for the name.
Human Factors
User Overreliance
Users assume search chatbots provide verified and factual information.
Process and Methods
Ineffective Data Deletion
Microsoft failed to permanently delete the false information from LLM.
Lack of Output Verification
No real-time fact-checking process exists before displaying results.
Management
Inadequate Incident Response
Microsoft applied temporary blocks that failed after a few days.
Reliance on Disclaimers
Terms of service disclaim liability instead of guaranteeing accuracy.
Regulatory Environment
Legal Accountability Gap
Prosecutors rejected charges because the AI is not a legal person.
GDPR Enforcement Deficit
Current regulations struggle to enforce data rectification in LLMs.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
