This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A Google AI-generated search summary incorrectly identified Canadian musician Ashley MacIsaac as a convicted sex offender by conflating his biography with another individual. This misinformation led to the cancellation of a concert by the Sipekne'katik First Nation and caused the musician significant reputational and economic harm, as well as fear for his personal safety. Google subsequently amended the search results after the error was identified.
A Google Search AI-generated summary reportedly falsely stated that Canadian musician Ashley MacIsaac had been convicted of sexual offenses, apparently due to mistaken identity. After a venue reportedly relied on the AI-generated information, a scheduled concert was cancelled. The reported false summary caused reputational and economic harm and left the musician concerned for his personal safety. Google later amended the search results after the error was identified.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The AI system failed to perform reliably by conflating two distinct individuals with the same last name.
The AI system generated and spread false and defamatory information about an individual, leading to inaccurate beliefs in users and causing material and emotional harm.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The false biography was generated by Google's AI system post-deployment as an unintended outcome of its summarization algorithm.
Added categories peak at 2: Minor (Psychological Harm, Direct)
A Google AI-generated search summary falsely labeled a Canadian musician as a convicted sex offender due to data conflation. While the incident caused localized reputational, psychological, and financial harm to the individual, it has negligible national security implications and represents an unintentional technical hallucination rather than a hostile information operation.
The AI system, Google AI Overviews, operates as an assistant by aggregating and summarizing web search results to provide information to users, requiring human oversight to verify its accuracy.
Limited Risk: The system is an AI-generated content summarizer (AI Overview) which falls under transparency obligations for users interacting with AI-generated content.
The AI system's primary function is to search and retrieve web content, and its secondary function is to curate and summarize that content into a cohesive overview for users.
The report provides clear, direct evidence of the AI system's error, the specific harm caused to the musician, and statements from both the affected party and Google. There is minimal ambiguity regarding the sequence of events.
None
Technology
AI Hallucination and Blending
AI blended biography of two different people with the same last name.
Misinterpretation of Web Content
Search algorithm failed to capture context of distinct web sources.
Data Inputs
Ambiguous Name Data
Identical last names in data sources led to cross-contamination of profiles.
Human Factors
Overreliance on AI Summaries
Venues and users viewed AI-generated summaries as highly authoritative.
Process and Methods
Lack of Pre-computation Verification
No process existed to validate AI-generated summaries before display.
Management
Prioritizing Market Speed Over Safety
Google raced to compete with rivals, prioritizing deployment over accuracy.
Regulatory Environment
Lack of AI Accuracy Mandates
Absence of regulatory standards governing the accuracy of search-generated AI.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
