This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Google's AI Overview feature provided incorrect information regarding the manufacturer of the aircraft involved in the fatal Air India Flight 171 crash, misidentifying a Boeing 787 as an Airbus A330. The error was attributed to the AI conflating references to Airbus as a competitor in news articles about the crash. Google acknowledged the error and manually removed the faulty response.
Following the fatal crash of Air India Flight 171, Google's AI Overview feature reportedly returned search results incorrectly identifying the aircraft involved as an Airbus A330 rather than the actual Boeing 787. The reported error, which varied across search queries, was traced to AI-generated summarizations that conflated references to Airbus as Boeing's competitor. Google later acknowledged and removed the faulty output.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The system failed to perform reliably, hallucinating and conflating facts due to a lack of capability to understand truth.
The AI system generated and spread incorrect factual information regarding a plane crash, leading to public misinformation.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The misinformation was generated by the AI system's non-deterministic hallucination post-deployment, which was an unintended outcome of its summarization goal.
Added categories peak at 2: Minor (Epistemic Harm, Inferred)
Google's AI Overview generated a factual error by misidentifying a Boeing aircraft as an Airbus model in a fatal crash summary. The incident represents a minor, unintentional AI hallucination in a public consumer tool with negligible national security implications, and was promptly corrected by human intervention.
The AI Overview acts as an information assistant, summarizing search results to help users find information, but it requires human verification as it is prone to mistakes.
Limited Risk: The system is a search summarizer generating AI content, which falls under Limited Risk (Level 3) due to transparency obligations for AI-generated content.
The AI Overview feature is designed to answer user search queries by summarizing web content.
The report clearly details the AI system involved, the specific error made, the likely technical cause of the hallucination, and the developer's response. There is very little ambiguity regarding the sequence of events.
None.
Technology
Non-deterministic Output
Generative AI produces different outputs even for identical inputs.
Lack of Semantic Understanding
AI lacks the ability to understand what is factually true.
Hallucination and Confabulation
System confidently generates plausible but false associations.
Data Inputs
Co-occurrence of Competitor Names
News articles mentioning competitor Airbus misled the model.
Human Factors
User Trust in Confident Output
Users overlook disclaimers when the AI presents answers confidently.
Low Prominence of Disclaimers
Users gloss over warnings placed inconspicuously at the bottom.
Process and Methods
Reactive Manual Removal
Google relies on manual deletion after errors are publicly reported.
Inadequate Validation of Summaries
No automated verification of factual claims before displaying them.
Management
Rapid Expansion of Unstable Tech
Google steadily expanded AI Overviews despite known accuracy issues.
Prioritization of Feature Rollout
Deploying AI features quickly over ensuring absolute factual accuracy.
Regulatory Environment
Absence of Generative AI Standards
Lack of industry standards for search engine AI accuracy.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
