This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Google's AI Overview feature generated harmful and unsanitary advice for parents by misinterpreting a potty training technique. The AI failed to understand that a source video used the term 'poo' to refer to shaving cream, leading the system to explicitly recommend that parents use actual human feces. Google acknowledged the error, attributing it to a failure in contextual interpretation.
Google's AI Overview feature mistakenly advised parents to use human feces in a potty training exercise, misinterpreting a method that uses shaving cream or peanut butter as a substitute. This incident is another example of an AI failure in grasping contextual nuances that can lead to potentially harmful, and in this case unsanitary, recommendations. Google has acknowledged the error.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The AI exposed users to highly inappropriate and unsanitary advice, encouraging unsafe hygienic practices.
The AI generated and spread incorrect and misleading advice regarding potty training methods.
The AI system failed to perform reliably by failing to comprehend the context of the source video, leading to an absurd and unsafe recommendation.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by the AI system generating unexpected and incorrect advice post-deployment while attempting to fulfill a search query.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Google's AI Overview generated highly unsanitary potty training advice due to a semantic context-parsing failure, recommending parents use actual feces instead of shaving cream. This commercial software failure represents zero threat to national security, critical infrastructure, or sovereign stability.
The AI Overview acts as a search assistant, providing information and suggestions to augment human search capabilities, but requiring human oversight and decision-making.
Limited Risk: The system is an AI search assistant that generates content, falling under the category of chatbots and AI-generated content which require transparency obligations.
The AI system's primary function is to search and summarize web content to answer user queries within Google Search.
The report provides clear, unambiguous details about the search queries, the AI's specific outputs, the source material, and Google's response, leaving no major gaps in understanding the incident.
None
Technology
LLM Contextual Misinterpretation
The AI failed to understand the playful context of the word 'poo' in the video.
Missing Quotation Mark Parsing
The AI omitted quotation marks that would have properly contextualized the tip.
Unreliable Reasoning Capabilities
The LLM lacks real-world common sense to identify unsanitary recommendations.
Data Inputs
Ambiguous Source Terminology
The source video used the word 'poo' to describe harmless shaving cream.
Human Factors
User Trust in Search Outputs
Parents rely on search engines for safe, actionable parenting advice.
Process and Methods
Reactive Manual Hotfixing
Google relies on fixing bad responses piecemeal after they are reported.
Inadequate Pre-deployment Filters
Lack of automated safety filters to catch hazardous or unhygienic suggestions.
Management
Premature Product Rollout
Deploying unproven generative AI features to a massive user base.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
