This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The Pak 'n' Save 'Savey Meal-bot' app, intended to help customers use up leftovers, began generating dangerous and toxic recipes when users input non-food household items. The AI suggested recipes for chlorine gas and other poisonous mixtures, presenting them as refreshing beverages or meals. The supermarket acknowledged the issue and stated they would continue to refine controls, while noting that recipes are not human-reviewed.
Pak 'n' Save's AI-based app, Savey Meal-bot, inadvertently suggested dangerous recipes, including one creating chlorine gas, when users entered non-food household items, raising safety and oversight concerns.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The bot exposed users to highly unsafe and harmful content by describing and recommending the creation of deadly chlorine gas and poison sandwiches.
The AI system failed to perform reliably when presented with non-food ingredients, lacking the robustness to filter out hazardous household chemicals and instead generating dangerous recipes.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was caused by the Savey Meal-bot AI system generating hazardous recipes post-deployment, which was an unexpected and unintentional outcome of its goal to generate meal plans.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
A supermarket meal-planning AI generated hazardous recipes, including chlorine gas, after users entered non-food items. While presenting a localized public safety concern, the incident did not compromise physical infrastructure, government functions, or national security, representing a commercial AI safety failure rather than a national security threat.
The Savey Meal-bot acts as an assistant designed to augment human capabilities by providing recipe suggestions based on user-inputted ingredients, requiring human oversight and decision-making to actually prepare the meals. The report notes that the recipes 'are not reviewed by a human being' and warns users to 'use your own judgement before relying on or making any recipe'.
Limited Risk: The system is a chatbot ('Savey Meal-bot') that generates content and interacts with users, which falls under Risk Level 3 requiring transparency obligations so users know they are interacting with an AI.
The system is a chatbot ('Savey Meal-bot') designed to interact with users, take ingredient inputs, and generate recipes with cheery commentary.
The report clearly details the AI system, its developer, the specific outputs generated, and the public reaction. There is no ambiguity about what the AI did or how users interacted with it.
None
Technology
No Input Filtering
App accepted hazardous non-food items like bleach and glue as ingredients.
Unconstrained LLM Generation
The AI model lacked safety guardrails to block hazardous recipe generation.
Data Inputs
Harmful User Inputs
Users entered toxic household chemicals into the recipe generator.
Lack of Ingredient Validation
The system did not validate if inputs were safe, edible food items.
Human Factors
Adversarial User Behavior
Users actively experimented with toxic inputs to test the AI's limits.
Overreliance on Disclaimers
Management assumed terms and conditions would prevent unsafe user actions.
Process and Methods
No Human in the Loop
Recipes were generated and displayed instantly without human review.
Inadequate Safety Testing
The application was deployed without testing against adversarial inputs.
Management
Transfer of Safety Risk
Management shifted safety responsibility to users via disclaimers.
Inadequate Risk Assessment
Failed to foresee risks of users entering non-food household items.
Regulatory Environment
Lack of AI Safety Standards
No regulatory mandates enforced safety guardrails on consumer AI apps.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
