This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Google's Smart Reply feature for Gmail and Inbox initially exhibited a 'propensity' to suggest inappropriate responses, most notably 'I love you' and 'Sent from my iPhone', due to the system's training on common email patterns. These suggestions were deemed inappropriate for professional settings, leading Google to tune the algorithm to filter out such responses. The incident highlights the challenges of training AI on large datasets without sufficient social context awareness.
Google's Gmail Smart Reply tool was over-recommending the response "I love you" in situations where it was deemed innappropriate.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Users relying on automated replies can lead to shallower human connections and dehumanized communication.
The AI system lacked the capability to understand social context, leading to inappropriate and robustly incorrect suggestions like 'I love you' in professional contexts.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The issue was caused by the Smart Reply AI system unintentionally generating inappropriate suggestions due to unexpected outcomes of its training process, which was observed during early post-deployment rollouts and testing.
A minor, unintentional software bug in Google's Smart Reply feature suggested inappropriate phrases like 'I love you' due to training data patterns. The issue was quickly mitigated by Google engineers prior to widespread impact and poses negligible national security risk.
The Smart Reply feature acts as an assistant by suggesting three short email responses that the user must manually select and send. As Report 8 states, 'It gives you three of these responses, and you can then choose the one that best suits what you want to say.'
Limited Risk: The system is an AI assistant that interacts with users to suggest text replies, falling under transparency obligations for AI systems interacting with humans.
The primary function of Smart Reply is to assist users in writing emails by predicting and suggesting short, contextually relevant responses.
The reports provide consistent, detailed accounts of the Smart Reply feature's development, testing, and rollout, including specific anecdotes about the 'I love you' and 'Sent from my iPhone' suggestions. Multiple sources confirm the technical details of the LSTM neural networks used and the organizational response by Google.
None
Technology
Frequent Phrase Bias
Model defaulted to common phrases like 'I love you' when signal was weak.
Lack of Social Context
Neural network could not distinguish between personal and professional context.
Semantic Duplication
Early algorithms suggested multiple replies with nearly identical meanings.
Data Inputs
Unfiltered Training Data
Dataset included default signatures like 'Sent from my iPhone' as replies.
Affectionate Tester Bias
Internal Google testers used 'I love you' frequently, biasing the prototype.
Human Factors
Implicit Signature Usage
Users frequently sending default iPhone signatures skewed the training corpus.
Process and Methods
Absence of SFW Filtering
Initial training lacked filters to block unprofessional or affectionate replies.
Lack of Diversity Parameters
Early tuning did not enforce sufficient semantic distance between suggestions.
Management
Lack of Pre-deployment Checks
Features were tested internally but lacked early workplace safety guidelines.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
