This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The Fable book-tracking app utilized OpenAI's API to generate personalized year-end reading summaries for its users. These AI-generated summaries produced offensive, bigoted, and biased commentary targeting users based on their reading choices related to race, gender, sexual orientation, and disability. Following public backlash and user reports, Fable apologized and removed the AI-powered summary features from its platform.
Fable, a book-focused social app, used OpenAI’s API to generate AI-powered year-end reading summaries in December 2024. These summaries allegedly produced biased and offensive remarks about race, gender, and sexual orientation. Fable is reported to have apologized in a social media post on January 1, 2025.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and unfair representation of those groups.
Also classified as
The AI exposed users to offensive, bigoted, and inappropriate language that violated community norms.
The AI system failed to perform reliably, and its built-in safety filters failed to catch the offensive outputs.
The AI system generated outputs that unfairly stereotyped and discriminated against users based on protected characteristics like race, gender, and disability.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The offensive outputs were generated post-deployment by the AI system itself, which was deployed by Fable to create playful summaries but unintentionally produced bigoted commentary.
Added categories peak at 2: Minor (Psychological Harm, Direct)
The incident involves an AI-powered feature in a commercial book-tracking app that generated offensive and biased reading summaries. While causing distress to users and commercial disruption for the company, the event is entirely domestic and commercial in nature, presenting negligible implications for national security.
The AI system operated independently to generate and deliver reading summaries directly to users without human review or oversight, responding to user reading data using learned patterns.
Limited Risk: The system is an AI-generated content feature (personalized summaries) which requires transparency obligations such as notifying users of AI interaction.
The AI system was used to automatically curate and summarize users' reading histories into personalized year-end recaps on a social platform.
The reports from multiple reputable sources (Wired, USA Today, NYT) provide consistent, detailed accounts of the incident, including direct quotes from affected users and Fable executives. The role of the AI system (OpenAI API) and the nature of the generated text are clearly documented, leaving very little ambiguity about what transpired.
None
Technology
Failure of Offensive Filters
Filters for offensive language and topics failed to block the bad outputs.
Inherent Model Biases
Underlying LLM carries societal biases that influenced generation.
Unchecked Generative AI Output
AI directly pushed out content to users without automated validation.
Data Inputs
Biased Training Data
Algorithms are built on data from a biased society, carrying those biases.
User Reading History Input
Personalized reading lists triggered inappropriate demographic profiling.
Human Factors
Programmer Biases
Developers' conscious or unconscious biases are carried into the machine.
Underestimation of AI Risks
Fable team underestimated work needed for responsible AI deployment.
Process and Methods
Lack of Human-in-the-Loop Review
Small team could not review 2 million individual summaries before release.
Flawed Prompting Strategy
Prompting the AI to 'playfully roast' users led to offensive commentary.
Inadequate Pre-release Testing
The feature lacked rigorous internal testing and safeguards before launch.
Management
Prioritizing Automation Over Safety
Relying heavily on AI to scale personalized summaries without oversight.
Underestimating Safety Work
Management underestimated the resources required for safe AI deployment.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
