This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A promotional demo for Microsoft's Bing Chat contained significant factual inaccuracies, including hallucinated product features for a vacuum, incorrect travel recommendations for Mexico City, and fabricated financial data for Gap Inc. and Lululemon. The AI confidently presented these errors as facts, even when citing sources that did not support the claims. This incident highlights the model's tendency to hallucinate information and its inability to reliably extract and summarize financial data.
Microsoft's demo video of Bing Chat reportedly featured false or made up information such as non-existent pet vacuums features or false figures on financial statements.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The model demonstrated a clear lack of capability and robustness in accurately extracting and summarizing structured financial data from documents.
The Bing Chat AI system generated and spread incorrect, fabricated, and misleading financial and product information during a highly publicized demonstration.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by the Bing Chat AI system which unintentionally generated incorrect and hallucinated information while attempting to fulfill user search and summarization queries.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
A public demonstration of Microsoft Bing Chat featured factual errors and financial data hallucinations. The incident highlights known technical limitations of large language models in data extraction and factual accuracy, but carries negligible national security implications.
Bing Chat operates as an AI assistant designed to help users search the web, summarize documents, and answer questions, requiring human oversight to verify its outputs.
Limited Risk: The system is a general-purpose conversational chatbot (Bing Chat) which falls under the Limited Risk category, requiring transparency obligations so users know they are interacting with an AI.
Bing Chat is a conversational AI chatbot designed to answer user queries, search the web, and summarize documents.
The report provides highly detailed, verifiable examples of the AI's errors during the demo, comparing the AI's outputs directly with the source documents (HGTV article, Gap and Lululemon financial statements). The evidence of hallucination is clear and unambiguous.
None; the errors are clear mathematical and factual hallucinations.
Technology
AI Hallucination of Facts
The model confidently generated false details about nightlife and vacuums.
Inaccurate Data Extraction
The model failed to extract correct numbers from financial PDF documents.
Product Version Confusion
The model confused corded and cordless product specs despite citation.
Data Inputs
Source Citation Discrepancy
Model claims to use specific sources but ignores or distorts their content.
Missing Real-Time Ratings
Lack of integration with current web reviews led to claims of no ratings.
Ambiguous Context Matching
Difficulty distinguishing between similarly named products in data sources.
Human Factors
Lack of Pre-Demo Verification
The Bing demo team failed to verify the accuracy of the generated outputs.
Overconfidence in AI Output
Presenters assumed AI-generated financial summaries were correct.
Process and Methods
Inadequate Demo QA Process
No rigorous review process was applied to pre-recorded demo outputs.
Flawed Evaluation Metrics
Evaluation methods failed to catch hallucinated financial metrics.
Management
Prioritizing Speed Over Safety
Rushing to launch and demo to compete with rivals led to missed errors.
Lack of Demo Accountability
No clear ownership for verifying the truthfulness of public demo content.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
