This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
ChatGPT hallucinated that the music software Soundslice supported ASCII tablature imports, causing users to upload incompatible files. This resulted in reputational strain for the company and forced them to develop an unplanned feature to accommodate the false expectations generated by the AI.
ChatGPT reportedly misinformed users that Soundslice's music software could import ASCII tablature, a feature the company did not offer. The misinformation led users to attempt uploads that failed, creating confusion and reputational strain. Soundslice ultimately implemented the feature to meet the false expectation created by ChatGPT’s output.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The incident stems from ChatGPT's technical limitation of hallucinating false facts, representing a lack of capability or robustness in generating accurate information.
ChatGPT generated false information claiming Soundslice supported ASCII tab imports, misleading users and forcing the company to adapt.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by ChatGPT hallucinating a feature post-deployment, which was an unintentional outcome of its text generation capabilities.
Added categories peak at 2: Minor (Epistemic Harm, Inferred)
An OpenAI ChatGPT hallucination falsely claimed that the music software Soundslice supported ASCII tab imports. This led users to upload incompatible files, straining the company's reputation and forcing them to build an unplanned feature. The incident represents a novel business impact of AI hallucinations but carries negligible national security implications.
The report describes ChatGPT acting as a conversational assistant that provides suggestions and instructions to users, who then manually act on those suggestions by uploading files to Soundslice.
Limited Risk: The system is a general-purpose chatbot (ChatGPT), which falls under Limited Risk due to transparency obligations to ensure users know they are interacting with an AI.
ChatGPT is primarily a conversational chatbot designed to answer user questions and assist with tasks, which in this case involved answering queries about music notation and software compatibility.
The report is a first-hand account from the founder of Soundslice, detailing the exact nature of the AI's hallucination, the evidence found in error logs, and the company's response. There is little ambiguity about what transpired.
None
Technology
LLM Hallucination
ChatGPT generated false facts about Soundslice capabilities.
Lack of Real-Time Verification
Model could not verify third-party features dynamically.
Data Inputs
Outdated Training Data
Training data lacked precise current features of third-party apps.
Unstructured Web Association
Model falsely linked Soundslice to ASCII tab processing.
Human Factors
User Overreliance on AI
Users blindly trusted ChatGPT instructions without verification.
Process and Methods
Inadequate Output Filtering
OpenAI lacked filters to catch false claims about external tools.
Management
Rapid Deployment of LLMs
Releasing conversational AI without robust accuracy safeguards.
Regulatory Environment
Lack of AI Liability Rules
No regulations hold AI creators liable for business disruptions.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
