This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
OpenAI's Whisper transcription model is widely reported to produce frequent hallucinations, including fabricated medical information and toxic content. Despite OpenAI's warnings against using the tool in high-risk domains, it has been integrated into medical transcription services like Nabla, which are used in thousands of clinical settings. Concerns are heightened by the fact that some implementations delete original audio, preventing verification of the AI-generated transcripts.
OpenAI's AI-powered transcription tool Whisper, used to translate and transcribe audio content such as patient consultations with doctors, is advertised as having near “human level robustness and accuracy.” However, software engineers, developers and academic researchers have alleged that it is prone to making up chunks of text or even entire sentences and that some of the hallucinations can include racial commentary, violent rhetoric, and even imagined medical treatments.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The model's hallucinations include generating violent rhetoric and unsolicited racial commentary, exposing users to inappropriate content.
Hospitals and medical centers are deploying Whisper in high-risk clinical environments despite explicit warnings from OpenAI against its use in such domains.
The incident stems from Whisper's technical failure to perform reliably, resulting in a high rate of hallucinations and fabricated text during transcription.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk is caused by the Whisper AI model generating unexpected and erroneous hallucinations during its post-deployment operation.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
The widespread deployment of OpenAI's Whisper model in healthcare and public transcription, despite severe hallucination flaws and toxic output generation, poses minor national security concerns. The primary risks are localized to healthcare data integrity, patient safety vulnerabilities, and minor societal inequalities, which remain manageable under standard procedures.
Whisper operates as an AI assistant designed to transcribe and summarize audio, augmenting human capabilities (such as medical note-taking) but requiring human oversight to verify and edit the outputs.
High Risk: The report describes Whisper being integrated into medical transcription tools used by clinicians in healthcare settings, which represents an AI system used in healthcare and diagnostic support.
Whisper is primarily designed as an automatic speech recognition (ASR) tool to convert spoken audio into written text, and is also used to translate text into multiple languages.
The reports provide detailed, multi-sourced evidence from academic researchers, machine learning engineers, and former OpenAI employees, citing specific studies and statistics regarding Whisper's hallucination rates.
The incident could be interpreted primarily as a user-error/overreliance issue (5.1) due to hospitals ignoring OpenAI's warnings, but the unique severity of Whisper's technical flaws supports a primary classification of lack of capability (7.3).
Technology
Frequent Model Hallucinations
Whisper generates fabricated text, including violent and racial commentary.
Acoustic Vulnerabilities
Fabrications tend to occur during pauses, background sounds, or music.
Data Inputs
Audio Ground Truth Erasure
Nabla erases original audio, preventing verification of transcript accuracy.
Confidentiality and Privacy Risks
Sharing intimate medical audio with tech vendors raises privacy concerns.
Human Factors
Overconfidence in AI Accuracy
Users trust Whisper's 'human-level' claims, ignoring potential errors.
Lack of Manual Verification
Clinicians may fail to double-check AI transcripts against actual audio.
Process and Methods
Deploying in High-Risk Domains
Using Whisper in medical transcription despite warnings against high-risk use.
Inadequate Verification Processes
Lack of robust mechanisms to catch and correct hallucinated transcriptions.
Management
Ignoring High-Risk Warnings
Hospitals deploy Whisper despite warnings against high-risk use.
Prioritizing Speed Over Safety
Rushing integration of unverified tools to save administrative time.
Regulatory Environment
Absence of AI Regulations
Lack of federal regulations governing AI safety and deployment standards.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
