This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A patient diagnosed with Chronic Lymphocytic Leukemia (CLL) used Perplexity AI to self-diagnose a rare complication, leading him to reject his oncologist's recommended Ven-Obi treatment for a year. The AI had incorrectly summarized medical research, a fact later confirmed by the study's authors. The patient's condition worsened significantly during this delay, and he passed away shortly after eventually starting the treatment.
Joseph Neal Riley reportedly used Perplexity AI to self-diagnose a rare complication of chronic lymphocytic leukemia (CLL) and, based on that output, allegedly delayed an oncologist-recommended Ven-Obi treatment for about a year. His son, Benjamin Riley, alleges that Perplexity misstated the medical research it cited, a conclusion he purportedly later confirmed with the study's authors.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The patient relied on the AI system for critical medical decisions, bypassing professional medical advice.
The AI system failed to accurately summarize the medical research paper, demonstrating a lack of capability in factual retrieval.
The AI system generated false and misleading medical information that led the patient to make an unsafe decision regarding his cancer treatment.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by the Perplexity AI system generating incorrect medical information after its deployment.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
A private citizen tragically passed away after delaying cancer treatment based on incorrect medical research summaries generated by Perplexity AI. While representing a serious consumer safety and AI reliability issue, the incident has negligible implications for national security.
The AI system acted as an assistant by providing information and suggestions to the user, requiring human oversight and decision-making.
Limited Risk: The system is a chatbot/conversational search engine, which falls under transparency obligations for interactive AI systems.
The AI system was used to answer questions and search for information regarding medical research and treatment options.
The report provides a detailed, first-hand account of the incident, specifying the AI system used (Perplexity AI), the exact medical context (CLL and Ven-Obi treatment), and the specific error made by the AI (misstating the conclusion of a medical study). The chain of causality from the AI's output to the patient's decision and subsequent health decline is clearly documented.
The incident could be viewed primarily as a case of human medical non-compliance driven by pre-existing fear of hospitals, with the AI merely serving as a convenient justification rather than the primary cause.
Technology
AI Hallucination of Facts
The model generated non-factual statements and misstated research findings.
Authoritative Tone of Output
The AI strikes an authoritative tone that makes errors seem highly credible.
Data Inputs
Misrepresented Research Data
The AI incorrectly summarized the conclusion of the cited medical study.
Human Factors
Automation Bias in Self-Diagnosis
The patient trusted the AI's diagnosis over his oncologist's recommendation.
Confirmation Bias Amplification
The AI's output affirmed the patient's existing fear of hospital treatments.
Process and Methods
Inadequate Output Verification
The system relies on users to verify complex and critical medical outputs.
Management
Prioritizing Utility Over Safety
Promoting AI as a health resource despite known factual unreliability.
Regulatory Environment
Lack of Medical AI Regulation
Absence of strict enforcement preventing AI from giving unlicensed advice.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
