This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Travelers using AI chatbots like ChatGPT and Layla for trip planning were misled by fabricated itineraries, resulting in financial losses, stranded tourists, and potential physical danger.
Two tourists in Peru reportedly relied on AI-generated travel information describing a nonexistent destination called the "Sacred Canyon of Humantay." One tourist had paid nearly $160 to reach a rural road near Mollepata without a guide or actual destination. Trek operator Miguel Ángel Góngora Meza intervened after seeing the purportedly AI-generated itinerary and warned that such misinformation could expose travelers to serious risks in high-altitude Andean terrain.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
Users relied on general-purpose chatbots for safety-critical planning, such as mountain hiking schedules and high-altitude routes.
The core technical failure is the AI models' lack of capability to prevent hallucinations and verify geographical facts.
The AI systems generated false and misleading travel information, leading users to form inaccurate beliefs and make decisions that resulted in financial loss and physical stranding.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incidents were caused by deployed AI systems generating unintended hallucinations and false information during user interactions.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Commercial AI chatbots generated fabricated travel itineraries and geographic details, leading to minor financial losses and localized stranding of tourists in countries like Japan and Peru. While highlighting safety concerns regarding user overreliance on LLMs for navigation, the incident has negligible national security implications and presents no threat to critical infrastructure, sovereignty, or state stability.
The AI systems act as travel assistants, providing suggestions and itineraries that require human validation and oversight.
Limited Risk: The systems described are chatbots (ChatGPT, Copilot, Gemini, Layla) used for generating travel itineraries, which fall under the category of chatbots and AI-generated content requiring transparency.
The primary function of the systems involved (ChatGPT, Gemini, Copilot, Layla) is acting as a conversational Chatbot to assist users with queries, while also serving as a Navigation Assistant to plan routes and itineraries.
The reports provide clear, consistent, and detailed accounts of multiple specific incidents across different countries, supported by quotes from affected travelers, local experts, and academic sources.
The incidents could be interpreted purely as user error due to overreliance, but the primary failure is the AI's generation of false information.
Technology
Inherent LLM Hallucinations
LLMs generate words statistically, leading to invented facts or destinations.
Lack of Physical World Understanding
AI cannot distinguish physical realities like mountain climbs from city walks.
Outdated Information Processing
AI provided outdated ropeway closing times, leaving tourists stranded.
Data Inputs
Statistical Word Association
AI combined regional terms to invent the non-existent Humantay Canyon.
AI-Generated Media on Social Media
TikTok videos generated by AI created realistic but fake tourist attractions.
Human Factors
Over-reliance on AI Planning
Travelers trusted AI itineraries without verifying safety-critical details.
Lack of Verification by Users
Users failed to double-check AI advice before hiking in remote areas.
Process and Methods
Lack of Fact-Checking Mechanisms
AI platforms lack real-time verification to filter out false travel details.
No Distinction in Output Presentation
AI presents hallucinations and facts with the same confident tone.
Management
Prioritizing Engagement Over Safety
Platforms deploy generative tools without ensuring factual accuracy for safety.
Regulatory Environment
Absence of AI Watermarking Rules
Lack of standard labels makes it hard to detect AI-generated travel media.
Lack of Prevention Regulations
Current regulatory focus is on mitigation rather than preventing AI errors.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
