This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An elderly Malaysian couple and other travelers were misled into traveling to non-existent tourist destinations due to highly convincing AI-generated deepfake videos and chatbot hallucinations.
An elderly couple from Kuala Lumpur reportedly drove three hours to Kuak Hulu, Perak, on June 30, 2025, after viewing a purportedly Google Veo 3-generated video depicting a nonexistent cable car attraction called the Kuak Skyride. Published on TikTok by "TV Rakyat," the video featured a synthetic reporter and purported visitors. The couple learned after arriving that the attraction did not exist.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The reports also detail multiple instances of ChatGPT and other travel chatbots hallucinating non-existent destinations and incorrect schedules due to a lack of capability and robustness.
The AI system was used to generate highly realistic but entirely false video content that misled users, causing them to make real-world decisions that resulted in material and emotional harm.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The primary incident was caused by a human intentionally using Google's Veo3 AI system to generate and post a deceptive, realistic deepfake video to gain social media engagement.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
This incident involving AI-generated travel deepfakes and chatbot hallucinations represents a consumer safety and misinformation issue with negligible national security implications. The impact is confined to localized financial and emotional distress for individual travelers, without threatening critical infrastructure, sovereignty, or national economic stability.
The AI systems involved, such as ChatGPT, Google Gemini, and Veo3, operate as Assistants. They generate travel itineraries, suggestions, or video content to augment human activities, but they do not execute independent real-world actions or make autonomous decisions without human prompts and oversight.
Risk Level 3: Limited Risk. The incident involves AI-generated content (deepfakes) and chatbots, which are classified as limited risk under the EU AI Act and are subject to transparency obligations such as watermarking.
The primary incident was caused by a highly realistic AI-generated video of a non-existent cable car, which falls under Deepfake Video Generation. The secondary incidents involved travelers using ChatGPT for travel planning and itinerary queries, which falls under Question Answering.
The reports provide consistent, detailed accounts of the Malaysian couple's experience across multiple reputable news outlets, along with specific statistics and examples of chatbot hallucinations in other countries. The role of AI (Veo3 for the video, ChatGPT/Gemini for the itineraries) is explicitly stated, and the resulting financial and emotional harms are well-documented.
The video could be interpreted as a creative art project or parody rather than deliberate misinformation, though its realistic style led to actual deception.
Technology
Highly Realistic Generative AI
Google Veo3 generated highly convincing video of a non-existent attraction.
Fictitious News-Style Narration
The AI video fabricated a realistic news segment with a fake reporter.
Inconspicuous Platform Watermarks
The Veo3 logo in the corner was easily missed by viewers.
Data Inputs
Fabricated Video Footage
AI training data enabled the generation of realistic fake scenery.
Misleading Social Media Metadata
The video lacked tags or metadata identifying it as AI-generated.
Human Factors
Lack of Digital Literacy
Elderly users struggled to identify subtle AI artifacts like the blob error.
Reluctance to Verify Information
The couple was too embarrassed to consult their children before traveling.
Misplaced Trust in News Formats
The presence of a fake reporter made the couple believe the broadcast was real.
Process and Methods
Inadequate Platform Moderation
Social media platforms hosted the misleading video without warning labels.
Lack of Pre-Travel Verification
No robust process for users to easily verify digital tourism claims.
Delayed Content Takedown
The video was only removed after it went viral and caused real-world harm.
Management
Lack of Platform Oversight
Platforms did not manage the risks of realistic AI videos on their feeds.
Regulatory Environment
Absence of Specific AI Legislation
Malaysia had no specific laws regulating AI-generated media at the time.
Lack of Mandatory Watermarking
No legal requirements existed to enforce clear, unmissable AI labels.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
