This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Firefighters in Manila were dispatched to a reported truck fire after receiving an AI-generated image of the incident. Upon arrival, they discovered the vehicle was intact and the report was false. The incident highlights the risks posed by AI-generated misinformation in emergency services, as the unnecessary deployment of fire trucks created potential safety hazards for the public and the responders.
Firefighters in Manila reportedly responded to a supposed burning truck after being alerted with a photo later determined to have purportedly been AI-generated. Four fire trucks, including units from the Bureau of Fire Protection, reportedly arrived on the scene but found the vehicle intact. The false alert allegedly wasted emergency resources.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The synthetic image was used by an anonymous actor to intentionally manipulate the behavior of emergency services through a targeted hoax.
The AI system was used to generate a highly realistic but false image of a burning truck, which misled emergency responders into believing a real fire was occurring.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by an anonymous human actor who intentionally generated and shared a fake image of a burning truck to mislead emergency services.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
An anonymous actor used generative AI to create a highly realistic image of a truck fire, prompting a false dispatch of Manila firefighters. While the national security impact is minor, the incident highlights how easily accessible AI tools can be used to manipulate local emergency services, creating minor public safety hazards and disrupting municipal government functions.
The AI system was used as a deterministic tool to generate a realistic image of a burning truck based on direct user inputs, without exhibiting independent planning, autonomy, or context retention.
Limited Risk: The system generated a deepfake image. The report describes 'an AI-generated photo of a burning truck' which falls under 'AI-generated content such as deepfakes' requiring transparency.
The AI was used to generate a highly realistic but entirely fabricated image of a burning truck to deceive others on social media.
The reports from multiple local news outlets consistently describe the same event: firefighters responding to a fake fire due to an AI-generated image. The details about the response, the location, and the reactions of the authorities are clear and uncontradicted.
None.
Technology
Highly Realistic AI Image Generation
The AI-generated photo of the burning truck looked extremely realistic.
Ease of Generating Fake Media
AI tools enable anonymous users to easily generate convincing disaster images.
Data Inputs
Unverified Social Media Image
The report relied on an unverified AI-generated image shared on social media.
Human Factors
Malicious Intent of Creator
An anonymous user created and spread the fake emergency image to deceive.
Public Credulity to AI Images
A follower believed the realistic AI photo was real and alerted authorities.
Urgent Response to Fake Reports
Firefighters rushed to the scene immediately, risking safety during transit.
Process and Methods
Lack of Image Verification
No verification protocol existed to check image authenticity before dispatch.
Reliance on Social Media Alerts
Emergency services relied on unverified reports originating from social media.
Regulatory Environment
Anonymity Impeding Accountability
Anonymous online profiles make enforcing the Revised Fire Code difficult.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
