This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Following a tragic school minibus crash in South Africa, social media users circulated fabricated, AI-generated memorial posters depicting fake faces of the deceased children, causing distress to grieving families.
Purportedly AI-generated memorial posters reportedly presented fabricated child portraits as pupils killed in a school-transport crash near Vanderbijlpark, South Africa. The posters circulated on Facebook, X and TikTok before authorities released the victims’ identities or photographs. AFP reported that the depicted faces did not match official victim images, and the Gauteng education department condemned the posts while families were grieving.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The AI-generated posters exposed grieving families and the public to inappropriate and distressing synthetic depictions of deceased children, violating community norms.
The AI tools were used to generate and spread completely fabricated images and identities of deceased children, misleading the public on social media platforms.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human actors intentionally using deployed AI image generation tools to fabricate and distribute fake memorial posters of deceased school children.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
The incident involved the creation and viral spread of AI-generated fake memorial posters following a tragic school minibus crash in South Africa. While causing significant emotional distress to grieving families and spreading localized social media misinformation, the event has negligible national security implications and does not threaten critical infrastructure, sovereignty, or state stability.
The AI system was used as a tool to generate synthetic images based on direct user inputs, performing a specific, well-defined function without autonomous decision-making or planning.
Limited Risk: The incident involves AI-generated content (synthetic images/deepfakes), which is subject to transparency obligations under the EU AI Act to ensure users are aware they are interacting with or viewing AI-generated media.
The AI system was used to generate synthetic images of children's faces to depict them as the deceased victims of the crash.
The report provides clear, verified facts about the creation and spread of the AI-generated posters, including statements from government officials and verification from Hive Moderation. There is little ambiguity about what occurred.
None
Technology
AI Image Generation Tools
AI tools allowed easy fabrication of realistic faces of deceased children.
Eerie Resemblance in AI Faces
Generated faces had eerie similarities, showing synthetic generation.
Data Inputs
Lack of Actual Victim Photos
Lack of official photos led creators to generate synthetic alternatives.
Unverified Fatality Statistics
Early incorrect victim counts were used to create the fake graphics.
Human Factors
Desire for Social Media Reach
Users shared fake posters thousands of times to gain online engagement.
Lack of Empathy by Creators
Creators fabricated images while families were actively grieving.
User Credulity and Resharing
Public reshared the fake graphics without verifying their authenticity.
Process and Methods
Lack of Platform Verification
Social networks lacked automated systems to flag fake memorial graphics.
Premature Media Dissemination
Graphics were published before official government verification.
Management
Slow Content Moderation
Social media platforms failed to quickly remove misleading AI graphics.
Regulatory Environment
No AI Content Labeling Laws
No regulations exist to force labeling of synthetic memorial images.
