This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Religious leaders and their congregations are increasingly targeted by AI-generated impersonation scams, where deepfakes and voice clones are used to solicit fraudulent donations or spread disinformation. These incidents involve the unauthorized use of likenesses on social media platforms like TikTok and Facebook. Additionally, there are concerns regarding the psychological impact of religious-themed chatbots and the potential for AI to reinforce delusions in vulnerable users.
Religious communities in the U.S. and abroad reported the circulation of purportedly AI-generated videos and cloned audio impersonating pastors and church leaders, including Father Mike Schmitz, to solicit donations and spread incendiary sermons. The impersonations reportedly leveraged synthetic voice and likeness tools and were distributed via platforms such as TikTok, Facebook, and messaging apps. Some content remained unlabeled as AI-generated and was treated as authentic by viewers.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
AI-generated sermons and videos are spreading false information that users mistake for genuine religious guidance.
Vulnerable users are relying on religious chatbots, which can reinforce delusions and exacerbate mental health issues like psychosis.
The primary harm involves scammers using AI-generated deepfakes and voice clones to impersonate trusted religious figures for financial gain and targeted manipulation.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was primarily caused by human malicious actors intentionally deploying AI deepfakes and voice clones to scam churchgoers.
Added categories peak at 4: Severe (Epistemic Harm, Direct)
This incident involves scammers using AI deepfakes and voice cloning to impersonate religious leaders in the US and Philippines for financial fraud. While it highlights the growing ease of creating convincing synthetic media and raises concerns about psychological vulnerability to AI, the national security threat remains low, as it does not target critical infrastructure, state sovereignty, or systemic economic stability.
The report describes scammers using generative AI tools to sample voices and create deepfake videos. The AI systems operated as tools, performing specific functions of media generation based on direct human inputs.
Risk Level 3. Limited Risk: AI systems that pose a moderate risk and require specific transparency obligations... Examples: Chatbots, AI-generated content such as deepfakes.
The primary method of scamming and influence described in the report is the creation of deepfake videos and cloned voices of religious leaders to deceive their congregations.
The report provides clear, detailed accounts of multiple religious figures being impersonated using AI deepfakes and voice clones, as well as the societal and psychological concerns surrounding religious chatbots. The roles of the AI systems as tools for human scammers are explicitly described.
None
Technology
Generative AI Voice Cloning
AI tools easily clone pastor voices from public weekly livestreams.
Realistic Deepfake Video Tech
Advanced AI video generation makes fake sermons appear highly authentic.
Data Inputs
Unprotected Public Sermons
Abundant online audio files provide easy training data for voice cloning.
Lack of AI Content Labeling
Missing labels make it hard for users to identify AI-generated content.
Human Factors
High Trust in Religious Leaders
Followers naturally trust pastors, making them highly vulnerable to scams.
Low Digital Literacy of Users
Many churchgoers struggle to identify subtle robotic cues in fake media.
Predisposition to Delusions
Vulnerable users have religious delusions reinforced by interactive AI.
Process and Methods
Inadequate Account Verification
Platforms fail to verify official figures, letting fake accounts proliferate.
Reactive Moderation Systems
Platforms rely on manual reports rather than proactive detection of deepfakes.
Management
Monetization of Viral AI Content
Platform creator funds reward high engagement over content authenticity.
Inadequate Platform Risk Review
Social media firms fail to assess safety risks of AI religious scams.
Regulatory Environment
Lack of Deepfake Regulations
Absence of clear laws governing unauthorized AI impersonation of citizens.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
