This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Extremist groups, including ISIS and neo-Nazi networks, are increasingly utilizing AI tools to produce propaganda, deepfake news broadcasts, and instructional content. By leveraging AI for voice cloning, image generation, and text-based chatbots, these groups have significantly reduced the time and cost required to create professional-grade radicalization materials. This trend represents a shift in how extremist organizations disseminate content and recruit followers, posing new challenges for content moderation and counter-terrorism efforts.
ISIS supporters have created an AI-generated media program called News Harvest to disseminate propaganda videos. The program produces near-weekly broadcasts featuring AI-generated news anchors discussing ISIS operations globally, using cheap and easy-to-use AI tools. This development showcases the use of AI as a powerful propaganda tool for extremist groups.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The AI tools are being used to generate and spread highly toxic content, including extremist propaganda, hate speech, and bomb-making instructions.
Far-right extremists are using voice cloning and deepfake technologies to impersonate public figures for targeted manipulation and recruitment.
Extremist organizations are using AI to conduct coordinated, large-scale propaganda and influence campaigns aimed at manipulating public opinion and recruiting followers.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk is directly caused by human extremists intentionally utilizing deployed AI systems to generate propaganda and bypass safety guardrails.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
Extremist and terrorist organizations are successfully exploiting consumer generative AI tools to lower the cost and accelerate the scale of propaganda, deepfakes, and recruitment. While not representing an immediate kinetic crisis, this evolved capability significantly enhances information warfare and societal polarization threats globally.
The reports describe extremists using AI systems like ElevenLabs and ChatGPT as tools to generate specific outputs such as cloned voices, translated text, and synthetic images based on direct user prompts.
Risk Level 3: Limited Risk. The reports describe the use of 'AI-generated content such as deepfakes' and 'chatbots' which require transparency obligations under the Act.
The primary application described is the creation of synthetic news broadcasts ('News Harvest') and deepfakes of public figures to spread propaganda.
The reports are highly detailed and consistent, drawing from credible monitoring organizations like SITE Intelligence Group and MEMRI. They provide specific examples of AI tools, platforms, and extremist groups involved.
None
Technology
Highly Accessible Generative AI
Cheap, easy tools like ElevenLabs and Wav2Lip enable rapid media creation.
Bypassable Chatbot Guardrails
Users easily sidestep safety protocols with simple social engineering prompts.
High-Fidelity Translation Tools
AI allows extremist materials to be translated with chilling accuracy.
Data Inputs
Extremist Text as Prompts
Official ISIS dispatches and Mein Kampf are fed directly into AI generators.
Lack of Input Sanitization
AI systems process extremist literature without blocking harmful source texts.
Human Factors
Proactive Extremist Adoption
Extremists quickly adopt new technologies to gain strategic advantages.
Social Engineering of Chatbots
Users trick AI by claiming to be police officers or certified experts.
Process and Methods
Inadequate Content Moderation
Platforms like Rocket.Chat, Telegram, and X fail to remove extremist media.
Innocuous Packaging of Propaganda
AI-generated news style hides extremist content, making moderation harder.
Management
Reckless Tech Company Governance
AI companies shirk responsibility and fail to proactively block bad actors.
Lack of Proactive Risk Assessment
Companies release powerful tools to the public without adequate safety fences.
Regulatory Environment
Lagging Government Regulations
Lawmakers and regulators cannot keep pace with rapidly evolving AI tech.
Under-Informed Regulators
Regulators lack the expertise to address AI-driven extremist threats.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
