This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
AI deepfake detection tools are exhibiting significant performance biases, as they are primarily trained on Western-centric data. This results in high error rates when analyzing content from the Global South, including false positives for non-native English speakers and failures to recognize lower-quality media common in those regions. These inaccuracies undermine election integrity and hinder the ability of journalists and researchers to combat misinformation effectively.
AI deepfake detection tools are reportedly failing voters in the Global South due to biases in their training data. These tools, which prioritize English language and Western faces, show reduced accuracy when detecting manipulated content from non-Western regions. As a result of this detection gap, election integrity faces threats from and the amplification of misinformation, which leaves journalists and researchers with inadequate resources to combat the issue.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
The failure of detection tools in the Global South allows political disinformation campaigns to spread unchecked during elections.
The detection models fail to perform robustly under real-world conditions such as low-quality media, background noise, and compression.
The AI detection tools perform with significantly lower accuracy and higher false-positive rates for non-Western faces and non-native English speakers due to biased training data.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident is caused by biased classification decisions made by deployed AI detection systems, which unintentionally fail to perform robustly on non-Western data.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
Technical bias in AI deepfake detection tools, which are trained primarily on Western data, leads to severe performance degradation in the Global South. This failure undermines the ability of researchers and journalists to counter political disinformation during elections, posing a substantial threat to information security and democratic processes in affected nations.
The AI systems described are deepfake detection models that perform the specific, well-defined function of classifying media as authentic or manipulated based on direct inputs. They operate as tools to assist journalists and researchers in verifying content.
Limited Risk: The report describes AI systems used for deepfake detection and synthetic media verification, which fall under the category of managing AI-generated content and deepfakes.
The primary goal of the AI systems (True Media and Reality Defender) is to verify whether media content (images, video, audio, or text) is authentic or AI-generated/manipulated.
The report provides clear, detailed, and consistent testimonies from multiple experts and organizations (Witness, Tech Global Institute, Thraets) regarding the specific technical biases and real-world impacts of AI detection tools in the Global South. The causal mechanisms (biased training data, lack of local compute, low-quality hardware inputs) are explicitly explained.
The failures could be interpreted primarily as a market access or economic inequality issue rather than an inherent AI safety failure, though the resulting algorithmic bias directly constitutes an AI safety risk.
Technology
Lack of local compute infrastructure
Global South lacks data centers to run local detection models.
Sensitivity to media compression
Compression and background noise trigger false detection results.
Western-biased model design
Detection tools are optimized for Western markets and demographics.
Data Inputs
Undigitised local data sources
African data in hard copy prevents training models on local contexts.
Lack of diverse training data
Models lack training on non-Western accents, languages, and faces.
Training on high-quality media
Models fail on low-quality photos from cheap smartphones.
Human Factors
Untrained researchers
Researchers mistake simple cheapfakes for AI manipulations.
Reliance on inaccurate free tools
Journalists use faulty free tools due to lack of better options.
Process and Methods
Significant verification lag time
Sending media overseas takes weeks, allowing disinformation to spread.
Overwhelmed rapid response teams
High volume of cases exceeds the capacity of verification programs.
Management
Misallocated safety funding
Funding goes to detection instead of resilient news ecosystems.
Prohibitive costs of commercial tools
High cost of reliable tools forces reliance on inaccurate free versions.
Regulatory Environment
Risk of misguided regulation
Inflated false positives risk driving legislation for imaginary issues.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
