This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The VioGén algorithm, used by Spanish police to assess domestic violence risk, has been linked to fatal outcomes for victims who were incorrectly classified as low or negligible risk. Despite its intended purpose of prioritizing resources for vulnerable individuals, the system's reliance on potentially flawed input data and its influence on human decision-making have resulted in cases where victims received insufficient protection and were subsequently killed by their abusers. Government figures and independent reviews indicate that a significant number of homicide victims had been previously assessed by the system as low risk.
The VioGén algorithm was designed to help Spanish police assess and prioritize the risk of repeat domestic violence incidents. However, its low-risk assessment of Lobna Hemid reportedly led to inadequate protection; her husband murdered her. Since 2007, 247 women have been killed after being assessed by VioGén. A review of 98 homicides found that 55 of the slain women were scored as negligible or low risk.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Police officers and judges heavily relied on the algorithm's risk scores, accepting them 95% of the time and failing to provide protection to victims.
The VioGén system failed to perform reliably under real-world conditions, frequently miscalculating domestic violence risk levels and leading to fatal errors.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The fatal outcomes resulted from the VioGén AI system's incorrect low-risk classifications post-deployment, which was an unintended consequence of a system designed to protect victims.
Added categories peak at 3: Substantial (Psychological Harm, Inferred)
The Spanish government's use of the VioGen algorithm for domestic violence risk assessment resulted in systemic failures, leading to incorrect low-risk classifications and subsequent deaths of protected individuals. While representing a tragic failure of state protection and human rights, the incident remains a domestic policing issue with minor national security implications.
VioGén acts as an assistant to law enforcement, designed to 'aid police with limited resources identify and protect women' by generating a risk score based on a questionnaire, but requiring human input and final decision-making.
High Risk: The system is used in law enforcement and safety-critical public services to assess risk of violence, which has 'significant implications for safety, fundamental rights, and public interests'.
VioGén is a statistical risk assessment tool used by law enforcement to predict the likelihood of repeat domestic violence and assist judges in deciding protective measures.
The report provides extensive, detailed statistics from Spain's Interior Ministry and judicial reviews, along with multiple specific case studies of victims, police officers, and creators of the system. The role of the algorithm and the resulting real-world harms are clearly documented.
The failures could be interpreted primarily as human errors in data entry or police resource constraints rather than algorithmic failure, though the report emphasizes the systemic reliance on the algorithm's scores.
Technology
Flawed Risk Assessment Logic
Rigid statistical model fails to capture complex domestic abuse escalation.
Inability to Predict Rare Events
Algorithm struggles to accurately predict rare high-consequence homicides.
Opaque Algorithm Logic
Proprietary algorithm prevents outside audits and peer review of its safety.
Data Inputs
Incomplete Victim Disclosures
Fear, shame, and immigration status led victims to withhold key details.
Binary Question Restrictions
A 35-question yes/no form fails to capture nuanced threat details.
Missing Collateral Information
Police missed critical clues by not interviewing neighbors or school staff.
Human Factors
Automation Bias by Police
Officers accepted the VioGen score 95 percent of the time without questioning.
Overwhelmed Police Staff
Overburdened agents with high caseloads relied heavily on the algorithm.
Judicial Reliance on Scores
Judges denied restraining orders based primarily on low VioGen risk scores.
Process and Methods
Inadequate Interview Methods
Police-led questioning of tired victims without trusted support or experts.
Lack of Independent Audits
Government refused to allow external audits of the VioGen algorithm.
Insufficient Follow-up Protocols
Retesting protocols failed to detect escalating threats or broken orders.
Management
Resource Prioritization Bias
Using the algorithm to ration scarce protection resources instead of funding.
Inadequate Officer Training
Failure to provide sufficient training for handling algorithmic outputs.
Lack of System Transparency
Management failed to release comprehensive data on system effectiveness.
Regulatory Environment
Lack of Algorithmic Oversight
No regulatory framework to audit or govern predictive AI in law enforcement.
No Mandated Legal Support
Regulations did not guarantee specialized legal or psychological support.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
