This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
VioGén is a Spanish police algorithm designed to assess recidivism risk in gender violence cases. External audits and reports indicate that the system frequently underestimates risk, often assigning 'low' or 'no risk' scores to victims who are subsequently murdered. The system's performance is criticized for lack of transparency, reliance on rigid questionnaires, and the influence of available police resources on risk classification, leading to systemic failures in protecting vulnerable individuals.
In Spain, the algorithm that assesses recidivism risk in gender violence, VioGén, have critically underestimated the level of risk in a series of cases that ended in homicide of women and children since its first deployment.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
Police officers blindly rely on the algorithmic score in 95% of cases, treating the recommender system as an automated decision-maker without exercising meaningful human oversight.
The system fails to perform reliably, frequently underestimating risk and failing to predict lethal violence, leading to the murder of unprotected victims.
The system exhibits significant demographic bias, systematically assigning lower risk scores to women without children, which resulted in a 44% recall difference in extreme risk cases.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The physical and psychological harms resulted from the VioGén algorithm's failure to accurately assess risk after it was deployed in police stations.
Added categories peak at 3: Substantial (Psychological Harm, Indirect)
The VioGen algorithmic risk assessment system in Spain failed to protect domestic violence victims due to systemic underestimation of risk and demographic biases. While causing tragic physical harm and violating fundamental human rights on a large scale, the national security threat remains limited to domestic societal stability and government police function efficacy.
VioGén is designed to assist police officers by providing a risk score to guide their protection decisions, requiring human input and intended to augment human decision-making.
High Risk: The system is a law enforcement risk assessment tool used by public authorities to evaluate the safety and fundamental rights of domestic violence victims.
The system's primary function is to predict the risk of domestic violence recidivism, while also dynamically adjusting risk thresholds to allocate police resources.
The reports provide extensive, detailed information from independent audits, academic researchers, and official government responses, allowing for a highly reliable classification.
The failures could be interpreted purely as human/organizational failures of the police force rather than algorithmic failures, though the systemic bias and resource-dependent thresholds point directly to algorithmic design issues.
Technology
Resource-Constrained Risk Tuning
Algorithm limits extreme risk scores based on available police funding.
Opaque Algorithmic Formula
The exact formula and weights used to calculate risk remain undisclosed.
Rigid Binary Output System
The system relies on binary inputs and lacks flexibility for nuance.
Data Inputs
Bias Against Childless Women
Victims without children are systematically assigned lower risk scores.
Underestimation of Non-Physical Abuse
The system prioritizes physical violence over psychological or digital abuse.
Compromised Input Data Quality
Victims in shock or under duress provide incomplete or distorted answers.
Human Factors
Automation Bias in Police Officers
Officers blindly stick to the algorithm's risk score in 95% of cases.
Lack of Specialized Gender Training
Police officers administering the test lack training in gender-based violence.
Victim Cognitive Overload
Traumatized victims struggle to recall details under pressure from officers.
Process and Methods
Rigid and Inflexible Questionnaires
Fixed question formats do not allow victims to provide necessary explanations.
No End-User Engagement
Victims and women's organizations were never consulted during system design.
Exclusion of Aggressor Assessments
Risk is calculated without conducting any direct evaluation of the aggressor.
Management
Budget-Driven Risk Calibration
Management scales back protective risk scores to match police funding cuts.
Rejection of External Audits
The Ministry of Interior repeatedly denied independent external audit requests.
Decoupled Technical Evolution
The transition to machine learning lacks public debate and research oversight.
Regulatory Environment
Delayed Istanbul Convention Alignment
System forms were not updated to comply with the treaty until years later.
Unenforced Interdisciplinary Mandate
The 2004 law requiring psychologist and social worker teams was not enforced.
Lack of Independent Oversight
No public or independent entity has regular access to audit VioGen data.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
