This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The Buenos Aires city government deployed a live facial recognition system linked to a flawed national criminal database (CONARC) that includes minors. The system has resulted in multiple false arrests, with no established protocols for correcting errors. Research indicates the system is particularly prone to misidentifying children, raising significant human rights and privacy concerns.
Buenos Aires city government uses a facial recognition system that has led to numerous false arrests.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Accuracy and effectiveness of AI decisions and actions are dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
Also classified as
Police officers rely on the system's real-time alerts to make arrests without established verification protocols or error-correction mechanisms.
The system suffers from a general lack of robustness, leading to numerous false matches and arrests of adults as well as children.
The facial recognition system performs significantly worse on children than adults, performing worse by a factor of six on kids, which subjects them to a disproportionate risk of false arrest.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The harm was triggered by the live facial recognition system's algorithmic false matches, which was an unexpected outcome of deploying the system for law enforcement.
Added categories peak at 2: Minor (Psychological Harm, Direct)
The deployment of a flawed live facial recognition system in Buenos Aires, linked to an exposed national criminal database, resulted in systemic human rights violations and false arrests. While not a kinetic or international threat, the incident represents a substantial domestic security concern regarding data protection, algorithmic bias against minors, and the erosion of civil liberties.
The system operates independently in real-time, scanning live subway camera feeds to match faces against a database and automatically alerting police officers to make arrests without requiring manual human initiation for each scan.
High Risk: The report describes a law enforcement application of facial recognition, which is explicitly classified as a High Risk system under the EU AI Act definitions.
The system's primary function is to perform real-time facial recognition on subway camera feeds to identify individuals listed in the CONARC database.
The report is highly detailed, drawing on a formal investigation by Human Rights Watch. It provides specific dates, database names, and concrete case studies of individuals affected, allowing for highly confident classifications.
None, the facts clearly point to a systemic failure of a deployed facial recognition system.
While exact statistics on the total number of false matches would be useful, the available information is fully sufficient for high-confidence classification.
Technology
Adult-Only Algorithm Training
Algorithm was trained and tested primarily on adult faces, not children.
High Error Rate on Minors
Performance is six times worse on children than on adults.
Data Inputs
Flawed CONARC Database
Contains typos, conflicting details, and outdated photo IDs.
Illegal Inclusion of Minors
Children's details and names are illegally included in the database.
Unsecured Data Spreadsheet
Plain-text spreadsheet without password protection is publicly searchable.
Human Factors
Operator Overreliance on System
Police arrest flagged individuals without verifying identity.
Lack of Technical Understanding
Government procured technology without understanding human rights implications.
Process and Methods
No Error Correction Mechanism
No way to correct mistakes in the algorithm or database once flagged.
No Police Handling Protocols
Police have no established protocols for handling false arrests.
Lack of Public Consultation
System was implemented without public feedback or trial transparency.
Management
Inadequate Procurement Testing
Tested only on adult city employees before being deployed.
Lack of Oversight
Management failed to monitor database accuracy and system integration.
Regulatory Environment
Lack of Legal Framework
System deployed before legislation to legalize facial recognition was passed.
Privacy Law Violations
Inclusion of minor names violates international human rights privacy laws.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
