This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
ShotSpotter's audio detection system was used as forensic evidence in the trial of a man accused of attempting to shoot a police officer. The system's analysis, which was subject to human review and modification, was later deemed unreliable by a judge, leading to the reversal of the defendant's weapons possession conviction. The case highlights concerns regarding the use of proprietary, non-peer-reviewed AI tools as primary evidence in criminal proceedings.
ShotSpotter's "unreliable" audio was used as scientific evidence to accuse and convict a Black man of attempting to shoot Rochester's city police, whose conviction was later reversed by a county judge.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Prosecutors over-relied on ShotSpotter as primary scientific evidence in court, despite it being designed primarily as an investigative alert tool.
The proprietary nature of ShotSpotter's algorithms prevented the defense from auditing the system's reliability or servicing history.
The ShotSpotter system failed to reliably detect and classify gunshots, requiring subjective human modifications that made the forensic evidence scientifically unreliable for a criminal trial.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident arose post-deployment due to the unintentional failure of the ShotSpotter AI system's algorithms and forensic analysis process to provide reliable scientific evidence, leading to a wrongful conviction.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
The incident involved the use of unreliable ShotSpotter acoustic AI forensic evidence in a local criminal trial, leading to a temporary wrongful conviction that was later overturned. This highlights localized civil rights and judicial integrity concerns rather than a direct threat to national security.
The ShotSpotter system functions as a Tool. It is designed to perform the specific, well-defined function of detecting and locating loud acoustic impulses (gunshots) using sensors and algorithms, operating on direct audio inputs without maintaining an independent goal-directed state.
High Risk: The AI system is used in law enforcement and judicial contexts (specifically as forensic evidence in criminal trials), which has significant implications for safety, fundamental rights, and public interests under the EU AI Act's high-risk categories.
The primary function of the ShotSpotter system is explicitly to detect and locate gunshots using acoustic sensors, which also serves the broader goal of threat detection for law enforcement.
The reports provide detailed, consistent accounts of the specific legal case, the technical limitations of the ShotSpotter system (such as misclassifying helicopter noise and manual analyst overrides), and the judicial outcome (reversal of the conviction). The role of the AI system is explicitly documented across multiple sources.
One could view the primary failure as human analyst bias rather than AI system failure, though the system's initial misclassification and the reliance on its algorithmic output make it a socio-technical AI safety incident.
Technology
Algorithmic Misclassification
System initially classified gunshots as helicopter noise, failing to alert.
Proprietary Closed Algorithms
Proprietary algorithms prevented independent peer review and defense scrutiny.
Lack of Forensic Reliability
System was not sophisticated enough to serve as reliable scientific evidence.
Data Inputs
Short Audio Windows
Narrow eight-second audio windows limited context for defense analysis.
Lack of Ambient Noise Filtering
Sensors struggled to differentiate gunshots from other loud street noises.
Human Factors
Cognitive Bias in Analyst Review
Human analysts were influenced by police narratives when reclassifying audio.
Overreliance on Automated Alerts
Prosecutors over-relied on the tool as primary scientific evidence in court.
Process and Methods
Subjective Post-Hoc Reclassification
System outputs were repeatedly modified from three to five shots post-incident.
Inappropriate Evidentiary Use
Tool designed for investigative alerts was misused as primary trial evidence.
Lack of Standardized Testing
The system lacked rigorous, unbiased testing for courtroom use.
Management
Coziness with Prosecution
ShotSpotter altered data findings to align with customer and police requests.
Lack of Quality Control Oversight
Management allowed subjective human overrides without robust validation.
Regulatory Environment
Absence of Scientific Standards
No regulatory standards governed the admission of acoustic sensor data in court.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
