This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
On March 30, 2009, an automated license plate reader (ALPR) in San Francisco misread a license plate, incorrectly identifying a vehicle as stolen. Despite the system's known flaws and lack of clear verification policies, police officers performed a high-risk felony stop on an innocent driver, holding her at gunpoint and in handcuffs. The incident highlights the risks of relying on automated surveillance technology without adequate human verification procedures.
In San Francisco, an automated license plate reader (ALPR) camera misread a number as belonging to a stolen vehicle having the wrong make, but its photo was not visually confirmed by the police due to poor quality and allegedly despite multiple chances prior to making a traffic stop, causing an innocent person to be pulled over at gunpoint and restrained in handcuffed.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Police officers relied on the automated ALPR hit to execute a high-risk felony stop without performing the required visual verification of the license plate.
The ALPR system failed to perform reliably under low-light conditions, misreading a license plate character and generating a false positive that led to a high-risk police intervention.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was initiated by the ALPR system misidentifying a license plate character due to poor image quality, which was an unexpected failure during active deployment.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
A 2009 incident where a San Francisco police ALPR misread a license plate, leading to an innocent driver being detained at gunpoint. The incident highlights minor national security concerns regarding automation bias, privacy, and the lack of verification policies in automated law enforcement surveillance.
The ALPR system operates as a Tool, performing the specific, deterministic function of scanning license plates and matching them against a database without maintaining an independent state or planning goals.
High Risk: The system is used in law enforcement applications for automated surveillance and identification, which has significant implications for public safety and fundamental civil rights.
The primary function of the ALPR system is to recognize and read license plates from passing vehicles and match them against a database of wanted vehicles.
The report provides a highly detailed account of the incident, including specific officer actions, the technical failure of the ALPR, and the legal proceedings of the Ninth Circuit Court of Appeals.
The incident could be interpreted primarily as human error due to the officers' failure to follow training, but the AI system's false positive was the critical trigger for the chain of events.
Technology
ALPR Character Misread
The ALPR misread plate 5SOW350 as 5SOW750 due to system limitations.
Image Blur in Low Light
Late and dark conditions rendered the ALPR photograph blurry and illegible.
Data Inputs
Mismatched Database Record
ALPR matched the misread plate with a wanted stolen vehicle database entry.
Unverified Image Input
System processed a low-quality image without flagging it as low-confidence.
Human Factors
Failure to Visually Verify
Officer followed the car but failed to read the full plate to confirm.
Assumed Verification
Officers assumed another colleague had already verified the ALPR hit.
Incomplete Communication
Officer did not inform dispatch that the plate was unverified.
Process and Methods
No Clear Verification Policy
SFPD lacked a policy designating which officer must verify an ALPR hit.
Inadequate Guidelines
No clear procedures existed on how to handle unverified ALPR alerts.
Management
Unsupervised Tech Deployment
Deployed ALPR technology without establishing proper safety protocols.
Failure to Audit Errors
Management failed to address known system flaws and false hit rates.
Regulatory Environment
Absence of ALPR Regulations
Lack of external standards governing police use of ALPR systems.
Lack of Independent Oversight
No regulatory body monitored police compliance with ALPR safety rules.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
