This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
In a Genoa bribery investigation, an automated speech-to-text software incorrectly transcribed a suspect's statement from 'licit financing' to 'illicit financing'. The error was discovered during a review of the audio recording, highlighting the risks of relying on unverified AI transcription in judicial settings. Experts suggest the error stems from probabilistic bias in the model's training data, which frequently associates the term 'financing' with 'illicit' conduct.
A reported AI transcription software error in a Genoa bribery investigation is alleged to have incorrectly recorded "illicit financing" instead of "licit financing," which could have significantly impacted the case. The mistake was discovered during a review and is an example of the risks of relying on AI in judicial settings.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The judicial system passively delegated transcription tasks to the software without establishing mandatory human review or verification protocols.
The speech-to-text software failed to perform robustly and accurately under real-world conditions, misinterpreting 'licit' as 'illicit' due to probabilistic bias.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by a transcription error generated by a deployed speech-to-text AI system, which was an unexpected and unintentional outcome of its operation.
Added categories peak at 2: Minor (Epistemic Harm, Indirect)
An automated speech-to-text tool used in an Italian court incorrectly transcribed 'licit financing' as 'illicit financing' due to probabilistic bias. The error compromised a judicial inquiry and led to false media reports before being corrected by defense lawyers. The incident highlights minor risks to sovereignty and civil rights through the unregulated adoption of AI in government functions.
The speech-to-text software acts as a tool that performs a specific, well-defined function of converting audio input into text without context retention or goal-directed planning.
High Risk: The AI system is used in the administration of justice (transcribing legal interrogations), which has significant implications for fundamental rights and public interests.
The primary function of the AI system is speech-to-text transcription, which falls under Voice Recognition. Its application in transcribing official court interrogations serves as Judicial Decision Support.
The report provides clear, detailed factual accounts of the transcription error, its consequences, and the underlying technical cause (probabilistic bias). The legal and systemic context in Italy is also well-explained.
None, the incident is clearly a technical failure of an AI transcription tool used in a judicial setting.
Technology
Probabilistic Word Prediction
Software predicts words based on probability rather than exact audio.
Acoustic Confusion
Difficulty distinguishing similar sounding words like leciti and illeciti.
Data Inputs
Biased Training Data
Training data from TV and news overrepresents illegal contexts.
Lack of Contextual Training
Dataset lacks specific judicial context needed for accuracy.
Human Factors
Passive Automation Bias
Users trust the software output blindly without critical verification.
Inattentive Clerking
The court clerk failed to summarize or note the statement manually.
Process and Methods
Lack of Human Verification
No mandatory human review process to verify automated transcriptions.
Flawed Procurement Process
Software purchased like standard IT equipment without safety evaluations.
Management
Budget and Staff Cuts
Staff reductions forced rapid adoption of AI tools to save costs.
Inadequate Risk Assessment
Management adopted AI tools without evaluating potential judicial risks.
Regulatory Environment
Absence of AI Regulations
No legal framework or standards govern the use of AI in courtrooms.
No Professional Protocols
Lack of official guidelines and certified registries for transcribers.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
