This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
In October 2017, a Palestinian construction worker was arrested by Israeli police after Facebook's automatic translation software incorrectly translated his Arabic caption 'good morning' into 'attack them' in Hebrew and 'hurt them' in English. The police, who were monitoring social media for potential threats, interpreted the mistranslated text alongside a photo of the man leaning against a bulldozer as a sign of an impending vehicle attack. The man was detained for several hours before police realized the error and released him. Facebook apologized for the incident, citing the difficulty of translating informal Arabic dialects.
Facebook's automatic language translation software incorrectly translated an Arabic post saying "Good morning" into Hebrew saying "hurt them," leading to the arrest of a Palestinian man in Beitar Illit, Israel.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The translation system exhibited lower accuracy and robustness for Arabic dialects compared to other languages, disproportionately exposing Arabic speakers to risk.
Israeli police relied entirely on the automated translation output to make an arrest decision without verifying it with an Arabic-speaking officer.
The AI system failed to perform reliably under varying linguistic conditions, misinterpreting a common colloquial Arabic greeting as a violent threat.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was triggered by an unexpected translation error produced by Facebook's deployed AI translation system.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
A Palestinian worker was wrongfully arrested by Israeli police after Facebook's automatic translation software incorrectly translated 'good morning' into 'attack them' in Hebrew. The incident highlights the risks of overreliance on automated AI tools in law enforcement and intelligence monitoring without human-in-the-loop verification, resulting in a localized human rights violation.
The AI system acts as a translation tool, performing a specific, well-defined function on direct text input without context retention or autonomous planning.
Risk Level 2: High Risk. While translation is generally low risk, the system was utilized in a 'law enforcement application' to monitor social media and make arrest decisions, which has significant implications for safety and fundamental rights.
The primary function of the AI system was to automatically translate user posts across languages on the Facebook platform.
The reports are highly consistent across multiple reputable news outlets, detailing the exact sequence of events, the specific mistranslation, and the subsequent actions of the police and Facebook.
The incident could be viewed primarily as a failure of police procedure rather than an AI failure, but the AI's translation error was the direct trigger.
Technology
Neural Translation Flaws
Proprietary CNN/RNN translation model made a critical semantic error.
Lack of Uncertainty Outputs
System did not convey translation confidence or alternative meanings.
Dialect Translation Complexity
AI struggled with colloquial Arabic dialects vs standard Arabic.
Data Inputs
Orthographic Similarity
Only one letter difference between good morning and verb to hurt.
Informal Language Slang
Informal social media text is poorly mapped in translation training data.
Human Factors
Automation Bias by Police
Officers relied entirely on automated translation without verification.
Contextual Bias from Image
Posing with a bulldozer triggered suspicion due to historical attacks.
Process and Methods
No Pre-Arrest Human Review
No Arabic-speaking officer read the post before the arrest was made.
Flawed Predictive Policing
Police used social media monitoring algorithms to trigger direct arrests.
Management
Lack of AI Safety Guardrails
Facebook deployed automatic translation without real-world risk checks.
Inadequate Risk Assessment
Management failed to assess translation risks in high-conflict zones.
Regulatory Environment
Policymaker Knowledge Gaps
Policymakers adopt automated tools without understanding their limitations.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
