This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
The Galileu AI tool, used by Brazilian labor courts, successfully detected and blocked a prompt injection attack embedded in a legal petition. The malicious instructions attempted to force the AI to superficially contest the petition and ignore documents. The system alerted the judge, who performed a manual review, preventing any potential procedural harm.
Galileu, an AI tool used by Brazil's labor courts, reportedly detected hidden instructions embedded in an initial petition before the 3rd Labor Court of Parauapebas. The text allegedly told the AI to contest the petition superficially and not challenge documents. Galileu reportedly alerted the judge and blocked the hidden content from processing; the judge then reviewed the material before imposing any procedural consequences.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
Also classified as
The attacker attempted to manipulate the AI system to gain an unfair advantage in a judicial proceeding.
The incident involved a prompt injection attack, which is a security vulnerability exploit aimed at manipulating the AI system's behavior.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was introduced by an intentional human action (a prompt injection attack) targeting the AI system after its deployment.
A prompt injection attack targeting the Brazilian labor court AI tool Galileu was successfully blocked. The attack attempted to manipulate judicial outcomes by forcing the AI to ignore documents. It was mitigated by the AI's detection mechanism and subsequent manual review by the judge, representing a minor, successfully managed threat to government functions.
The Galileu tool acts as an Assistant by processing documents, identifying security anomalies, and alerting the judge, who retains full decision-making authority and performs the final review.
High Risk: The system is used in the administration of justice and democratic processes, specifically assisting a court in processing legal petitions, which has significant implications for fundamental rights.
The Galileu tool is designed to assist the Brazilian labor courts in processing and analyzing legal petitions, which falls under Judicial Decision Support. It also performed Threat Detection by identifying the prompt injection attack.
The report is clear, detailed, and comes from an official judicial source. It explicitly describes the technical nature of the attack (prompt injection), the AI's response, and the human oversight process, leaving little room for ambiguity.
None
Technology
Prompt Injection Attack
Malicious instructions hidden in petition to manipulate AI output.
Complexity of Attack Detection
Detecting prompt or data injection is not trivial and requires specialized tech.
Data Inputs
Untrusted Document Inputs
Petition contained hidden malicious instructions targeting the AI system.
Human Factors
Need for Human Oversight
Judges must review AI alerts and not rely solely on automated outputs.
Process and Methods
Lack of Input Filtering
Standard document ingestion processes failed to block hidden prompt injections.
Management
Adoption of Non-Institutional AI
Risk of using non-approved AI tools lacking specialized security controls.
Regulatory Environment
Need for Compliance Standards
Systems must align with CNJ Resolution 615/2025 and international frameworks.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
