This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Following a fatal shooting in Minneapolis, users prompted xAI's Grok chatbot to 'unmask' an ICE agent from eyewitness footage. Grok generated fabricated facial images that were circulated online alongside a false name, 'Steve Grove.' This led to a coordinated disinformation campaign and targeted harassment against two innocent individuals who shared the name, including a gun shop owner and a newspaper publisher.
After the fatal shooting of Renee Nicole Good in Minneapolis by an ICE officer, users on X reportedly asked Grok to "unmask" the masked agent shown in eyewitness footage. Grok reportedly generated a fabricated face that spread widely online, along with the false name "Steve Grove." The output and claim allegedly led to harassment and reputational harm against at least two uninvolved men.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The system failed to perform reliably by hallucinating facial details and generating a false identity when prompted to unmask an individual.
The AI system generated a fabricated image and associated it with a false name, leading to inaccurate beliefs in users who then harassed innocent individuals.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was caused by Grok's algorithmic hallucination of facial details and a false name, which was an unexpected and unintentional outcome of the system's deployment.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
An AI chatbot generated a fabricated image and false identity in response to user prompts seeking to unmask an ICE agent, leading to the targeted online harassment of two innocent citizens. While causing localized distress, the incident does not present significant national security implications or state-sponsored coordination.
The report describes Grok as a 'generative AI chatbot' that responded to users asking the bot to 'unmask' the agent. It acted as an assistant by generating content based on user prompts.
Risk Level 3. Limited Risk: AI systems that pose a moderate risk and require specific transparency obligations. Grok is described as a 'generative AI chatbot' and the incident involved an 'AI-generated image'.
Grok is a generative AI chatbot that was prompted by users to generate an image unmasking an individual, effectively creating a deepfake/fabricated image.
The report clearly details the AI system (Grok), the developer (xAI), the specific failure (hallucinating facial details and generating a false name), and the direct consequences on the affected individuals (harassment of the two Steve Groves).
None
Technology
Facial Detail Hallucination
AI enhancement hallucinates details devoid of biometric reality.
Lack of Image Generation Safeguards
Grok allowed users to generate unmasked faces of real-world individuals.
Data Inputs
Incomplete Visual Evidence
Eyewitness videos contained masked faces, requiring AI to guess details.
Unverified Name Association
The name 'Steve Grove' was falsely associated with the generated image.
Human Factors
Malicious User Prompting
Users actively prompted the AI bot to 'unmask' a masked law enforcement agent.
Rapid Public Dissemination
Social media users shared the unverified image and false name rapidly.
Targeted Online Harassment
Angry users attacked innocent individuals sharing the name Steve Grove.
Process and Methods
Insufficient Verification Process
Lack of verification before publishing or sharing AI-generated evidence.
Unregulated Chatbot Generation
No process to prevent Grok from executing harmful 'unmasking' requests.
Management
Inadequate Risk Assessment
Platform management failed to restrict dangerous image manipulation features.
Regulatory Environment
Lack of Generative AI Standards
Absence of regulations on AI-generated depictions of real people.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
