This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
During Operation Midway Blitz, an immigration agent was recorded using ChatGPT to draft official use-of-force reports. A federal judge determined that these AI-generated narratives were inaccurate and conflicted with body-worn camera footage, raising significant concerns regarding the integrity of law enforcement reporting and the potential for AI to facilitate the creation of unreliable official documentation.
Body-worn camera footage released in litigation over Operation Midway Blitz reportedly showed that an immigration agent used ChatGPT to generate a long-form use-of-force narrative based on minimal inputs. A federal judge found that multiple AI-assisted reports conflicted with contemporaneous video evidence, undermining their accuracy and credibility and raising concerns about the use of generative AI in official law enforcement reporting.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The agent inappropriately relied on a general-purpose chatbot to draft critical, legally binding use-of-force reports, representing unsafe use in a high-stakes situation.
The AI system was used to generate inaccurate and misleading narratives for official law enforcement reports, which falsely identified peaceful citizens as professional agitators.
Human
Due to a decision or action made by humans
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was caused by the human agent's decision to use ChatGPT to draft official reports, leading to the unintentional outcome of generating inaccurate and discredited legal narratives.
A US immigration agent used ChatGPT to fabricate official use-of-force reports during a federal operation, falsely labeling peaceful citizens as agitators. While the incident exposes vulnerabilities in administrative integrity and civil rights protections, its national security impact remains minor as it was a localized event resolved through standard judicial oversight.
The agent used ChatGPT to 'compile a narrative for a report based on a brief sentence about an encounter and several images', which fits the definition of an Assistant that augments human capabilities but requires human oversight.
High Risk: The AI system was used in a law enforcement application ('To the extent that agents use ChatGPT to create their use of force reports...'), which is explicitly listed as a High Risk category under the EU AI Act.
The agent used ChatGPT to compile a narrative for an official use-of-force report based on a brief sentence and images, which is a writing assistant function.
The report provides clear, direct evidence from a federal judge's ruling and body-worn camera footage showing the agent using ChatGPT to draft a report. The role of the AI is explicitly described, and the consequences (inaccurate reports, loss of credibility) are well-documented.
None
Technology
AI Generation of Fabricated Facts
ChatGPT fabricated details like consensual encounters, making reports unreliable.
Model Hallucinations in Narratives
The LLM generated boilerplate narratives that did not match real-world events.
Data Inputs
Extremely Sparse Input Prompts
Agent provided only a brief sentence and images to generate a complex report.
Reliance on Generic Templates
The AI relied on a generic go-by template, leading to inaccurate narratives.
Human Factors
Overreliance on AI for Reporting
Agents used ChatGPT to automate drafting of critical use-of-force reports.
Lack of Review of AI Output
Agents failed to correct inaccuracies in the AI-generated narratives.
Process and Methods
Unregulated AI Use in Reporting
No formal process governed the use of AI tools for writing official reports.
Lack of Bodycam Verification
No procedure existed to verify AI-generated reports against bodycam footage.
Management
Inadequate Supervisory Oversight
Management failed to monitor agents using unauthorized AI tools on duty.
Tolerance of Inaccurate Reporting
Management accepted reports that directly contradicted bodycam evidence.
Regulatory Environment
Absence of AI Standards in Law
No clear legal frameworks restricted AI use in law enforcement documentation.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
