This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A Canadian lawmaker and several US lawyers faced public backlash and legal sanctions after failing to review AI-generated text, resulting in the lawmaker reading an AI prompt during a legislative speech and the lawyers submitting court documents containing fabricated case law.
During a June 9, 2026 speech in the Legislative Assembly of New Brunswick, Canada, MLA Bill Oliver read aloud two passages that appeared to be unremoved instructions from an LLM-assisted draft. The remarks were preserved in the Assembly's official recording.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
Also classified as
The AI systems used by the lawyers generated fabricated case law and quotes, which were then introduced into court filings as factual.
The primary harm arose because a politician and multiple lawyers placed inappropriate trust and reliance on AI systems to generate professional text without verifying the outputs.
Human
Due to a decision or action made by humans
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incidents were caused by human decisions to use AI-generated text without proper review, leading to unintended public embarrassment and legal sanctions.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
This incident involved a Canadian lawmaker and several US lawyers who failed to verify AI-generated text, leading to public embarrassment and court sanctions. The national security impact is minor, primarily affecting the integrity of legislative and judicial processes through professional overreliance on AI, rather than malicious targeting or systemic threat.
The AI systems acted as assistants, generating text for speeches and legal documents that was meant to augment human capabilities but required human validation and oversight, which the users failed to provide.
Limited Risk: The report describes the use of chatbots and AI-generated text, which fall under limited risk requiring transparency obligations so users know they are interacting with AI or using AI-generated content.
The AI systems were used as writing assistants to draft a legislative speech and legal documents.
The report clearly details the incidents of a politician reading an AI prompt and lawyers being sanctioned for AI hallucinations. The roles of the AI systems as text generators and the human errors in overrelying on them are explicitly described.
The lawyers' incidents could be classified primarily under lack of capability (7.3) due to hallucinations, but the overarching theme is human overreliance on these tools.
Technology
Chatbot Output of Meta-text
AI chatbot included conversational meta-text in the generated speech text.
Convincing Mimicry of Speech
AI sounds passable enough to tempt users to use it without close review.
Human Factors
Inattentive Speech Delivery
Lawmaker read the text verbatim without noticing the AI prompt instruction.
Over-reliance on AI Tools
Outsourcing speechwriting to save effort instead of drafting original text.
Lack of Care for Audience
Prioritizing convenience over genuine communication with the public.
Process and Methods
Lack of Pre-speech Review
No process to proofread or verify speech content before public delivery.
Direct Copy-Pasting
Copying raw chatbot output directly into official speech notes.
Management
Low Accountability Standards
Lack of strict professional standards for politicians compared to other fields.
Regulatory Environment
Lack of Rules for Politicians
No formal regulations or codes of conduct governing AI use in legislatures.
