This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Multiple state wildlife agencies have issued warnings after hunters were cited for violating regulations due to relying on inaccurate AI-generated search results. These AI tools incorrectly cited failed legislative bills as current law. Agencies emphasize that hunters are legally responsible for verifying information through official regulation booklets and websites.
A hunter in Idaho was reportedly cited for hunting a day early after relying on an AI-generated search result that misrepresented state regulations, pulling information from a failed proposal. Wildlife agencies in Idaho, Wyoming, Illinois, and Maine later warned of similar AI errors.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
Hunters relied on general-purpose search AI for critical regulatory compliance instead of verifying with official sources.
The AI system failed to perform reliably when parsing complex, multi-page regulatory documents, leading to errors and hallucinations.
The AI system generated false and misleading information by presenting failed legislative bills as active laws, leading a hunter to act on inaccurate beliefs and receive a citation.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by deployed AI search systems generating inaccurate and hallucinated regulatory information, which was an unintentional failure of the system's capabilities post-deployment.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
Inaccurate AI-generated search results led to minor hunting regulation violations and citations in several US states. The incident represents a minor consumer-level AI hallucination issue with negligible national security implications.
The AI systems function as search assistants designed to summarize web results and answer user queries. The report notes that these tools 'offer quick answers' but require users to 'verify these responses through official information sources,' indicating they augment human search capabilities rather than acting autonomously.
Risk Level 3: The systems described are search engine chatbots and AI-generated overviews. According to the definitions, 'AI systems that pose a moderate risk and require specific transparency obligations... such as notifying users when they are interacting with an AI system or when AI-generated content is being used' fall under Risk Level 3.
The primary function of Google Search AI Overviews and the Scout chatbot is to answer user questions regarding hunting regulations by searching and summarizing online content.
The reports from official state wildlife agencies and reputable news sources clearly document the specific failure mode (AI misinterpreting failed bills as law) and the resulting real-world harm (a hunter receiving a citation). There is no conflicting information, and the role of Google Search AI Overviews is explicitly discussed.
None. The cause-and-effect relationship between the AI's incorrect output and the hunter's regulatory violation is clear.
Technology
AI Hallucination and Source Confusion
AI pulled rules from a similar river in Arkansas instead of Idaho.
Inability to Parse Complex Rules
AI gets overwhelmed by complex, multi-page hunting regulation booklets.
Data Inputs
Inclusion of Failed Legislative Bills
AI scraped failed proposals from legislative sites and treated them as law.
Lack of Contextual Data Labeling
Failed bills and active laws were not distinct enough for the AI to discern.
Human Factors
Overreliance on AI Search Results
Users relied on quick AI summaries instead of verifying with official booklets.
Habitual Search Engine Usage
Hunters carried daily Google habits into complex hunting regulation searches.
Process and Methods
Lack of Direct Source Verification
Users bypassed checking the official Game and Fish agency website or booklets.
Ineffective AI Disclaimers
Disclaimers urging users to verify information were ignored or ineffective.
Management
Rapid AI Deployment Without Guardrails
Search engines deployed AI overviews before ensuring accuracy on legal matters.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
