This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A 67-year-old Chinese farmer lost his entire 150-mu sesame crop, valued at 150,000 RMB, after following incorrect herbicide advice generated by an unnamed AI application.
A 67-year-old farmer in Chuzhou, Anhui, reportedly used an AI-generated weed-and-pest-control plan to spray about 150 mu (25 acres) of sesame by drone. By the next day, the crop had largely withered. Agricultural technicians said the recommended mixture included fomesafen, a herbicide unsuitable for blanket application to sesame. The specific AI product and model have not been publicly confirmed.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
The farmer developed excessive trust in the AI over a year of use, leading him to apply dangerous chemical recommendations without verifying them with agricultural experts.
The AI system failed to provide accurate and safe agricultural advice, recommending a lethal herbicide for sesame crops due to a lack of robust domain-specific knowledge.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The crop damage was caused by incorrect chemical recommendations generated by the AI system after it was deployed to the public, which was an unintended outcome of its operation.
Added categories peak at 2: Minor (Psychological Harm, Indirect)
A localized incident where a Chinese farmer suffered crop loss after executing incorrect herbicide advice from a commercial AI chatbot. The event represents a consumer product safety issue and user overreliance on unverified AI outputs, carrying negligible national security implications.
The AI system acted as an Assistant, providing agricultural recommendations and pest-control plans to the farmer. It did not take independent action in the physical world; rather, the human user (Wu) had to manually accept and execute the advice using a drone.
Limited Risk: The system is a general-purpose chatbot/AI assistant providing information. Under the EU AI Act, chatbots are classified as Limited Risk, requiring transparency obligations to ensure users are informed that they are interacting with an AI system.
The farmer used the AI app to ask specific questions about weed and pest control, and the AI generated answers and plans based on web-scraped information.
The reports from multiple sources are highly consistent regarding the facts of the incident, the specific chemicals recommended, the acreage destroyed, and the financial loss. The role of the AI is explicitly documented with chat logs.
None. The causal link between the AI's recommendation, the farmer's action, and the crop destruction is clear and verified by agricultural experts.
Technology
No Independent Knowledge Base
AI relied on web-scraping without verifying domain-specific safety.
Inappropriate Plan Recommendation
AI suggested full-field spraying instead of targeted application.
Lack of Domain Safety Guardrails
System lacked safety filters for hazardous chemical recommendations.
Data Inputs
Unverified Web Data Integration
AI synthesized crawled web data containing incorrect agricultural advice.
Lack of Input Verification
No validation layer to check chemical compatibility before output.
Human Factors
User Over-reliance on AI
Gradual trust over a year led the farmer to stop verifying suggestions.
No Verification with Experts
The farmer did not consult agricultural technicians before spraying.
Unnoticed Warning Label
The tiny warning text on the interface was missed by the elderly farmer.
Process and Methods
Inconspicuous Warning Design
Warning was in small font at the top, easily missed by users.
Lack of Domain-Specific Review
No expert review process for safety-critical agricultural recommendations.
Management
Inadequate Risk Control
AI developer failed to implement safety filters for chemical advice.
Lack of Audience Adaptation
Failed to design interface warnings suitable for elderly users.
Regulatory Environment
Weak Warning Prominence Standards
Lack of strict enforcement of prominent warnings for high-risk fields.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
