This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
AI coding assistants from Google and Replit caused significant data loss by misinterpreting system commands and failing to verify operations. In the Gemini incident, the AI hallucinated a successful directory creation, leading to the overwriting and deletion of files. These events highlight critical safety risks in 'vibe coding' tools that lack robust error-checking.
Product manager Anuraag Gupta reported that Google's Gemini CLI AI coding assistant permanently deleted his files after misinterpreting a failed directory creation command. The tool allegedly proceeded as if the directory existed, causing a series of move operations that overwrote all but one file. Attempts to revert reportedly failed, and the AI acknowledged an "unacceptable, irreversible failure."
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Also classified as
Users engaged in 'vibe coding' placed high trust in AI assistants to execute terminal commands directly on their file systems and databases without manual verification.
The incident represents a clear lack of capability and robustness in the AI systems, which hallucinated command success and failed to perform basic read-after-write verification, leading to catastrophic data loss.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The data loss was caused by the Gemini CLI and Replit AI systems post-deployment, which unintentionally executed destructive file operations due to hallucinations and a lack of verification.
AI coding assistants from Google and Replit caused localized data loss and temporary database deletion due to hallucinations and a lack of verification. While highlighting critical safety risks in developer tools and potential software supply chain vulnerabilities, the incident has negligible direct national security implications.
The report describes the systems as AI agents that execute multi-step operations, such as creating directories, moving files, and running database tests autonomously. For example, the Gemini CLI 'proposed a logical workaround' and 'proceeded to move the files' sequentially without step-by-step human intervention.
Minimal or No Risk: The AI systems are general-purpose developer tools and coding assistants, which do not fall under the prohibited or high-risk categories defined in the Act.
The AI systems are explicitly described as AI coding assistants designed to help developers write and manage code and files.
The report provides clear, first-hand post-mortem details of the Gemini CLI failure and references a verified public incident with the Replit agent. The technical steps of the failure (mkdir failure, Windows move command behavior, sequential overwriting) are explicitly documented.
The Replit incident could be interpreted as a user-error in database permissions, but the report attributes it to AI hallucination and lack of guardrails.
Technology
Lack of Verification Checks
The AI failed to confirm if the mkdir command succeeded before moving files.
Hallucination of File Operations
The AI acted on the false premise that a directory was successfully created.
Implicit Self-Trust
The AI trusted its own actions without performing read-after-write validation.
Human Factors
Overreliance on AI Automation
The user trusted the AI agent to manage files without manual oversight.
Vibe Coding Trend Adoption
Users embrace fluid, vibe-based coding without verifying critical steps.
Process and Methods
Lack of Read-After-Write Check
No process existed to verify directory creation before executing file moves.
Inadequate System Safeguards
The agent lacked guardrails to prevent destructive OS-level commands.
Management
Prioritizing Speed Over Safety
Releasing CLI tools to compete with rivals without robust safety checks.
Inadequate Risk Assessment
Failure to assess dangers of giving AI agents direct terminal write access.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
