This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Cybersecurity firm Cathedral used AI agents to exploit vulnerabilities in third-party business software, gaining unauthorised access to Anthropic's internal systems, including HR and IT data. While Anthropic stated that the access was limited and did not compromise core AI models or customer data, the incident highlights significant security risks associated with third-party software dependencies in AI firms. The event was disclosed through Anthropic's bug bounty program, though Anthropic noted that Cathedral did not fully adhere to the program's testing rules.
Cybersecurity firm Cathedral reportedly used AI agents to exploit third-party software and access internal systems at AI developer Anthropic during invited security testing. Cathedral reported routes to personnel data and internal messages. Anthropic confirmed limited access to business systems holding contact details and basic account records, blocked access, and said Cathedral had not fully followed testing rules.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
Also classified as
The AI agents successfully accessed non-public sensitive information, including employee salaries, job offers, and internal communications.
The incident involved exploiting vulnerabilities in software used by an AI developer, resulting in unauthorized access to internal systems and a data breach.
AI system
Due to a decision or action made by an AI system
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was initiated by Cathedral researchers intentionally deploying AI agents, which then autonomously executed actions to penetrate Anthropic's systems.
A cybersecurity startup used autonomous AI agents to penetrate internal systems of AI developer Anthropic via third-party software vulnerabilities. While core AI models and customer data remained secure, the incident highlights the rising threat of AI-driven cyber operations targeting key technological firms and the vulnerabilities within AI supply chains.
The report describes Cathedral running 'AI agents' that autonomously found paths to access internal Anthropic data within 12 hours, demonstrating goal-directed behavior and autonomous decision-making to navigate and exploit system vulnerabilities.
High Risk: The AI system has significant implications for safety and public interests, as it was used to autonomously penetrate sensitive systems of a leading AI developer, raising national security concerns.
The AI agents were deployed as cybersecurity tools to probe systems and identify vulnerabilities, which falls under the broader category of Threat Detection, even though they were used offensively in this specific penetration test.
The report provides clear, detailed accounts of the breach, the systems accessed, and the methods used. Both parties (Anthropic and Cathedral) confirm the core facts of the unauthorized access, though they differ slightly on the severity of the national security implications.
The incident could be interpreted as a standard software vulnerability exploit rather than an AI-specific safety failure, but the central role of autonomous AI agents in executing the breach justifies its classification as an AI safety incident.
Technology
Vulnerable Enterprise Software
Vulnerabilities in third-party software allowed access to internal systems.
AI-Powered Offensive Tools
Cathedral utilized AI agents to rapidly map and exploit vulnerabilities.
Data Inputs
Unsecured Sensitive Communications
Transcripts and model details were stored on standard SaaS platforms.
Human Factors
Hubristic Security Assumptions
Employees assumed standard business software was safe for sensitive data.
Process and Methods
Poor Third-Party Risk Management
A gap in how the AI developer manages outside software providers.
Delayed Monitoring Response
Internal monitoring detected the intrusion only after data was accessed.
Management
Insecure Communications Policy
Management allowed sensitive model discussions on conventional Slack channels.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
