This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A Guardian Australia investigation revealed that the Nuance-powered voiceprint biometric authentication system used by Centrelink and the ATO is vulnerable to bypass via AI-generated voice cloning. A journalist successfully accessed their own account using a clone created from four minutes of audio. This highlights significant security risks for biometric authentication systems relying on voice patterns.
A Guardian journalist was able to verify their identity and gain access to their own Centrelink self-service account using AI-generated audio of their own voice along with their customer reference number, shortly after voiceprint was deployed for ID verification.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
Also classified as
The voiceprint system lacks the technical robustness to reliably distinguish between genuine human voices and high-quality synthetic voice clones.
The incident highlights a critical security vulnerability in a deployed biometric AI system that can be exploited using AI-generated voice clones to gain unauthorized access to sensitive accounts.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The security vulnerability is an unintentional capability flaw in the deployed Nuance voiceprint AI system, which fails to distinguish between real and synthetic cloned voices.
A journalist demonstrated that the Australian government voiceprint biometric system used by ATO and Centrelink can be bypassed using an AI-generated voice clone. While no hostile exploitation was reported, the vulnerability exposes over 10 million citizen accounts, highlighting a significant evolutionary threat to biometric security and government service integrity.
The Nuance Voiceprint system acts as a tool that performs a specific, well-defined function of identity verification based on direct voice inputs, without maintaining independent planning or autonomous goal-directed behavior.
Risk Level 2. High Risk: The system is used for biometric identification and verification of individuals, which has significant implications for safety, fundamental rights, and public interests under the EU AI Act.
The primary goal of the Nuance system is to verify and authenticate the identity of users using their unique voice characteristics.
The report provides clear, first-hand investigative evidence of the vulnerability being successfully tested by a journalist, along with official statistics on user adoption and statements from relevant government agencies.
None
Technology
Vulnerability to Voice Cloning
Biometric voiceprint systems fail to distinguish AI-cloned voices.
Advanced Voice Cloning Tools
AI allows high-fidelity voice cloning with very little training data.
Imperfect Synthetic Voice Detection
Detection algorithms fail to flag cloned voices in up to 14% of cases.
Data Inputs
Exposed Customer Reference Numbers
CRNs are printed on correspondence and not treated securely like passwords.
Publicly Available Voice Data
Social media and public recordings provide audio to train voice-cloning AI.
Short Audio Sample Requirement
AI models require only four minutes of audio to generate a realistic clone.
Human Factors
Overconfidence in Biometrics
Agencies falsely assumed voiceprints were secure and could detect clones.
Process and Methods
Single-Factor Biometric Auth
Using voiceprint in lieu of passwords without multi-factor verification.
Automated IVR Vulnerability
Automated systems lack human interaction to detect synthetic voice delays.
Management
Inadequate Risk Assessment
Failure to proactively update systems against emerging AI voice spoofing.
Overreliance on Vendor Claims
Trusting vendor claims of security without independent validation.
Regulatory Environment
Absence of Clear Legislation
Lack of legal frameworks governing government use of biometric verification.
Lack of Standardized Guardrails
No basic standards to ensure technology is safe from AI cybercrime.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
