This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Security experts and Gmail users have reported a sophisticated phishing campaign where attackers use AI-generated voices to impersonate Google support staff. The scammers combine spoofed phone numbers, fake account recovery notifications, and deceptive emails to create a sense of urgency, aiming to trick users into approving account recovery requests that grant attackers access to their accounts. While some targets have successfully identified the ruse, the high level of realism in the AI-generated audio poses a significant risk to the 2.5 billion Gmail users.
Scammers allegedly used a purportedly AI-generated voice to impersonate a Google representative in an attempt to steal Gmail account credentials from security expert Sam Mitrovic. The reportedly AI-driven phishing call used a spoofed Google phone number and a fabricated email, making the scam appear legitimate. Mitrovic reportedly noted that the caller's professional demeanor, coupled with AI-generated speech and a Google-related number, could easily deceive unsuspecting users.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The ultimate objective of the AI-driven phishing scam is account takeover, which directly threatens the privacy of sensitive personal data stored in Gmail accounts.
The incident involves cybercriminals using AI-generated voices to execute highly realistic impersonation scams targeting specific individuals for personal and financial gain.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The phishing campaign and the deployment of the AI voice bot were intentionally orchestrated by human cybercriminals to defraud users.
Added categories peak at 2: Minor (Psychological Harm, Inferred)
Cybercriminals executed a sophisticated phishing campaign using highly realistic AI-generated voices to impersonate Google support staff, aiming to compromise Gmail accounts. While the national security impact is minor, the incident highlights how AI voice-cloning advances the scale and credibility of social engineering attacks against high-profile individuals and the broader public.
The AI system is a voice generation model used by scammers to generate realistic speech during phone calls. It performs the specific function of converting text to speech or playing pre-recorded AI voice segments without autonomous planning or goal-directed behavior of its own.
Limited Risk: The report describes the use of AI-generated content such as deepfakes and synthetic voices, which falls under the transparency obligations of Risk Level 3.
The AI system's primary function in the scam is generating highly realistic human-like voices to conduct interactive conversations with targets.
The reports provide consistent, detailed, and first-hand accounts from cybersecurity experts (Sam Mitrovic and Garry Tan) regarding the mechanics of the AI-driven scam. The technical details of the spoofing and voice generation are well-documented across multiple reputable sources.
None. The incident is clearly a malicious phishing campaign leveraging synthetic voice technology.
Technology
Realistic AI voice generation
Generative AI voice models simulate perfect human pronunciation and spacing.
Scalable automated calling
AI voice bots allow simultaneous execution of thousands of scam calls.
Email and phone number spoofing
Attackers mask true identities by spoofing Google domains and phone numbers.
Data Inputs
Abuse of CRM sender fields
Salesforce CRM allowed custom sender addresses to bypass validation.
Abuse of Google Forms data
Google Forms response receipts sent via Google servers added legitimacy.
Fake recovery request triggers
Maliciously triggered account recovery notifications initiated the workflow.
Human Factors
Susceptibility to panic
Scammers engineered panic through false claims of compromised accounts.
Cognitive overload from busy users
Busy individuals are more likely to miss subtle red flags during calls.
Trust in official caller ID
Users trusted caller IDs matching legitimate Google business numbers.
Process and Methods
Inadequate recovery verification
Recovery processes lacked robust checks on device name fields.
Permissive CRM email routing
Salesforce allowed setting unverified sender addresses through Google servers.
Lack of real-time signal sharing
Sectors historically lacked a shared database for real-time scam signals.
Management
Lack of default passkey enforcement
Standard security settings relied on phishable recovery options by default.
Delayed security integration
Collaborative scam signal exchanges were only recently initiated.
Lack of human-based support
Lack of human support channels makes it hard for users to verify calls.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
