This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Voice actors were targeted in a harassment campaign where AI-generated voice clones were used to read out their private home addresses on Twitter. While some clips were attributed to ElevenLabs, the company stated that many of the specific clips in this campaign were not created using their software, suggesting a smear campaign or the use of alternative platforms. The incident highlights the risks of AI voice synthesis for individuals with public audio footprints.
Twitter users allegedly used ElevenLab's AI voice synthesis system to impersonate and dox voice actors.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The AI-generated voice clones exposed users and victims to highly toxic content, including hate speech, racist slurs, and homophobic slurs.
The incident involves the targeted harassment and humiliation of specific voice actors using synthetic voice clones to read out their private home addresses and offensive slurs.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The harm was directly caused by the intentional actions of human trolls who used AI voice cloning technology to harass and dox specific victims.
Added categories peak at 2: Minor (Psychological Harm, Direct)
An online harassment campaign used AI voice cloning to dox and abuse voice actors. While showcasing the evolving capabilities of synthetic media for targeted harassment, the incident remains a localized cybercrime issue with minor national security implications.
The AI system acted as a Tool, generating synthetic speech based on direct text inputs and voice samples provided by the users. The report describes the technology as 'text to audio software' used to create specific audio clips.
Limited Risk: The report describes an AI system used to generate deepfakes (voice clones), which falls under Limited Risk and requires specific transparency obligations to ensure users are informed that the content is AI-generated.
The primary function of the ElevenLabs platform is to generate synthetic speech and clone voices from audio samples.
The report from Motherboard provides detailed first-hand accounts from the victims, direct quotes from the affected voice actors, and statements from the AI developer ElevenLabs. The nature of the harassment, the platforms used, and the specific content of the audio clips are clearly documented.
The doxing clips could have been generated using open-source voice cloning tools rather than ElevenLabs' commercial platform.
Technology
Low Barrier to Voice Cloning
AI tools allowed easy replication of voices with minimal audio samples.
Lack of AI Platform Safeguards
Initial lack of voice verification allowed unauthorized cloning.
Data Inputs
Publicly Available Voice Data
Abundant online audio from podcasts and streams used to train models.
Unprotected Personal Addresses
Doxers easily obtained and input victims' home addresses.
Human Factors
Malicious Intent of Trolls
Online trolls actively sought to harass and provoke voice actors.
Public Backlash Against AI
Trolls targeted actors who spoke out against AI voice cloning.
Process and Methods
Slow Content Moderation
Twitter's support system failed to quickly remove doxing content.
Inadequate Verification Processes
Lack of validation that user owns the voice being cloned.
Management
Gutted Trust and Safety Teams
Twitter's reduced staff led to ineffective content moderation.
Delayed Safeguard Implementation
ElevenLabs introduced safety features only after abuse occurred.
Regulatory Environment
Lack of AI Voice Regulations
No clear legal framework protecting individuals' voice likeness.
Weak Enforcement of Doxing Laws
Difficulties in legally prosecuting anonymous online harassers.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
