This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A deepfake video featuring an AI-generated voice and manipulated footage of podcaster Joe Rogan was circulated on TikTok to fraudulently promote a supplement brand called 'Alpha Grind'. The video misled viewers by fabricating an endorsement that never occurred, leading to its removal by TikTok for violating policies against harmful misinformation. The incident highlights the growing ease with which AI tools can be used to create deceptive content for commercial fraud.
A purported deepfake video featuring podcast host Joe Rogan reportedly advertising to his listeners about a "libido-boosting" supplement was circulating on TikTok and other platforms before being removed by TikTok along with the account which posted it.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
The incident involved the unauthorized use of Joe Rogan's public likeness and cloned voice to deceive consumers into purchasing a product for financial gain, which constitutes commercial fraud.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The scam was caused by human actors who intentionally used deployed AI voice-cloning tools to create a deceptive advertisement for financial gain.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
An AI-generated deepfake advertisement featuring cloned voices of public figures was used on TikTok for commercial fraud. While demonstrating the evolving sophistication and accessibility of synthetic media tools, the incident represents minor national security impact as it was limited to commercial scamming rather than state-sponsored information warfare.
The AI system used was a voice-cloning tool that performed the specific, well-defined function of speech synthesis based on direct inputs provided by the human creator.
Limited Risk: The incident involves the creation of a deepfake video and voice clone, which falls under Risk Level 3 due to transparency obligations for synthetic media.
The AI system was primarily used to clone and generate Joe Rogan's voice to fabricate an endorsement, and secondarily to match video footage to the generated audio.
The reports consistently describe the same event: a deepfake ad of Joe Rogan promoting Alpha Grind on TikTok, which was confirmed as fake by his guest Andrew Huberman. The details regarding the platforms involved, the viral reach, and the removal of the content are clear and uncontradicted.
None
Technology
Accessible Voice Cloning Tools
Easy-to-use AI tools cloned celebrity voices with high accuracy in seconds.
Advanced Cheapfake Techniques
Altering mouth movements and splicing audio made fake videos highly convincing.
Data Inputs
Abundant Audio Training Data
Massive volumes of public podcast audio allowed precise voice model training.
Unrelated Video Source Material
Real interview footages were easily harvested to splice with synthetic audio.
Human Factors
User Susceptibility to Scams
Social media users struggled to distinguish the synthesized voice from reality.
Malicious Intent by Scammers
Bad actors deliberately leveraged AI to deceive buyers and boost product sales.
Process and Methods
Slow Content Moderation Systems
Platforms relied on manual reports rather than proactive automated AI detection.
Inadequate Ad Screening Processes
TikTok approved and promoted the scam video as an ad without verification.
Management
Prioritizing Ad Revenue
Platforms prioritized ad volume over rigorous safety checks of promoted content.
Inadequate Proactive Risk Plans
Platform management failed to prepare for rapid consumer-grade AI abuse.
Regulatory Environment
Slow Legislative Response
Federal regulations lack immediate protections against commercial deepfake scams.
Lack of Industry Standards
No universal detection system or standard exists across AI software developers.
AI system involved
An AI system was involved, but the reports do not identify it.
Identified by automated extraction; can differ from the alleged developers above.
