This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Internal Meta documents reveal that the company's automated ad-personalization and targeting systems have been used to facilitate billions of scam impressions daily, generating significant revenue. Despite internal awareness of the scale of fraud, Meta implemented 'revenue guardrails' that limited enforcement actions against profitable scam advertisers. The system's design, which targets ads based on user interests, inadvertently creates feedback loops that expose vulnerable users to repeated fraudulent content, leading to substantial financial losses for users globally.
Internal Meta documents reviewed by Reuters reportedly show that the company's automated ad and targeting systems generated major revenue from fraudulent ads and exposed users to billions of scam impressions daily. Enforcement was limited by internal revenue guardrails. In one example, a Royal Canadian Air Force recruiter's account was hacked and used to run a crypto scam that defrauded several colleagues after repeated reports allegedly went unanswered.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The platforms exposed users to toxic content including sextortion scams targeting teenagers and the sale of banned medical products.
Meta's automated content moderation and scam detection systems failed to robustly identify and filter out fraudulent advertisements.
The incident involves the widespread facilitation of financial scams, crypto fraud, and impersonation schemes targeting users on Meta's platforms, leveraged through automated systems.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk is driven by the post-deployment actions of Meta's automated ad-personalization system, which unintentionally creates feedback loops that amplify scam exposure for vulnerable users.
Added categories peak at 4: Severe (Epistemic Harm, Direct)
Meta's automated ad-personalization systems have systematically facilitated billions of scam impressions daily, creating severe economic and societal harms globally. While primarily a financial and societal threat, the compromise of military personnel accounts highlights potential vectors for targeted information operations.
The report describes an automated ad-personalization system that 'tries to deliver ads based on a user's interests' and automatically 'floods that user's feed with more scams' if they click on one, operating autonomously based on learned user patterns.
Unacceptable Risk: The system is described as an 'exploitative system targeting vulnerabilities' and 'AI used for subliminal manipulation or behavioral influence leading to harm,' specifically targeting vulnerable users who click on scams with feedback loops that flood their feeds with more fraudulent content.
The primary function of the system is to deliver and personalize advertisements to users based on their interests and behavioral data.
The reports are based on a highly detailed Reuters investigation of leaked internal Meta documents spanning several years, providing specific revenue figures, daily scam ad counts, and internal policy thresholds (e.g., the 95% certainty rule, 0.15% revenue guardrails). The details are consistent across both sources, and specific real-world victim cases are documented with names and locations.
Meta argues that the internal revenue estimates were rough and overly-inclusive, and that they have aggressively fought scams, suggesting the scale of deliberate negligence might be overstated.
Technology
Ad-Personalization Feedback Loop
System delivers more scam ads to users who previously clicked on them.
High Automated Ban Threshold
Automated systems only ban if 95 percent certain of fraud, letting scams pass.
Compute Diverted to AI Training
Safety teams compute resources restricted to prioritize AI and virtual reality.
Data Inputs
Privacy-Invasive Surveillance Data
Detailed user dossiers are sold to scammers for highly targeted pitches.
Ignored User Fraud Reports
Meta ignored or rejected 96 percent of valid user scam reports.
Permissive Policy Data Rules
Current policies failed to flag obvious scams like fake prime minister ads.
Human Factors
User Trust in Hacked Accounts
Victims lowered their guard when messaged by hacked trusted contacts.
Staff Burnout from Layoffs
Massive layoffs left remaining staff unable to handle high volume of scams.
Process and Methods
Revenue Guardrails on Enforcement
Anti-fraud teams limited to actions costing under 0.15 percent of revenue.
Penalty Bids instead of Bans
Charging suspected scammers higher rates rather than banning them.
High Value Account Strike Policy
Large advertisers allowed over 500 strikes before getting banned.
Management
Growth and AI Prioritization
Management prioritized massive AI investments over safety operations.
Reactive Regulatory Strategy
Leadership decided to act against fraud only in response to regulatory action.
Underinvestment in Safety Tech
Management classified scam ads as low severity, starving safety teams.
Regulatory Environment
Lax Advertising Regulations
Lack of regulatory oversight allows tech companies to profit from fraud.
Fines Lower than Scam Revenue
Anticipated 1 billion fine is dwarfed by 7 billion annual scam revenue.
Absence of Consumer Privacy Laws
Lack of modern privacy laws lets Meta spy on users without limit.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
