This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Facebook and Google's automated advertising systems were found to be generating and allowing advertisers to target users based on anti-Semitic and hate-speech categories. These categories were created by algorithms that processed user-provided profile data, such as 'fields of study' or 'interests'. The investigation by ProPublica demonstrated that these categories were active and could be used to purchase and successfully run advertisements, leading both companies to disable the specific functionality and review their automated processes.
Facebook's automated advertising categories generated using users' declared interests contained anti-Semitic categories such as "Jew hater" and "How to burn Jews" which were listed as fields of study.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
Also classified as
The automated systems enabled potential discrimination by allowing advertisers to target or exclude users based on protected characteristics like religion.
The algorithms lacked the capability or robust filtering mechanisms to distinguish benign academic topics from explicit hate speech.
The AI systems generated and exposed toxic, anti-Semitic hate-speech categories within the advertiser portals and allowed ads to be targeted at these groups.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk was caused by the automated ad-targeting algorithms of Facebook and Google generating hate-speech categories, which was an unexpected and unintentional outcome of their post-deployment operation.
Added categories peak at 2: Minor (Psychological Harm, Inferred)
An investigation revealed that Facebook and Google automated ad systems generated anti-Semitic targeting categories from user data without human oversight. While presenting potential vulnerabilities for information operations and discrimination, the incident was quickly mitigated by the platforms and did not escalate into a direct national security crisis.
The AI systems acted as assistants by automatically generating targeting categories and suggesting related keywords (such as suggesting 'Second Amendment' or 'Hitler did nothing wrong') to help advertisers build their campaigns, while still requiring the advertiser to make the final purchasing decision.
Minimal or No Risk: The system is an automated ad targeting and keyword suggestion algorithm, which generally falls under minimal risk as it does not belong to the prohibited or high-risk categories defined in the Act.
The primary function of the AI systems was to automatically group user data and deliver targeted advertisements to specific audiences based on those algorithmically generated categories.
The reports from ProPublica, The Guardian, and TechCrunch are highly detailed, consistent, and include direct confirmation and statements from Facebook and Google executives. The mechanics of how the algorithm generated the categories from user profile inputs are clearly explained.
None.
Technology
Algorithmic Category Generation
Algorithms converted user profile text into targetable ad categories.
Automated Recommendation Systems
System suggested related offensive terms to expand target audience size.
Lack of Automated Hate Speech Filters
No automated filters blocked offensive terms from becoming ad categories.
Data Inputs
User-Generated Profile Data
Users entered offensive terms in fields like education or employer.
Implicit User Activity Tracking
System derived interests from implicit user behavior without validation.
Human Factors
Lack of Human Oversight
No human review of automatically generated ad categories before deployment.
Advertiser Exploitation
Advertisers deliberately targeted hate-based groups for campaigns.
Process and Methods
Flawed Ad Review Process
Self-service system approved offensive ads automatically within 15 minutes.
Hands-off Ad Model
Business model prioritized automated scaling over manual content verification.
Management
Prioritizing Growth Over Safety
Management focused on scaling the ad platform without safety guardrails.
Inadequate Risk Assessment
Failed to anticipate risk of hate speech in automated targeting fields.
Regulatory Environment
Lack of External Oversight
No regulatory standards governed automated ad-targeting categories.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
