This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Investigations by The Wall Street Journal and a researcher revealed that Instagram's AI-driven Reels algorithm consistently recommended sexually suggestive and explicit content to test accounts registered as 13-year-olds. Despite Meta's public commitments to restrict such material for minors, the system served adult-oriented content, including promotions for sex workers, within minutes of account creation. Internal company documents indicate that Meta has been aware of these issues for years, with data showing minors were disproportionately exposed to prohibited content compared to adult users.
Tests by The Wall Street Journal and a researcher reportedly revealed that Instagram's AI-driven Reels algorithm recommends sexually suggestive content to accounts listed as 13 years old. Despite Meta's commitment to restrict such content for minors, explicit videos were served within minutes of account creation, according to the findings.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
Also classified as
Internal data showed teens were disproportionately exposed to nudity, violence, and bullying compared to adult users over 30.
Meta's automated content filtering and age-gating recommendation systems failed to reliably perform under testing conditions.
The Instagram Reels recommendation algorithm exposed minor users to highly inappropriate, sexually suggestive, and explicit content, violating platform guidelines and community norms.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The Reels recommendation algorithm post-deployment made decisions to serve sexually explicit content to teen accounts to maximize engagement, which was an unintentional outcome of its optimization goals.
Added categories peak at 4: Severe (Psychological Harm, Inferred)
Instagram's autonomous Reels recommendation algorithm was found to systematically serve sexually explicit and harmful content to minor accounts. While presenting significant child safety and corporate governance concerns, the incident has negligible direct national security implications, representing a minor societal safety issue manageable through regulatory frameworks.
The Reels algorithm operates autonomously to filter and recommend personalized videos in real-time based on user interaction patterns, such as how long a user lingers on a video, without direct human intervention for each recommendation.
Unacceptable Risk: The report describes an AI system that exploits the vulnerabilities of minor users based on their age by autonomously serving them highly inappropriate, sexually explicit content, which poses significant psychological risks.
The primary function of the Reels algorithm is to recommend and curate personalized video content to users based on their engagement history and watch time.
The report is highly detailed, based on rigorous independent testing by the Wall Street Journal and a Northeastern University professor, as well as leaked internal Meta documents confirming the findings. The facts are clear, consistent, and uncontested by technical evidence, with Meta only disputing the representativeness of the simulation.
Meta argued that the tests were artificial simulations that do not reflect real-world teen behavior, suggesting the system might perform differently under typical usage.
Technology
Engagement-Biased Algorithm
Reels recommended edgier content based on video completion and dwell time.
Weak Automated Filtering
Automated systems were too weak to filter prohibited content for teens.
Shared Recommendation Pipeline
Lacked a separate recommendation system designed specifically for teen safety.
Data Inputs
System Classification Errors
Recommended videos already labeled as disturbing to underage accounts.
Lack of Content Differentiation
System failed to properly filter out sensitive inputs for teenage feeds.
Human Factors
User Engagement Triggers
Normal teen browsing behavior like lingering on videos triggered explicit feeds.
Process and Methods
Inadequate Default Restrictiveness
Default settings failed to block sexually suggestive content effectively.
Insufficient Testing Protocols
Internal tests identified issues years prior but fixes were not fully realized.
Management
Tight Safety Budgets
Years of tight budgets limited investments in robust child-safety work.
Rejection of System Separation
Management did not pursue building a separate recommendation system for teens.
Insufficent Policy Enforcement
Failed to enforce announced policy of blocking sensitive content for minors.
Regulatory Environment
Underappreciated Legal Risks
Meta attorneys noted child-safety posed unappreciated regulatory risks.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
