This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A media outlet known as 'The BL' operated a large-scale coordinated inauthentic behavior network on Facebook and Instagram, utilizing hundreds of fake accounts to amplify pro-Trump content. To enhance the credibility of these fake personas, the network employed AI-generated profile photos created by Generative Adversarial Networks (GANs). Facebook eventually removed the network, which included over 600 accounts and 150 groups, after identifying the deceptive practices and the use of AI-generated imagery to evade detection.
A large network of pages, groups, and fake accounts having GAN-generated face photos associated with The BL, a US-based media outlet, reportedly bypassed Facebook moderation systems to push "pro-Trump" narratives on its platform and Instagram.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim of manipulating political processes, public opinion, and behavior.
Also classified as
The network used hundreds of fake accounts to artificially amplify content, polluting social media feeds and eroding shared consensus reality for millions of users.
The incident involved a highly coordinated, large-scale influence campaign using AI-generated profile photos to manipulate public opinion and political processes.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The risk and subsequent platform manipulation were caused by the intentional actions of human operators at 'The BL' who deployed GAN-generated images to deceive users.
Added categories peak at 4: Severe (Epistemic Harm, Indirect)
In late 2019, a coordinated inauthentic behavior network linked to The BL used GAN-generated profile pictures to bypass platform detection and run a large-scale influence campaign targeting US users. Reaching 55 million followers, this marked the first major deployment of synthetic faces in information operations, highlighting the evolving national security challenge of AI-enabled disinformation.
The AI system (GAN) operated as a tool to perform a specific, well-defined function: generating realistic human faces based on direct inputs or training patterns, without maintaining an internal state, planning, or acting autonomously in the environment.
Limited Risk: The report describes an AI system used to generate synthetic profile photos (AI-generated content/deepfakes), which falls under the category of systems requiring transparency obligations so users are aware they are interacting with AI-generated content.
The GANs were specifically used to generate highly realistic, synthetic profile photos of non-existent people to masquerade as real Americans on social media.
The reports from Snopes, Facebook, Ars Technica, and Wired provide highly consistent and detailed accounts of the coordinated inauthentic behavior network, the specific role of GAN-generated profile photos, and the scale of the operation (55 million users reached, 900+ accounts/pages/groups removed). The technical details of the GAN-generated faces and their detection are well-documented.
None.
Technology
GAN-Generated Profile Pictures
GANs created highly realistic fake faces to evade reverse-image searches.
Automated Posting Systems
High-frequency automated posting bypassed manual rate-limiting controls.
Evasion of Automated Detection
Inauthentic accounts successfully bypassed Facebook's automated filters.
Data Inputs
Use of Synthesized Face Data
Neural networks trained on real human faces generated non-existent profiles.
Repurposed Foreign Profile Data
Existing Vietnamese accounts were altered with fake US demographic details.
Stock Photo Ingestion
Unsplash stock photos were ingested to masquerade fake administrators.
Human Factors
User Susceptibility to Fake Profiles
Real users engaged with fake profiles due to realistic GAN-generated faces.
Manual Coordination by Staff
Real BL staff manually set up groups and assigned fake accounts as admins.
Exploitation of Social Trust
Using names like 'Josephson Magnolia' to build trust with conservative users.
Process and Methods
Automated Account Creation
Rapid automated registration allowed quick replacement of banned fake accounts.
Circumvention of Link Bans
Cloning banned sites onto new domains bypassed Facebook's URL filters.
Group Admin Loophole Exploitation
Assigning real staff as fallback admins preserved groups after profile bans.
Management
Prioritizing Engagement over Safety
Facebook prioritized platform engagement over aggressive CIB mitigation.
Lack of Coordinated Investigation
Facebook failed to take holistic action against the parent media network.
Inadequate Proactive Monitoring
Platform relied heavily on third-party journalists to uncover fake networks.
Regulatory Environment
Lack of Platform Policy Enforcement
Facebook's terms of service on inauthentic behavior were poorly enforced.
Absence of Synthetic Media Laws
No regulatory framework penalized the deployment of GAN-generated personas.
Weak Penalties for Repeat Violations
The BL continued operating despite 1,929 historical ad policy violations.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
