This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
An anonymous actor deployed a bot using the Philosopher AI (powered by GPT-3) to post content on the /r/AskReddit subreddit. The bot impersonated a human user for over a week, generating a high volume of posts that included conspiracy theories and sensitive discussions. The incident highlights the risks of AI-generated content being used for deceptive impersonation and manipulation on social media platforms.
Philosopher AI, a GPT-3-powered controversial text generator, was allegedly used by an anonymous actor on AskReddit subreddit, whose posts featured a mixture of harmless stories, conspiracy theories, and sensitive topic discussions.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The bot generated and spread false stories and conspiracy theories, misleading users on the platform.
Vulnerable users trusted and relied on the bot's fabricated advice regarding suicide, believing it was a real human sharing genuine personal experience.
The bot was used to impersonate a fake human individual on Reddit to deceptively manipulate users' beliefs and emotions through fabricated personal stories.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by an anonymous human actor who intentionally deployed the GPT-3 bot to impersonate a human and post on Reddit.
Added categories peak at 2: Minor (Psychological Harm, Direct)
An anonymous actor deployed a GPT-3-powered bot on Reddit that autonomously posted high-frequency content, including conspiracy theories and sensitive advice, successfully impersonating a human for a week. While demonstrating the evolving threat of AI-enabled information manipulation, the immediate national security impact remains minor.
The report describes a bot that was 'churning out a post per minute' and 'automatically generating posts' on Reddit without human intervention, responding to user queries and threads independently.
Risk Level 3: The incident involves a chatbot generating text and posing as a human on a social media platform, which falls under the category of AI systems requiring transparency obligations so users know they are interacting with AI.
The AI system was used to automatically generate posts and comments on the Reddit social media platform, acting as an automated user.
The report provides clear and specific details about the bot's behavior, the platform, the developer's response, and the specific posts generated. There is minimal ambiguity regarding the events.
None
Technology
Broken Bot Detection
Philosopher AI bot detection was broken, allowing automated scraping.
Convincing Text Generation
GPT-3 generated highly realistic text that easily mimicked human responses.
Signature Leakage
The AI outputted 'Philosopher. AI' text, revealing its automated nature.
Data Inputs
Sensitive Input Processing
The system processed highly sensitive prompts regarding suicide without blocks.
Human Factors
User Trust in AI Output
Reddit users accepted the bot's fake personal stories as genuine human advice.
Deceptive Persona Posing
The bot posed as a human, sharing fabricated personal struggles.
Process and Methods
Absent Rate Limiting
No rate limits prevented the bot from generating one post per minute.
Weak Platform Moderation
Reddit's systems failed to flag suspicious rapid posting behavior.
Management
Inadequate Risk Mitigation
The developer failed to secure the API against unauthorized automated use.
Regulatory Environment
Lack of AI Disclosure Rules
No regulations required AI text generators to label outputs as synthetic.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
