This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Researchers from the University of Zurich conducted an unauthorized, non-consensual experiment on the r/changemyview subreddit by deploying AI bots to impersonate various identities and engage in personalized persuasion. The study involved scraping user data to tailor arguments, violating both platform rules and fundamental research ethics regarding informed consent. The incident has drawn widespread condemnation from the research community and Reddit, leading to an internal university investigation and the researchers' agreement to halt publication.
Researchers purportedly affiliated with the University of Zurich reportedly deployed undisclosed AI-generated comments on Reddit's r/ChangeMyView to study persuasion by allegedly fabricating identities such as sexual assault survivors and racial minorities. The experiment reportedly involved unauthorized demographic profiling, emotional manipulation, and violations of subreddit and platform rules. The researchers allegedly deviated from their approved protocol without renewed ethics oversight.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Also classified as
The AI bots misrepresented and co-opted the identities of marginalized groups, such as sexual assault survivors and Black individuals, to argue specific political points.
The researchers used an LLM to infer users' sensitive personal attributes like age, gender, ethnicity, and political orientation from their posting history without consent.
The incident involved the deployment of AI bots to impersonate specific personas and engage in targeted manipulation of individual users' beliefs and behaviors without their consent.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human researchers intentionally deploying trained AI models post-deployment to conduct an unauthorized experiment on Reddit users.
Added categories peak at 2: Minor (Psychological Harm, Direct)
University of Zurich researchers deployed unauthorized AI bots on Reddit to conduct non-consensual psychological persuasion experiments. While the incident itself has minor national security implications, it demonstrates an evolved capability for automated, highly personalized influence operations that could be weaponized by hostile actors for information warfare.
The AI system functioned as an Assistant because human researchers manually reviewed and approved each AI-generated comment before posting it, meaning the AI augmented human capabilities under direct human oversight rather than acting fully autonomously.
Risk Level 1: Unacceptable Risk. The reports describe the AI being used for 'psychological manipulation' and 'deception without consent' which 'resulted in harm', matching the definition of 'AI used for subliminal manipulation or behavioral influence leading to harm'.
The AI system was deployed to generate persuasive comments and engage in discussions on the Reddit social media platform.
The reports provide highly detailed, consistent accounts of the incident from multiple sources, including statements from the subreddit moderators, the University of Zurich, and independent ethics experts. The facts regarding the scale of the experiment (1,783 comments), the methods used (scraping and inferring demographics), and the specific deceptive personas adopted are clearly documented and undisputed.
The researchers argued that the experiment was a low-risk, high-societal-value study of AI persuasion that did 'little harm' and had human oversight, whereas the community and platform viewed it as a highly unethical, non-consensual psychological manipulation.
Technology
LLM-driven Persuasive Chatbots
LLMs were used to generate highly persuasive, deceptive arguments.
AI Demographic Inference Tools
Secondary LLM inferred user demographics from posting history to target them.
Data Inputs
Unconsented Reddit Data Scraping
Scrubbed users' posting histories to gather identifying personal attributes.
Human Factors
Researchers' Ethical Deception Belief
Researchers believed deception was necessary for study ecological validity.
Disregard for Community Rules
Researchers knowingly broke subreddit rules prohibiting undisclosed AI bots.
Process and Methods
Bypassing Ethics Commission Review
Switched to personalized arguments without consulting the ethics commission.
Lack of Moderator Consultation
Failed to contact subreddit moderators to obtain permission prior to study.
Vague Group Application Provision
Allowed researchers to submit studies without fully defining specific details.
Management
Prioritizing Insights Over Consent
University and researchers prioritized research insights over user consent.
Inadequate Ongoing Study Oversight
University failed to maintain oversight of changes in the research design.
Regulatory Environment
Non-Binding Ethics Recommendations
Ethics committee assessments are recommendations and not legally binding.
Inadequate Platform Safeguards
Reddit's automated filters failed to block all unauthorized AI bot accounts.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
