This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
A sophisticated phishing campaign targeted Meta Business and Instagram users by employing a fake tech support chatbot and deceptive 2FA setup guides to hijack business accounts.
A phishing campaign has reportedly been impersonating Meta support using a fake chatbot interface to hijack Instagram Business accounts. Victims received emails claiming ad violations and were directed to a fraudulent site mimicking Meta's support. There, a simulated chatbot instructed users to add the attacker’s Authenticator app as a secure login method, enabling account takeover. It remains unclear whether the chatbot used AI or was human-operated via a bot-like interface. See editor's note.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
The threat actors used a fake tech support chatbot and deceptive social engineering tactics to defraud users and hijack their Meta Business accounts for personal gain.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The incident was caused by human threat actors intentionally deploying a malicious phishing campaign and fake chatbot to hijack user accounts.
Added categories peak at 2: Minor (Epistemic Harm, Direct)
A sophisticated cybercrime campaign utilized a fake tech support chatbot and social engineering to hijack Meta Business and Instagram accounts. While representing an evolved cyber threat to individual businesses and commercial entities, the incident has negligible direct national security implications.
The fake chatbot operates as a tool deployed by threat actors to automate the collection of user credentials and screenshots through a structured conversation.
Limited Risk: The report describes a chatbot, which falls under Risk Level 3 due to transparency obligations, though its deployment here is entirely malicious.
The AI system is explicitly described as a 'fake tech support chatbot' designed to interact with victims and guide them through a fraudulent support process.
The reports provide a highly detailed, step-by-step analysis of the phishing campaign's mechanics, including the email templates, landing pages, chatbot dialogues, and 2FA hijacking methods. While the exact number of victims and the specific technical architecture of the chatbot are not detailed, the overall flow and intent of the campaign are extremely clear.
The chatbot could be interpreted as a simple non-AI automated script rather than an AI system, which would place this outside the scope of traditional AI incidents.
Technology
Fake Tech Support Chatbot
Malicious automated chatbot used to mimic official support and steal data.
Abuse of Salesforce Platform
Attackers leveraged legitimate Salesforce email systems to bypass spam filters.
Lookalike Domain Hosting
Fraudulent domain businesshelp-manager hosted to mimic Meta Business site.
Data Inputs
Malicious Screenshot Requests
Attackers collected account screenshots to screen and target high-value victims.
Phished User Credentials
Victims inputted sensitive login details and passwords into fraudulent forms.
Human Factors
Urgency and Fear of Suspension
Fear of losing advertising capabilities drove hasty, unsafe decision-making.
Lack of URL Verification
Users failed to verify domain names in browser address bars before acting.
Trust in Automated Chat
Interactive nature of the fake chatbot created a false sense of security.
Process and Methods
Flawed 2FA Setup Process
Users manually added attacker's authenticator app under 'System Check' guise.
Inadequate Email Verification
Lack of strict validation processes allowed spoofed sender addresses.
Management
Insufficient Security Training
Lack of employee training on social engineering and fake support chatbots.
Regulatory Environment
Weak Lookalike Domain Controls
Lack of strict regulations on registering domains mimicking major brands.
