Skip to main content
This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
MIT AI Risk Initiative
BackMisuse tactics that exploit GenAI capabilities (Use of generated content)
Counterfeit
Targeting & Personalisation

Misuse tactics that exploit GenAI capabilities (Use of generated content)

Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data

Marchal & Xu (2024)

SourceDOI
Category
Risk Domain
4Malicious Actors & Misuse
4.3Fraud, scams, and targeted manipulation

Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.

Entity: Who or what caused the harm

Human

Due to a decision or action made by humans

AI system

Due to a decision or action made by an AI system

Other

Due to some other reason or is ambiguous

Intent: Whether the harm was intentional or accidental

Intentional

Due to an expected outcome from pursuing a goal

Unintentional

Due to an unexpected outcome from pursuing a goal

Other

Without clearly specifying the intentionality

Timing: Whether the risk is pre- or post-deployment

Pre-deployment

Occurring before the AI is deployed

Post-deployment

Occurring after the AI model has been trained and deployed

Other

Without a clearly specified time of occurrence

Sub-categories (1)

Targeting & Personalisation

"Refine outputs to target individuals with tailored attacks"

4.3 Fraud, scams, and targeted manipulation
HumanIntentionalPost-deployment

Other risks from Marchal & Xu (2024) (22)

Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness)

4.3 Fraud, scams, and targeted manipulation
HumanIntentionalPost-deployment

Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) > Impersonation

4.3 Fraud, scams, and targeted manipulation
HumanIntentionalPost-deployment

Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) > Appropriated Likeness

4.3 Fraud, scams, and targeted manipulation
HumanIntentionalPost-deployment

Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) > Sockpuppeting

4.1 Disinformation, surveillance, and influence at scale
HumanIntentionalPost-deployment

Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) > Non-consensual intimate imagery (NCII)

4.3 Fraud, scams, and targeted manipulation
HumanIntentionalPost-deployment

Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) > Child sexual abuse material (CSAM)

4.3 Fraud, scams, and targeted manipulation
HumanIntentionalPost-deployment
View all 22 risks from this paper →