This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
AI capabilities have the potential to be exploited for personal gain at the expense of others via deception and manipulation. This can take various forms including cheating, fraud, scams, and the use of deepfakes for blackmail or humiliation. It is currently very difficult to distinguish human text from text that is AI-generated. This increases opportunities for cheating in settings where rewards depend on the communication of original thought. In academia, students may use AI to quickly generate essays or other coursework and claim it as their own. If students regularly and inappropriately rely on AI for their schooling, this could undermine academic integrity and genuine intellectual development. In science, researchers could use AI unscrupulously to produce professional outputs. If widely adopted, this practice could dilute the overall quality of scientific discourse.
Generative AI products may also be used to increase the reach and potency of various dishonest schemes. Advanced AI assistants can produce HTML, CSS, and other web development languages, allowing for the rapid creation of convincing fraudulent websites and applications at scale. In the context of social media, generative adversarial networks (GANs) have been used to create images of human faces that look authentic. AI models can also be trained on speech or writing data from a specific individual, allowing the model to impersonate someone very convincingly without consent. Scammers could use this capability to request sensitive information or financial aid by pretending to be a trusted contact. A particularly damaging type of abuse facilitated by deep fakes involves creating non-consensual sexual imagery with the intent to cause a subject social injury or manipulate them into performing desired actions. Even if a deep fake is exposed as inauthentic, it can continue to impact a person's life in significant ways through the loss of job opportunities, social isolation, and ongoing harassment or defamation.
Excerpt from the MIT AI Risk Repository full report
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Incident volume relative to governance coverage; each dot is one of 24 subdomains
Entity
Who or what caused the harm
Intent
Whether the harm was intentional or accidental
Timing
Whether the risk is pre- or post-deployment
Spain's People's Party (PP) circulated an AI-generated video on official social media channels that used deepfake technology to impersonate political rivals, including the prime minister's wife and other Socialist party figures. The video attributed false, defamatory statements to these individuals, prompting a legal complaint from a former party member who reported suffering from anxiety and social isolation as a result. While the party defended the content as political satire, the incident has sparked significant debate regarding the ethics and legality of using AI-generated disinformation in political campaigns.
Developers: Voice cloning technology developers, Synthetic video generation technology developers, Synthetic media technology developers, Synthetic audio generation technology developers, Deepfake technology developers
Deployers: Synthetic media creators, political organizations, People's Party (Spain), Partido Popular, Deepfake creators
Criminals in the Netherlands used AI-generated voice clips to impersonate bank officials in a sophisticated phone scam, defrauding approximately 170 victims of 380,000 euros.
Developers: Synthetic media technology developers, Synthetic audio generation technology developers
Deployers: Voice phishers, Synthetic media creators, Scammers, Deepfake creators, Cybercriminals in the Netherlands, Cybercriminals, Bank helpdesk fraud ring in the Netherlands
A man in Brazil was arrested after using AI software to create a fake photograph of himself holding a pistol. He sent this image to his former partner via WhatsApp to intimidate her and her father. Police searches confirmed no real firearm existed, and the suspect confessed to using AI to simulate the weapon for the purpose of domestic intimidation.
Developers: Synthetic media technology developers, Synthetic image generation technology developers
Deployers: Synthetic media creators, Men and boys, Men, Domestic abuse perpetrators
of the 272 AI experts surveyed named fraud, scams, and targeted manipulation one of the three AI risks they are most concerned about over the next five years (2025–2030).
average probability experts placed on harm from fraud, scams, and targeted manipulation reaching catastrophic levels (>1M deaths, >$100B loss, or civilization-scale damage) within five years, falling to 8.7% with pragmatic mitigations.
With pragmatic mitigations
Average severity falls from severe to substantial.
136 classified mentions from 42 companies.
Browse all 136 mentions →AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
235 shared governance docs
Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
227 shared governance docs
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
224 shared governance docs
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
213 shared governance docs
Requires operators to disclose AI interactions, especially to minors. Prohibits manipulative tactics, sexually explicit content, or misleading mental health claims. Mandates crisis intervention protocols. Subjects violations to penalties and limits enforcement to the attorney general or consumer protection director.
Directs the Committee on National Security Systems, Secretary of War, and Homeland Security to prioritize AI-enabled cyber defense. Establishes AI cybersecurity clearinghouse. Develops classified benchmarking for AI models. Enhances AI capacity to protect against illegal computer access and exploitation.
Requires operators of conversational AI services to disclose AI status to all users, apply added safeguards for minors, such as preventing sexually explicit content and simulated emotional dependence, and implement suicide and self-harm crisis-referral protocols. Prohibits presenting outputs as provided by or equivalent to licensed professionals, and requires annual reporting to the Attorney General on crisis referrals and related protocols.
