This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
AI capabilities have the potential to be exploited for personal gain at the expense of others via deception and manipulation. This can take various forms including cheating, fraud, scams, and the use of deepfakes for blackmail or humiliation. It is currently very difficult to distinguish human text from text that is AI-generated. This increases opportunities for cheating in settings where rewards depend on the communication of original thought. In academia, students may use AI to quickly generate essays or other coursework and claim it as their own. If students regularly and inappropriately rely on AI for their schooling, this could undermine academic integrity and genuine intellectual development. In science, researchers could use AI unscrupulously to produce professional outputs. If widely adopted, this practice could dilute the overall quality of scientific discourse.
Generative AI products may also be used to increase the reach and potency of various dishonest schemes. Advanced AI assistants can produce HTML, CSS, and other web development languages, allowing for the rapid creation of convincing fraudulent websites and applications at scale. In the context of social media, generative adversarial networks (GANs) have been used to create images of human faces that look authentic. AI models can also be trained on speech or writing data from a specific individual, allowing the model to impersonate someone very convincingly without consent. Scammers could use this capability to request sensitive information or financial aid by pretending to be a trusted contact. A particularly damaging type of abuse facilitated by deep fakes involves creating non-consensual sexual imagery with the intent to cause a subject social injury or manipulate them into performing desired actions. Even if a deep fake is exposed as inauthentic, it can continue to impact a person's life in significant ways through the loss of job opportunities, social isolation, and ongoing harassment or defamation.
Excerpt from the MIT AI Risk Repository full report
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Incident volume relative to governance coverage; each dot is one of 24 subdomains
Entity
Who or what caused the harm
Intent
Whether the harm was intentional or accidental
Timing
Whether the risk is pre- or post-deployment
Scammers used AI to create deepfake videos of Italian TV journalist Safiria Leccese offering fraudulent personal loans, leading to financial losses for multiple victims who believed the fake content.
Developers: Deepfake Technology Developers, Voice Cloning Technology Developers
Deployers: Scammers
Scammers used AI voice cloning technology to impersonate a woman's daughter, claiming she was kidnapped by a Mexican drug cartel, and successfully defrauded the mother of $5,400 through wire transfers.
Developers: Deepfake Technology Developers, Voice Cloning Technology Developers
Deployers: Scammers, Fake Kidnapping Scammers
Two men, Cornelius Shannon and Arturo Hernandez, were arrested for using AI to create and distribute non-consensual deepfake pornography of celebrities and private individuals, with their content receiving millions of views online.
Developers: Deepfake Technology Developers
Deployers: Arturo Hernandez, Deepfake Creators
of the 272 AI experts surveyed named fraud, scams, and targeted manipulation one of the three AI risks they are most concerned about over the next five years (2025–2030).
average probability experts placed on harm from fraud, scams, and targeted manipulation reaching catastrophic levels (>1M deaths, >$100B loss, or civilization-scale damage) within five years, falling to 8.7% with pragmatic mitigations.
With pragmatic mitigations
Average severity falls from severe to substantial.
136 classified mentions from 42 companies.
Browse all 136 mentions →AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
199 shared governance docs
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
196 shared governance docs
Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
188 shared governance docs
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
187 shared governance docs
Directs the Committee on National Security Systems, Secretary of War, and Homeland Security to prioritize AI-enabled cyber defense. Establishes AI cybersecurity clearinghouse. Develops classified benchmarking for AI models. Enhances AI capacity to protect against illegal computer access and exploitation.
Requires the Secretary of Defense to develop a cybersecurity policy for AI/ML systems no later than 180 days after the act is passed. Develop a comprehensive review of the effectiveness of the AI/ML policies. Addresses potential security risks, implements methods to mitigate those risks, and establishes standard policy. Requires a comprehensive report of the threats and cybersecurity measures by August 31, 2026.
Regulates companion chatbots by requiring operators to disclose their artificial nature, implement protocols against suicide-related content, and apply heightened protections for minors. Mandates annual reporting to the Office of Suicide Prevention and subjects violators to civil liability.
