This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
As AI systems become increasingly capable and intelligent, humans may be tempted to delegate many of their decisions and actions to AI. Although such delegation can be beneficial (e.g., by saving time or money), it may lead to undesirable outcomes where unconstrained or inappropriate. For example, if AIs take over tasks that typically require human creativity and analytical thinking, humans may engage less frequently in these cognitive processes. Over time, this may lead to a decrease in our ability to think critically and solve problems independently.
As individuals become more reliant on AI for everyday decisions – from what to eat and how to spend to more significant choices like career and relationships – there is a risk that they will lose their sense of free will and autonomy. If AIs begin to shape a person's life path in ways that do not align with their original aspirations and desires, this could limit their personal growth and prevent the pursuit of a fulfilling life.
At a societal level, organizations may hand over control to AI systems to stay competitive or reduce costs. If a significant number of organizations adopt AI systems and automate decision-making processes, especially in a way that is opaque and difficult to challenge, it could lead to widespread job displacement and a growing sense of helplessness among the general population.
Excerpt from the MIT AI Risk Repository full report
Delegating by humans of key decisions to AI systems, or AI systems that make decisions that diminish human control and autonomy, potentially leading to humans feeling disempowered, losing the ability to shape a fulfilling life trajectory, or becoming cognitively enfeebled.
Incident volume relative to governance coverage; each dot is one of 24 subdomains
Entity
Who or what caused the harm
Intent
Whether the harm was intentional or accidental
Timing
Whether the risk is pre- or post-deployment
An automated AI curation agent, 'dragon-ai-agent', eagerly picked up a GitHub issue labeled 'good first issue' and created a draft PR, depriving a new human contributor of a learning opportunity.
Developers: Anthropic
Deployers: Monarch Initiative, DisMech maintainers
Following the death of a delivery driver in a traffic accident, Glovo's automated management system sent an email to the deceased's family terminating his account for non-compliance. The incident sparked public outcry and labor strikes in Florence, with critics highlighting how the company's algorithmic management system incentivizes dangerous driving behaviors to meet delivery targets.
Developers: Glovo
Deployers: Glovo
Uber drivers in the UK and EU filed legal actions against the company for failing to provide access to personal data and transparency regarding automated decision-making and profiling systems used in the Uber Driver App. The litigation revealed that Uber used opaque algorithms to manage drivers, including performance tagging and automated account deactivations based on fraud allegations. A Dutch appeals court eventually ruled that Uber violated drivers' rights by failing to provide sufficient transparency and meaningful human review for these automated decisions.
Developers: Uber
Deployers: Uber
of the 272 AI experts surveyed named loss of human agency and autonomy one of the three AI risks they are most concerned about over the next five years (2025–2030).
average probability experts placed on harm from loss of human agency and autonomy reaching catastrophic levels (>1M deaths, >$100B loss, or civilization-scale damage) within five years, falling to 6.7% with pragmatic mitigations.
With pragmatic mitigations
Average severity falls from severe to substantial.
27 classified mentions from 7 companies.
Browse all 27 mentions →Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
212 shared governance docs
AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or cause loss of confidential intellectual property.
198 shared governance docs
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
198 shared governance docs
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
161 shared governance docs
Requires operators to disclose AI interactions, especially to minors. Prohibits manipulative tactics, sexually explicit content, or misleading mental health claims. Mandates crisis intervention protocols. Subjects violations to penalties and limits enforcement to the attorney general or consumer protection director.
Requires operators of conversational AI services to disclose AI status to all users, apply added safeguards for minors, such as preventing sexually explicit content and simulated emotional dependence, and implement suicide and self-harm crisis-referral protocols. Prohibits presenting outputs as provided by or equivalent to licensed professionals, and requires annual reporting to the Attorney General on crisis referrals and related protocols.
Regulates the use of automated decision-making technology (ADMT) in consequential decisions, requiring deployers to provide disclosures, facilitate consumer rights for correction and human review, and mandates record-keeping by developers. Grants the Colorado Attorney General exclusive enforcement authority, effective January 1, 2027.
