This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
Researchers conducted a systematic audit of Amazon's search and recommendation algorithms and discovered that they amplify vaccine misinformation. The study found that 10.47% of search results for vaccine-related queries promoted misinformative products, which were often ranked higher than debunking content. Additionally, the researchers identified a 'filter-bubble' effect where user interaction with misinformative products led to further recommendations of similar misinformation.
Evidence of the "filter-bubble effect" were found by vaccine-misinformation researchers in Amazon's recommendations, where its algorithms presented users who performed actions on misinformative products with more misinfomative products.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
The algorithms created personalized filter bubbles, progressively showing users more misinformation based on their history, contributing to the degradation of the information ecosystem.
Amazon's search and recommendation algorithms spread false and misleading vaccine information by ranking misinformative products highly and creating personalized filter bubbles.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The amplification of vaccine misinformation was driven by the post-deployment operations of Amazon's search and recommendation algorithms, which unintentionally prioritized engaging but misinformative content.
Added categories peak at 4: Severe (Epistemic Harm, Inferred)
An academic audit revealed that Amazon's search and recommendation algorithms unintentionally prioritized and amplified vaccine misinformation, creating filter bubbles for users. While this poses public health and information quality challenges, it represents a minor national security threat as it lacks kinetic impact, state-sponsored coordination, or direct threats to sovereign stability.
The AI system acts as an assistant by providing product recommendations and search suggestions to augment the user's shopping experience, though it does so by autonomously filtering and ranking content based on user history.
Minimal or No Risk: The system is an e-commerce search and recommendation algorithm, which generally poses minimal risk to users and does not fall under the high-risk or prohibited categories of the EU AI Act.
The primary function of the audited AI systems is to recommend products to users on homepages and product pages, and to allow users to search for content/products via search queries.
The reports are based on a peer-reviewed academic audit with clear methodology, quantitative findings (e.g., 10.47% misinformation rate), and explicit descriptions of the algorithmic mechanisms (ranking bias, personalization, filter bubbles). There is little ambiguity about how the system behaved or what the researchers found.
One could argue that the primary issue is the third-party sellers uploading misinformative products rather than the AI itself, but the audit specifically isolates the algorithmic amplification and ranking bias as key drivers of exposure.
Technology
Flawed Recommendation Algorithm
Algorithms prioritize engagement over credibility of health products.
Misinformation Ranking Bias
Search algorithms rank misinformative results higher than debunking ones.
Filter Bubble Perpetuation
Personalization traps users in a loop of increasing misinformation.
Data Inputs
Lack of Product Credibility Data
Algorithm inputs do not include trustworthiness or medical verification data.
User History Tracking
Algorithm uses real-world actions like clicks to build harmful history.
Unverified Product Metadata
Lack of system checks on health-related metadata of listed products.
Human Factors
User Trust in Search Engines
Users assume search results are vetted and credible sources of health info.
Anti-Vaccine Movement Engagement
Active user groups engage with misinformation, driving algorithmic metrics.
User Click Behavior
Users clicking on dubious items triggers recommendation contamination.
Process and Methods
Inadequate Algorithmic Audits
Absence of internal automated audits to detect health misinformation spread.
Weak Product Removal Processes
Relying on external media reports to identify and remove fake products.
No Credibility Filtering Methods
Information retrieval metrics do not evaluate medical accuracy.
Management
Engagement Prioritization
Management prioritizes sales metrics over information safety.
Lack of Health Product Regulation
Failure of Amazon leadership to regulate health-related listings.
Inadequate Risk Assessment
Management failed to assess risks of vaccine misinformation on platform.
Regulatory Environment
Lack of E-commerce Regulations
No laws hold online retailers to medical accuracy standards.
Inadequate Regulatory Pressure
Regulators fail to enforce strict policies on online marketplaces.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
