This is a research prototype. The data and analyses are preliminary and not yet validated — we'd welcome your .
BNN Breaking, a news outlet, utilized generative AI to mass-produce low-quality, often fabricated news stories to drive advertising revenue. This resulted in significant harm, including the publication of a defamatory story about an Irish broadcaster and the unauthorized use of journalists' bylines. The site's content was promoted by MSN, leading to widespread misinformation before the site eventually went dormant.
BNN Breaking, an AI-driven news site, published a false story about Irish DJ Dave Fanning, damaging his reputation. The site used AI to generate error-filled content, leading to numerous complaints and a defamation lawsuit against BNN and Microsoft. The site has since gone dormant.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Also classified as
BNN used AI to churn out thousands of unverified, low-quality articles daily, polluting search engines and major platforms like MSN.
The AI tool was used to scrape and paraphrase human journalists' work, siphoning ad revenue away from legitimate local news outlets.
The AI system generated a false and defamatory news article that misidentified a prominent broadcaster, leading to significant reputational harm and a lawsuit.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
The immediate cause of the defamation was an AI association error during automated content generation, which was an unintentional outcome of BNN's deployment of the tool.
Added categories peak at 3: Substantial (Epistemic Harm, Inferred)
A commercial news site, BNN Breaking, used generative AI to mass-produce fabricated news stories for ad revenue, resulting in defamation and information pollution. While representing a significant evolution in AI-enabled misinformation, the incident has minor national security implications as it lacks state-sponsored coordination or targeting of critical national infrastructure.
The AI system functioned as an Assistant designed to augment content creation by paraphrasing articles, though it was deployed unsafely without the required human validation.
Risk Level 3: Limited Risk. The system is a generative AI tool used to produce news articles, which falls under the category of AI-generated content requiring transparency obligations so that users are aware the content is AI-generated.
The primary AI system (ePiphany AI/ChatGPT) was used as a writing assistant to draft and paraphrase news articles, while Microsoft used automated content curation to promote these stories on MSN.
The report provides comprehensive, first-hand accounts from former BNN employees, the defamed individual, and screenshots of internal communications, leaving little ambiguity about the events.
None
Technology
AI Hallucination in Image Pairing
AI paired text with generic photo of a prominent Irish broadcaster.
ePiphany AI Chatbot Errors
The AI tool generated paraphrased stories containing factual errors.
Automated Content Curation
Microsoft replaced human editors with AI systems to curate MSN homepage.
Data Inputs
Unverified Source Data
AI uploaded articles from other sites to paraphrase without verification.
Lack of Proper Image Metadata
AI matched text to generic images lacking precise identifier metadata.
Human Factors
Lack of Manual Validation
Staff failed to check stories due to the high volume of AI output.
Forced Byline Attribution
Founder forced journalists' bylines on unchecked AI-generated stories.
Process and Methods
Prioritizing Speed Over Accuracy
Process optimized for high-volume automated publishing over factual checks.
Inadequate Content Review
Microsoft lacked rigorous human-in-the-loop review for licensed feed.
Algorithmic Monetization
Programmatic advertising incentivized mass-producing AI clickbait.
Management
Hostile Management Practices
Founder dismissed employee concerns about AI errors and integrity.
Growth-First Business Model
Management focused on scaling output via AI to maximize ad revenue.
Regulatory Environment
Lack of AI Disclosure Rules
No strict regulations requiring clear disclosure of AI-generated news.
Ineffective Platform Oversight
Search engines and portals lacked strict standards for AI news sites.
AI system involved
Identified by automated extraction; can differ from the alleged developers above.
