AI-facilitated Misinformation and Violence
Measures addressing AI-generated misinformation, deepfakes and incitement to violence.
AI-facilitated misinformation and violence refers to harms arising when generative AI, synthetic-media tools, coordinated botnets, or AI-enabled targeting are used to produce or amplify personalised, false, misleading, or inciting content that drives social unrest, hate, harassment, psychological trauma, physical threats, or the recruitment and operational capacity of violent actors. This includes synthetic media used to impersonate or intimidate, algorithmically amplified disinformation campaigns designed to destabilise communities or elections, use of AI to manipulate images and create synthetic child sexual abuse material (CSAM), AI-facilitated doxxing, harassment, and gender-based violence.
Country rankings by AI-facilitated Misinformation and Violence
Background of AI-facilitated Misinformation and Violence
Generative AI has sharply lowered the technical capabilities and costs to producing hyper-realistic, persuasive false content and to automating large-scale targeting, driving both the volume and effectiveness of misinformation and disinformation. A study from the European Commission found that pornographic material accounts for around 98% of deepfakes, with the vast majority of those targeted being women and girls, indicating that those who are already marginalised in their communities are made vulnerable by AI tools being used for image manipulation.
Relevance of AI-facilitated Misinformation and Violence
AI-enabled disinformation and technology-facilitated gender-based violence increasingly undermine equitable participation in public life. These harms disproportionately target journalists from marginalized communities and human rights defenders, contributing to self-censorship, withdrawal from digital spaces, and, in many cases, escalating into offline harassment and violence. Children are especially exposed given their high trust in AI systems and limited ability to detect manipulation, with rising synthetic CSAM, grooming, and sextortion risks underscoring the urgency of governance responses that anticipate AI's role in accelerating both the scale and realism of these harms.
Evidence for AI-facilitated Misinformation and Violence
Performance by Region
How different regions perform in the ai-facilitated misinformation and violence indicator, revealing both progress and persistent challenges
AI-facilitated Misinformation and Violence
Global Index on Responsible AI — 2024 Report
Download the 2024 edition of the Global Index on Responsible AI, offering comparative insights into how countries govern and use artificial intelligence.
