29
novembre
2023

Online content moderation - Current challenges in detecting hate speech

Online hate speech is a growing problem in today’s digitalised societies. Women, Black people, Jews and Roma are often targets of online hate speech. Online hate proliferates where human content moderators miss offensive content. Also, algorithms are prone to errors. They may multiply errors over time and may even end up promoting online hate. This report presents the challenges in identifying and detecting online hate. Hate of any kind should not be tolerated, regardless of whether it is online or offline. The report discusses the implications for fundamental rights to support creating a rights-compliant digital environment.
Overview

The emergence of online platforms and social media has transformed modern communication. Online platforms provide many opportunities to express opinions and participate in public and political discussions. However, just as offline discussions are replicated or amplified online, so are expressions of hate. This is of increasing concern.

The EU has updated its laws and implemented policies to tackle illegal content online, such as through the Digital Services Act (DSA), to more effectively regulate online content, including hate speech. However, these changes are relatively recent. In addition, there are still uncertainties concerning how to better protect human rights online, with regard to combating online hate while protecting freedom of expression, and how to efficiently implement existing and newly developed laws.

This report aims to better understand whether standard tools to address online hate speech, hereafter referred to as ‘online hate’, are effective by looking at manifestations of online hate after social media platforms have applied their content moderation controls. This report presents findings covering four social media platforms – Telegram, X (formally Twitter), Reddit and YouTube. The platforms were selected based on their accessibility for research purposes, their popularity (i.e. audience reach) and the assumed magnitude of hate speech on them.

The report aims to achieve the following:

  • It provides a targeted overview and analysis of online manifestations of misogyny and of hate against people of African descent, Jewish people and Roma. It is based on data collected on four online platforms, in four languages, and encompasses four EU Member States: Bulgaria, Germany, Italy and Sweden.
  • Based on data analysis of social media posts to identify potential online hate that has already gone through the online platforms’ content moderation systems, the report offers a critical assessment of the limitations of online content moderation tools in detecting online hate against specific groups. At the same time, the report highlights these very challenges as they relate to researching and measuring online hate.

The study focuses specifically on online hate in social media posts targeted at women, people of African descent, Jews and Roma to explore the limits of online content moderation and the extent of hate speech against these groups. The study collected social media posts over 6 months, using specific keywords that could indicate potential online hate against these target groups.

In this report:

  • Key findings and FRA opinions
  1. Challenge of online hate and how to research it
    • Need to address online hate
    • Online hate - Legal framework and definitions
    • Scope and methodology
  2. How online hate manifests itself - A snapshot
    • Characteristics of online hate
    • Forms of harassment - Online hate directed at individuals
    • Counterspeech
    • Target groups and intersectionality
    • Uncertainty and consistency in coding online hate
  3. Addressing online hate
    • Addressing fundamental rights-compliant content moderation of online hate
    • Understanding platforms' conduct in terms of safeguarding fundamental rights
  4. Ways forward
  • Annex 1: Technical details of the methodoology

 

FRA opinions
FRA opinion 1 - Regulators and online platforms should step up their efforts to offer a safer and non-discriminatory online environment for women and other groups

Online platforms should have specific regard to protected characteristics of users in the context of their terms and conditions, content moderation practices and monitoring policies, including addressing sexist online hate. Performance indicators should be in place to record the volume of misogyny online and the effectiveness of content moderation, looking at developments over time.

For very large online platforms (VLOPs), such as X and YouTube, misogyny should be one of the systemic risks considered in the context of the risk assessment and risk mitigation measures required by Articles 34 and 35 of the DSA.

The Council of Europe Convention on Preventing and Combating Violence against Women creates a coherent legal framework for the prevention, support and protection of women from violence – both offline and online. The EU is now party to this convention. EU Member States that have not yet signed and ratified the convention are urged to do so.

FRA opinion 2 - A variety of approaches to tackling online hate are needed, including different ways to detect hate speech

The European Commission and national governments should support, practically and financially, the creation of a wide and heterogeneous network of organisations acting as trusted flaggers to ensure that different forms of online hate are widely and reliably detected. Organisations representing groups with limited resources should not be put at a disadvantage in combating online hate. Users need to be made aware of easy ways to notify companies of hate speech, in line with Articles 16 and 22 of the DSA.

Given that views on what constitutes online hate may differ, a variety of measures to detect and report hate speech are needed. These include training on legal thresholds for identifying online hate for police, content moderators and trusted flaggers. The training could also ensure that platforms do not over-remove content.

The threshold for and description of what constitutes illegal online hate need more clarity. The changing landscape of expressions of hate and the magnitude of online hate mean that those involved in detecting and those responsible for addressing hate speech should not be left in any doubt regarding the rules. The EU and national legislators should consider clearer guidance and rules on what kind of online hate is illegal.

FRA opinion 3 - The use of artificial intelligence to detect and moderate online hate must comply with fundamental rights

It must be ensured that AI-supported online content moderation decisions are not discriminatory. Providers and users (i.e. platforms) must assess the fundamental rights compliance of any AI system in line with the DSA and current and developing standards regulating the use of AI.

The EU should ensure that applicable EU law, such as the DSA, appropriately addresses potential discrimination through the use of AI content moderation and requires that these systems are not used in a discriminatory way.

FRA opinion 4 - Regulators and online platforms must ensure that independent researchers have easier access to information required to analyse online hate

The European Commission should ensure that risk assessments under the DSA – including with regard to online hate – are complemented by extensive independent research using a variety of methods to ensure the accuracy and diversity of assessments. This is necessary, as any single method for analysing online hate remains limited. Only a variety of approaches and tests will provide a fuller picture of the challenges linked to identifying and combating online hate. Independent research can offer critical views and the appropriate methodologies required to further the understanding of the complex and fast-changing landscape of online hate and content moderation.

The European Commission should ensure that independent research institutes and academic researchers can access the data of online platforms without burdensome administrative procedures or other potential obstacles and in line with data protection safeguards. This will allow researchers to better investigate:

  • what kinds of online hate are not identified and appropriately taken down;
  • where content that is not illegal online hate is taken down;
  • what online hate’s impact is with regard to fundamental rights.