Promising Practice

Pretest of a question module on discrimination experiences in the innovation sample of the Socio-Economic Panel 2020

Pretest eines Fragenmoduls zu subjektiven Diskriminierungserfahrungen in der SOEP-Innovationsstichprobe 2020
In 2020/2021, the German Institute for Economic Research (DIW Berlin) conducted a comprehensive pretest of a question module on self-reports of discrimination in various areas of life. The aim was to prepare the module for the inclusion in the core sample of the Socio-Economic Panel (SOEP).
Land
Germany
Type
Operational guidelines
Mainstream equality data
Ensure regular and timely data collection
Ground of discrimination
All grounds / Cross-cutting

 


In this page:


Why was the practice needed?

In Germany, there is a lack of data for monitoring discrimination within the scope of protection enshrined in the General Equal Treatment Act. The data situation could be improved particularly by integrating a comprehensive module on discrimination experiences in a large-scale representative household survey, such as the Socio-Economic Panel (SOEP), at regular intervals. These data could not only inform the reporting on discrimination by Germany’s equality body, the Federal Anti-Discrimination Agency; it would, among other things, also provide an important source of data for indicators 10.3.1 and 16.b.1 of the Sustainable Development Goals (SDGs). Finally, the scientific community could use these data for investigating discrimination.

To prepare for the inclusion of such a module in the core sample of the SOEP and to test and optimise the questions, the German Institute for Economic Research (DIW Berlin) conducted a comprehensive pretest in 2020/2021. The pretest was also to investigate if respondents understood the wording of the questions and if they had reservations about answering rather sensitive questions on discrimination experiences.

How was it implemented?

The test module asked for self-reports of discrimination on all grounds defined in German law (plus some further characteristics such as social status) and in various areas of life. Following best practice in asking such questions, it captured discrimination experiences both within the last 5 years and in the last 12 months. In addition, the module comprised questions about whether respondents were aware of the existence of the General Equal Treatment Act. Furthermore, respondents who reported discrimination experiences were also asked how they reacted (e.g. by reporting the incident) and, if they did not react, why.

The project started with a cognitive pretest that investigated if respondents understood the wording and how they interpreted the questions. This cognitive pretest consisted of 10 interviews in three successive waves. The wording was adapted (where needed) after each of the three waves. It was followed by a quantitative test of the module in an online access panel with a sample size of 100. This step was needed to get an overview of the distribution of frequencies for the different variables and to assess if further answer options or categories were necessary.

Finally, the adapted question module was included in the Innovation Sample of the SOEP (SOEP-IS). The SOEP-IS is smaller than the SOEP-Core sample but shares some of its important design elements. It has been developed to include and test new and innovative questions. The questions on discrimination were asked to all SOEP-IS respondents (n = 3,785). Most of the interviews were conducted as computer-assisted personal interviews. Because of COVID-19, however, 30 % of the interviews were conducted by phone.

Outputs

  • The results of the pretest and a descriptive analysis of the data collected in the SOEP-IS have been published in a report. The authors concluded that the pretest of this module was successful and recommended the integration of such a module in the core sample of the SOEP at regular intervals.
  • The DIW Berlin decided that the module would be included in the 2022 SOEP core sample, albeit in a reduced form for capacity reasons. Thus, approximately 23,000 interviewees would be asked about discrimination experiences. The data will be available at the beginning of 2024. Thus, a comprehensive database for investigating discrimination experiences on all protected grounds and in various areas of life will be available in the foreseeable future.

Key success factors

  • The pretest was able to build upon previous work in the SOEP on self-reports on discrimination, such as the collection of data on discrimination experiences in the 2016 SOEP-IS or in the SOEP-IAB Migration Sample. But experiences and best practices of other data collectors such as FRA have also been taken into due consideration.
  • Furthermore, the combination of both qualitative methods (cognitive pretest) and quantitative methods proved to be particularly helpful for testing and optimising the module.

Technical information

  • Data sources covered: Household or individual surveys.
  • Areas of life covered: Employment; Education; Health; Housing and living conditions; Access to goods and services; Others: Public spaces, public authorities.
  • Target audience: Academia, research institutes, public administration, civil society organisations
  • Duration: May 2020–June 2021
  • Geographical scope: National
  • Leading institution: German Institute for Economic Research (DIW Berlin)
  • Other organisations involved/consulted: The Federal Anti-Discrimination Agency funded the pretest. Data collection was carried out by the polling institute Kantar Public Germany.
  • Financial & human resources: Budget: €143,829; Source of the budget: The Federal Anti-Discrimination Agency; Staff: Data not available.

Contacts

Federal Anti-Discrimination Agency / Antidiskriminierungsstelle des Bundes

poststelle@ads.bund.de