EU-MIDIS II: Roma - Survey methodology Q & A

This is a brief overview of how the Agency’s Second European Union Minorities and Discrimination Survey (EU-MIDIS II) was carried out.

The survey in a nutshell

The selected findings in this report are based on a survey interviews of Roma in nine EU Member States. It was part of the Agency’s EU-MIDIS II survey, which collected data on immigrants and ethnic minorities’ experiences and opinions in all 28 EU Member States. The survey methodology builds on the Agency’s first survey on immigrants and ethnic minorities in 2008 (EU-MIDIS I) and the Roma survey in 2011. The set of questions was extended and the coverage of the survey’s target groups was improved through advanced sampling approaches. The full results on Roma and a comprehensive technical report will be published in 2017.

1. Who does the term ‘Roma’ refer to?

The Council of Europe uses ‘Roma’ as an umbrella term. It refers to Roma, Sinti, Kale and related groups in Europe, including Travellers and the Eastern groups (Dom and Lom), and people who identify themselves as Gypsies.

EU-MIDIS II applied this umbrella term interviewing people who self-identified as ‘Roma’. The survey does not include ‘Roma’ who have moved from one EU Member State to another.

2. Why is survey data on Roma needed?

EU institutions have long underlined the need to ensure that Roma are treated equally with respect to their fundamental rights, as enshrined in the EU’s Charter of Fundamental Rights.

Although Roma are the largest ethnic minority in Europe, there is no systematic data collection on Roma in the EU Member States. The Agency fills this data gap with its surveys on ethnic minorities and Roma.

3. Why were Roma not interviewed in all 28 EU Member States?

The distribution and density of Roma populations differs across Member States and a random sampling method as used in this survey is not possible in all of them. For the countries not covered by the survey different data collection methods are needed. In 2017, the Agency will collect data in these countries

4. What questions did the survey ask?

Respondents were asked for information about their personal situation and living conditions, as well as basic socio-demographic characteristics of all household members. They were also asked about their experiences of discrimination, crime victimisation (including hate crime), profiling and policing as well as their rights awareness.

5. When and where was the survey carried out?

EU-MIDIS II interviewed Roma in nine EU Member States: Bulgaria, Croatia, the Czech Republic, Greece, Hungary, Portugal, Romania, Slovakia and Spain between October 2015 and April 2016.

6. How many Roma respondents/households were interviewed?

In total interviews were conducted in 7,947 Roma households. The sample sizes per country varied between 508 in Greece and 1,408 in Romania. The number of interviews allocated to each country is in proportion to the estimated Roma population size in each country and a minimum sample size of 500 per Roma country.

Roma respondents/households interviewed in survey and number of household members per EU Member State
Country	Respondents	Individuals in Roma households
Bulgaria	1,078	4,278
Croatia	538	2,800
Czech Republic	817	3,245
Greece	508	2,719
Hungary	1,171	4,941
Portugal	553	1,992
Romania	1,408	5,764
Slovakia	1,098	4,987
Spain	776	3,059
Total (9 MS)	7,947	33,785

7. Are the results representative?

The data aim to be representative for Roma living in the nine EU Member States who self-identify as ‘Roma’ or as members of the groups covered by this umbrella term. The survey covers geographic or administrative areas where Roma made up over 10% of the local population. The findings presented in this report therefore reflect the living conditions, enjoyment of fundamental rights and discrimination experiences of up to 80% of Roma living in the nine Member States surveyed.

8. How was the survey carried out?

The Agency designed the survey with input from academic experts and civil society representatives. An international survey company, Ipsos MORI managed the data collection in all 28 EU Member States.

Together with the Agency, Ipsos MORI designed an interviewer training programme and trained the national trainers in 2015. The national trainers trained interviewer teams in each country. FRA participated in a number of training sessions to monitor the quality of the training and its content.

The survey questionnaire for Roma was translated into nine EU languages, which were used to interview respondents.

FRA carried out the analysis for the current report.

9. How was the data collected?

The main interview mode was Computer Assisted Personal Interview (CAPI) – that is, face-to-face interviews by interviewers using a computerised questionnaire, and paper showcards to aid understanding.

10. How long was each interview?

The length of interviews with each respondent depended a lot on their personal experiences and was about 45 minutes.

11. How was the privacy, anonymity and confidentiality of respondents assured?

The survey was carried out by professional interviewers, trained to ensure confidentiality.

The survey data set is anonymised and does not contain any personal information, which would enable respondents to be identified. Care was taken during the data analysis so that nobody can be recognised from the results.

Sampling and selection of respondents

12.Who was eligible to participate in the survey?

Individuals aged 16 years or older, living in private households, who self-identify as Roma and whose usual place of residence was one of the nine EU Member States for at least 12 months.

13. How were Roma respondents selected?

The survey used a multistage random probability design. Areas with higher Roma densities (10 % and more) were initially selected and then a random number of selected addresses were screened, not knowing if a Roma person would be living there or not.

After contacting a household the interviewer asked a screening question (‘Is there anyone aged 16 or over living in this household who is Roma?’) to determine whether there were eligible Roma in the household to survey. In some countries, the term ‘Roma’ was replaced with a showcard listing all relevant terms commonly used in the country and included under the umbrella term ‘Roma’.

Only one randomly selected person per household was interviewed, while some survey questions asked about the situation of each household member.

Comparability between FRA surveys

14.Are the three FRA Roma surveys comparable?

To date, three FRA surveys – EU-MIDIS I (2008), Roma survey (2011) and EUMIDIS II (2016) – have collected data on Roma in six EU Member States: Bulgaria, the Czech Republic, Greece, Hungary, Romania and Slovakia. In addition to this, Roma in Portugal and Spain were covered in both the 2011 Roma survey and EUMIDIS II.

While all three surveys collected data on issues such as discrimination and rights awareness, results concerning living conditions such as poverty, educational 4 attainment and housing were only collected in the 2011 Roma survey and EUMIDIS II.

In some instances, the wording of questions changed slightly between surveys, which may limit the comparability of results.

EU-MIDIS I, the 2011 Roma survey, and EU-MIDIS II all used a similar methodology, applying a multistage selection of respondents. To optimise the sampling approach, EU-MIDIS II further developed the methodology applied in 2011. Despite similar methodological approaches used in the surveys, there are some limitations, which need to be considered when comparing the results.

15. Do improvements of the methodology lead to more accurate results?

The EU-MIDIS II survey further improved the sampling and weighting methods developed for the 2011 Roma survey. Therefore, the results of EU-MIDIS II should be a more accurate representation of the situation and experiences of Roma in the countries covered in the two surveys.

The 2016 EU-MIDIS II survey data were adjusted, taking into account selection probabilities. In 2011 only household size was adjusted when analysing results for respondents. This means that even if the sample in a country is similar, the 2016 data would yield more accurate results.

Results and analysis

16.Which results are presented in this report?

The results in this report reflect the priority areas of the EU Framework for National Roma Integration Strategies up to 2020: employment and poverty, education, housing, health, discrimination and rights awareness. Further and more detailed results and analysis will be published in 2017.

Where possible, and to allow for analytical comparison, the results use same or similar indicators as those in standard European surveys, such as the EU Statistics on Income and Living Condition survey or the EU Labour Force Survey.

Comparisons with the 2011 Roma survey and EU-MIDIS I are only reported when larger differences are observed. Any comparison should consider the methodological caveats and statistical variation as indicated in the notes to the tables and footnotes.