Using Paradata to Assess Respondent Burden and Interviewer Effects in Household Surveys: Evidence from Low- and Middle-Income Countries

Abstract

Over the past decade, national statistical offices in low- and middle-income countries have increasingly transitioned to computer-assisted personal interviewing and computer-assisted telephone interviewing for the implementation of household surveys. The byproducts of these types of data collection are survey paradata, which can unlock objective, module- and question-specific, actionable insights on respondent burden, survey costs, and interviewer effects. This study does precisely that, using paradata generated by the Survey Solutions computer-assisted personal interviewing platform in recent national household surveys implemented by the national statistical offices in Cambodia, Ethiopia, and Tanzania. Across countries, the average household interview, based on a socioeconomic household questionnaire, ranges from 82 to 120 minutes, while the average interview with an adult household member, based on a multi-topic individual questionnaire, takes between 13 to 25 minutes. Using a multilevel model that is estimated for each household and individual questionnaire module, the paper shows that interviewer effects on module duration are significantly larger than the estimates from high-income contexts. Food consumption, household roster, and non-farm enterprises consistently emerge among the top five household questionnaire modules in terms of total variance in duration, with 5 to 50 percent of the variability being attributable to interviewers. Similarly, labor, health, and land ownership appear among the top five individual questionnaire modules in terms of total variance in duration, with 6 to 50 percent of the variability being attributable to interviewers. These findings, particularly by module, point to where additional interviewer training, fieldwork supervision, and data quality monitoring may be needed in future surveys.