Respondent Burden Effects on Item Non-Response and Careless Response Rates: An Analysis of Two Types of Surveys

Álvaro Briz-Redón

doi:10.3390/math9172035

Abstract

The respondent burden refers to the effort required by a respondent to answer a questionnaire. Although this concept was introduced decades ago, few studies have focused on the quantitative detection of such a burden. In this paper, a face-to-face survey and a telephone survey conducted in Valencia (Spain) are analyzed. The presence of burden is studied in terms of both item non-response rates and careless response rates. In particular, two moving-window statistics based on the coefficient of unalikeability and the average longstring index are proposed for characterizing careless responding. Item non-response and careless response rates are modeled for each survey by using mixed-effects models, including respondent-level and question-level covariates and also temporal random effects to assess the existence of respondent burden during the questionnaire. The results suggest that the sociodemographic characteristics of the respondents and the typology of the question impact item non-response and careless response rates. Moreover, the estimates of the temporal random effects indicate that item non-response and careless response rates are time-varying, suggesting the presence of respondent burden. In particular, an increasing trend in item non-response rates in the telephone survey has been found, which supports the hypothesis of the burden. Regarding careless responding, despite the presence of some temporal variation, no clear trend has been identified.

Highlights

For each of the covariates considered at the individual or the question level, a reference level is used to which all of the other levels are compared so that each estimate must be interpreted with regard to the corresponding reference level
As the covariates have not been standardized, the fixed-effects estimates corresponding to different covariates are not comparable
There are two main types of methodologies available for measuring respondent burden: those that explicitly measure the level of burden experienced by the respondent throughout the interview and those that indirectly assess the existence of respondent burden by the subsequent analysis of respondents’ answers

Summary

Introduction

Identifying the existence of some form of respondent burden, determining its potential causes, and mitigating its consequences on survey estimates are essential to ensure that data collected through surveys are reliable and useful for some fields such as the social and health sciences in which surveys are widely used. This is a difficult task given the large degree of disagreement that still exists around the drivers and consequences of response burden, as has been revealed in a recent review of the currently available literature [1]. The authors of this review highlight that response burden is often conceptualized and measured by very different mechanisms

Objectives

Results

Conclusion