The effects of nonresponse and follow-up mailings on a citizen survey of law enforcement services

Abstract

ABSTRACT Law enforcement agencies in many countries conduct surveys to learn about community opinions and concerns but low response rates can raise doubts about the generalizability of the results. This study examined differences between responders vs. nonresponders, and early vs. late responders, in a mailed survey of citizen satisfaction about law enforcement, and whether survey estimates changed with follow-up mailings. The univariate analysis revealed significant but small race/ethnicity and age differences between responders and nonresponders. Nonresponders tended to be nonwhite rather than white and to be younger. Late responders reported more negative attitudes toward law enforcement than did early responders with respect to perceived procedural justice and police legitimacy, although differences were small. While follow-up mailings increased the number of returns, this study found that the characteristics and attitudes of responders changed very little with additional mailings. The multivariate analysis revealed no evidence of multivariate bias, in particular, late survey responses did not change the positive association between procedural justice and police legitimacy. The findings imply that criminal justice scholars and law enforcement agencies that conduct citizen satisfaction surveys might consider increasing the initial sample size to boost statistical precision and power, or targeting unlikely respondents to reduce nonresponse bias, instead of investing in costly follow-up mailings. For police practice, the findings do not rule out invariance assertions about procedural justice. To the extent late responders resemble nonresponders, the impact of procedural justice may not depend on citizens’ survey response propensity.