Identifying and Avoiding Survey Nonresponse and Noncompliance

Abstract

In the last five years, researchers have seen a notable increase in difficulty of collecting valid and useful survey data. Respondents, likely attracted to the prospect of receiving prizes or other compensation after survey completion, submit survey responses that are irrelevant to the questions asked, relevant but unusable due to the respondent not identifying as a member of the target population, or comprised of nonsense (such as paragraphs of lorem ipsum text or copy-pasted from other sources). As a result of experiencing a high volume of invalid survey responses to a qualitative survey aiming to research identity narrative formation among Irish women, this project highlights the forms in which invalid survey responses appear and analyzes the ways in which this might be avoided. Previous research has suggested the use of internet protocol (IP) address analysis and attention checks to prevent survey nonresponse as well as responses from non-target populations and invalid responses. For this project, tactics such as attention checks, skip logic, CAPTCHA, and highly specific population-relevant questions were experimented with to lessen invalid responses from both human respondents and bots, as well as to lessen the amount of data cleaning needed to find relevance in responses. Analysis of these methods and a conclusion of suggested best practices to avoid survey nonresponse and noncompliance are offered for future surveying.