Not All Survey Effort is Equal

Abstract

Nonexperimental and experimental studies have shown a lack of association between survey effort and nonresponse bias. This does not necessarily mean, however, that additional effort could not reduce nonresponse bias. Theories on nonresponse would suggest the use of different recruiting methods for additional survey effort in order to address nonresponse bias. This study looks at changes in survey estimates as a function of making additional calls under the same protocol and additional calls under a different protocol. Respondents who were interviewed as a result of more than five call attempts were not significantly different on any of the key survey variables than those interviewed with fewer than five calls. Those interviewed under a different survey protocol, however, were different on 5 of 12 measures. Additional interviews under both the same and different protocols contributed to the reduction of total nonresponse error. In sum, the use of multiple protocols for part of the survey effort increased the response rate, changed point estimates, and achieved lower total nonresponse error. Future work is needed on optimizing survey designs that implement multiple survey protocols.