Comparing Response Rates, Costs, and Tobacco-Related Outcomes Across Phone, Mail, and Online Surveys

Abstract

Tobacco control evaluation relies on surveillance documenting attitudes and behaviors. Practitioners and evaluators seek the most efficient, high-quality surveillance methodology. This paper explores the feasibility of mail and online survey data collection protocols using address-based sampling (ABS) to complement landline and cell phone surveys as part of a comprehensive tobacco control program evaluation. We used a comparative study of response rates, costs, and key outcomes across phone, mail, and online survey protocols. In 2015, we supplemented the phone-administered NY Adult Tobacco Survey with an ABS for paper and online data collection. For each survey protocol, we calculated response rates; compared unweighted demographic characteristics; and compared weighted outcome data for smoking prevalence, quit attempts, and tobacco control policy support. We assessed relative cost-per-complete per protocol. Response rates were highest for paper surveys (38.9%), followed by online (28.6%), landline (22.2%), and cell phone (14.7%) surveys. Respondent demographics differed across protocols; landline, mail, and online respondents were more likely than cell phone respondents to be older, female, white, and higher educated. Smoking prevalence varied by protocol, but quit attempts and tobacco control policy support were similar across protocols. Cost-per-complete estimates were lowest for paper surveys. Programs rely on efficient and representative methodologies, and paper and online surveys with ABS show promise for supplementing phone surveillance to improve response rates and lower costs per completed survey.