Comparison of a web-push vs. mailed survey protocol in the Monitoring the Future panel study among adults ages 35 to 60

Abstract

Introduction:Updating the mode of data collection may affect response rates or survey results. The ongoing, national Monitoring the Future (MTF) panel study has traditionally used mailed paper surveys. In 2018, MTF experimented with a web-push data collection design for young adults ages 19–30, concluding that the web-push design improved response rates and did not change substance use estimates after controlling for sociodemographic characteristics (Patrick et al., 2021). The current study sought to replicate the web-push experiment with MTF adults ages 35 to 60 in 2020. Methods:In 2020, the MTF panel study included an experiment to test a web-push protocol for respondents ages 35 to 60 (N=14,379). Participants were randomized to the web-push (i.e., a web survey invitation, with paper surveys available for non-respondents) or traditional MTF (i.e., mailed paper surveys) data collection condition. Results:Results indicated no significant difference in overall response rate for the web-push vs. standard MTF conditions in this age group. Differences in reported estimates of past 30-day substance use prevalence by condition were not significant after adjusting for sociodemographic characteristics. In multivariable models, participants in the web-push condition were less likely to respond via web (than paper) if they were Black, smoked cigarettes in the past 30 days, were unmarried, or did not have a college degree. Conclusions:Overall, the move to the web-push design had minimal impact on response rates and substance use prevalence estimates for this age group. However, in the web-push condition, sociodemographic differences were associated with mode of response.