Using proxy pattern-mixture models to explain bias in estimates of COVID-19 vaccine uptake from two large surveys

Abstract

Abstract Recently, attention was drawn to the failure of two very large internet-based probability surveys to correctly estimate COVID-19 vaccine uptake in the U.S. in early 2021. Both the Delphi-Facebook COVID-19 Trends and Impact Survey (CTIS) and Census Household Pulse Survey (HPS) overestimated uptake substantially, by 17 and 14 percentage points in May 2021, respectively. These surveys had large numbers of respondents but very low response rates (<10%), thus, nonignorable nonresponse could have had substantial impact. Specifically, it is plausible that ‘anti-vaccine’ individuals were less likely to participate given the topic (impact of the pandemic on daily life). In this article, we use proxy pattern-mixture models (PPMMs) to estimate the proportion of adults (18 +) who received at least one dose of a COVID-19 vaccine, using data from the CTIS and HPS, under a nonignorable nonresponse assumption. Data from the American Community Survey provide the necessary population data for the PPMMs. We compare these estimates to the true benchmark uptake numbers and show that the PPMM could have detected the direction of the bias and provide meaningful bias bounds. We also use the PPMM to estimate vaccine hesitancy, a measure for which we do not have a benchmark truth, and compare to the direct survey estimates.