Interim Estimates in Null Models of COVID-19 Vaccine Effectiveness

Abstract

Recently released interim numbers from several ongoing vaccine candidate clinical trials suggest that a COVID-19 vaccine effectiveness (VE) above 90% is achievable. However, SARS-CoV-2 transmission dynamics is highly heterogeneous and exhibits localized bursts of transmission, which may lead to sharp localized peaks in the number of new cases, often followed by longer periods of zero incidence. Here we show that, for interim estimates of VE, this characteristic burstiness in SARS-CoV-2 infection dynamics may introduce a strong positive bias in VE. Specifically, one can generate null models of vaccine effectiveness, i.e., random models with burstiness that over longer times converge to exactly zero VE, but that for interim times frequently produce apparent VE near 100%. For example, by following the relevant clinical trial protocol, one can reproduce recently reported interim outcomes (VE > 90%, for 94 COVID-19 cases) from an ongoing phase 3 clinical trial of a RNA based vaccine candidate. Thus, to avoid potential random biases in VE, it is suggested that interim estimates on COVID-19 vaccine effectiveness should control for the intrinsic inhomogeneity in both SARS-CoV-2 infection dynamics and in reported cases.