Measuring Vaccine Efficacy Against Infection and Disease in Clinical Trials: Sources and Magnitude of Bias in Coronavirus Disease 2019 (COVID-19) Vaccine Efficacy Estimates.

Lucy R Williams,Nicholas C Grassly,Neil M Ferguson,Christl A Donnelly

doi:10.1093/cid/ciab914

Lucy R Williams, Nicholas C Grassly + Show 2 more

Open Access

https://doi.org/10.1093/cid/ciab914

Copy DOI

Abstract

BackgroundPhase III trials have estimated coronavirus disease 2019 (COVID-19) vaccine efficacy (VE) against symptomatic and asymptomatic infection. We explore the direction and magnitude of potential biases in these estimates and their implications for vaccine protection against infection and against disease in breakthrough infections.MethodsWe developed a mathematical model that accounts for natural and vaccine-induced immunity, changes in serostatus, and imperfect sensitivity and specificity of tests for infection and antibodies. We estimated expected biases in VE against symptomatic, asymptomatic, and any severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections and against disease following infection for a range of vaccine characteristics and measurement approaches, and the likely overall biases for published trial results that included asymptomatic infections.ResultsVE against asymptomatic infection measured by polymerase chain reaction (PCR) or serology is expected to be low or negative for vaccines that prevent disease but not infection. VE against any infection is overestimated when asymptomatic infections are less likely to be detected than symptomatic infections and the vaccine protects against symptom development. A competing bias toward underestimation arises for estimates based on tests with imperfect specificity, especially when testing is performed frequently. Our model indicates considerable uncertainty in Oxford-AstraZeneca ChAdOx1 and Janssen Ad26.COV2.S VE against any infection, with slightly higher than published, bias-adjusted values of 59.0% (95% uncertainty interval [UI] 38.4–77.1) and 70.9% (95% UI 49.8–80.7), respectively.ConclusionsMultiple biases are likely to influence COVID-19 VE estimates, potentially explaining the observed difference between ChAdOx1 and Ad26.COV2.S vaccines. These biases should be considered when interpreting both efficacy and effectiveness study results.

Highlights

M vaccine efficacy (VE) against asymptomatic infection measured by PCR or serology is expected to be low or negative d for vaccines that prevent disease but not infection
VE against any infection is overestimated when te asymptomatic infections are less likely to be detected than symptomatic infections and the vaccine p protects against symptom development
A competing bias towards underestimation arises for e estimates based on tests with imperfect specificity, especially when testing is performed frequently. c Our model indicates considerable uncertainty in Oxford-AstraZeneca ChAdOx1 and Janssen Ac Ad26.COV2.S VE against any infection, with slightly higher than published, bias-adjusted values of

Summary

Introduction

The coronavirus disease (COVID-19) phase III vaccine trials have demonstrated efficacy against symptomatic infection for multiple vaccines, with estimates ranging from 50% to 95% [1]. Sc The predominant primary outcome of the COVID-19 vaccine trials is vaccine efficacy (VE) against the u first case of PCR-confirmed symptomatic disease,. M either i) regular swabbing and PCR-testing, or ii) serological testing for anti-nucleocapsid antibodies at pre-specified time intervals, which allows seroconversion after infection to be identified for ted vaccines based on the spike protein (Table 1). Both strategies allow for estimation of VE against asymptomatic infection We explore the direction and magnitude of potential biases in these estimates and their implications for vaccine protection against infection and against disease in breakthrough infections

Methods

Results

Conclusion