Vaccine Development during a Pandemic: General Lessons for Clinical Trial Design

Abstract

The COVID-19 pandemic triggered an unprecedented research effort to develop vaccines and therapeutics. Urgency dictated that development and regulatory assessment were accelerated, while maintaining all standards for quality, safety and efficacy. To speed up evaluation the European Medicines Agency (EMA) implemented “rolling reviews” allowing developers to submit data for assessment as they became available. We discuss the clinical trial designs and the applied statistical approaches in vaccine efficacy trials, focusing on aspects such as multiple testing, interim and updated analyses, and reporting of results for the first four vaccines recommended for approval by the EMA. The fast accrual of COVID-19 cases in the clinical vaccine efficacy trials led to multiple data updates within a short time frame, which had consequences for the evaluation and interpretation of results. Key trial results are discussed in the light of these aspects. Notably, the aspects discussed did not affect the benefit/risk relationship in a meaningful way, which was clearly positive for all four vaccines. Assessment of the development and evaluation of the four vaccine trials during the pandemic has led to a proposal for standardized terminology for trials with multiple analyses and a recommendation to appropriately preplan the timing of primary and updated analyses. For the reporting of updated estimates of vaccine efficacy, we discuss how to best describe the uncertainty around estimates of vaccine efficacy (e.g., via confidence intervals). Finally, we briefly highlight the benefit of a comprehensive discussion on estimands for vaccine efficacy trials.