A Systematic Review of Methodological Approaches for Evaluating Real-World Effectiveness of Covid-19 Vaccines: Advising Resource-Constrained Settings

Abstract

Background: Real-world effectiveness studies are important for monitoring performance of COVID-19 vaccination programmes and informing COVID-19 prevention and control policies. We aimed to synthesise methodological approaches used in COVID-19 vaccine effectiveness studies, in order to evaluate which approaches are most appropriate to implement in low- and middle-income countries (LMICs). Methods: For this rapid systematic review, we searched PubMed and Scopus for articles published from inception to July 7, 2021, without language restrictions. We included any type of peer-reviewed observational study measuring COVID-19 vaccine effectiveness, for any population. We excluded randomised control trials and modelling studies. All data used in the analysis were extracted from included papers. We used a standardised data extraction form, modified from STrengthening the Reporting of OBservational studies in Epidemiology (STROBE). Study quality was assessed using the REal Life EVidence AssessmeNt Tool (RELEVANT) tool. This study is registered with PROSPERO, CRD42021264658. Findings: Our search identified 3,214 studies, of which 26 were eligible for analysis. All studies were conducted in 7 high-income countries and the majority assessed mRNA vaccines (81% mRNA only, 15% mRNA and viral vector). Twenty-one of the 26 studies (81%) used a cohort study design. There was significant heterogeneity for full vaccination effectiveness estimates across studies (infection: n=17, mean=79%; hospitalisation: n=7, mean=89%; death: n=3, mean=92%). Follow-up time for all studies was short (mean=9.5 weeks). Across studies, short follow-up time and limited assessment and mitigation of potential confounders, including previous SARS-CoV-2 infection and healthcare seeking behaviour, were major limitations. Interpretation: This review summarises methodological approaches for evaluating real-world effectiveness of COVID-19 vaccines and highlights the lack of such studies in LMICs, as well as the importance of context-specific vaccine effectiveness data. Further research in LMICs will refine guidance for conducting real-world COVID-19 vaccine effectiveness studies in resource-constrained settings. Funding: Health Systems Research Institute.