A tool for evaluating scientific independence, integrity and transparency in observational studies of vaccine effectiveness

Abstract

Introduction Observational studies are widely used to evaluate the impact of vaccines. They are being conducted by a variety of institutions and collaborations, including public-private partnerships (PPPs). The European Union promotes PPPs through the Innovative Medicines Initiative (IMI): examples of its vaccine-related IMI PPPs include the ADVANCE and DRIVE consortia (respectively, Accelerated Development of VAccine benefit–risk Collaboration in Europe & Development of Robust and Innovative Vaccine Effectiveness). However, practices and attitudes towards PPPs in the public health field differ, and concerns over their scientific independence have been put forth. While there have been efforts to evaluate and strengthen the methodology and reporting of observational studies, clear and comprehensive guidelines on how to evaluate their scientific independence, integrity and transparency (especially in the presence of diverse governance and funding models) are lacking. Methods A review of existing guidelines touching upon the topics of scientific independence, integrity and/or transparency in the context of observational studies was performed (consisting of a literature search and screening of the reference lists of retrieved documents). Key indicators of scientific independence, integrity and transparency were identified with the focus on relevance and measurability. Based on these, a checklist for evaluating individual studies and study protocols has been compiled. Results Several guidelines include recommendations on topics related to scientific independence, integrity and transparency that can be applied to observational studies of vaccines’ impact. The ADVANCE Code of Conduct was selected as the pivotal reference as it already considered relevant guidelines during its preparation; examples of these include the International Epidemiological Association's Good Epidemiological Practice (GEP) and the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP) Checklist for Study Protocols. We present an overview of the various definitions and principles, as well as a checklist-based tool for evaluating the scientific independence, integrity and transparency of observational studies. The tool can be applied by independent reviewers to assess studies retrospectively as well as by study teams to inform the planning of new ones. Conclusions Questions of scientific independence, integrity and transparency are central in evaluating evidence from observational studies of vaccines. Our tool aims to provide a systematic way to address these topics; the next step will be to validate it in relation to existing protocols.