Systematic Review Reveals Lack of Causal Methodology Applied to Pooled Longitudinal Observational Infectious Disease Studies

Heather Hufstedler,Sabahat Rahman,Alexander M Danzer,Hannah Goymann,Valentijn M.T De Jong,Harlan Campbell,Paul Gustafson,Thomas P.A Debray,Thomas Jaenisch,Lauren Maxwell,Ellicott C Matthay,Till Bärnighausen

doi:10.1016/j.jclinepi.2022.01.008

Abstract

Among ID studies seeking to make causal inferences and pooling individual-level longitudinal data from multiple infectious disease cohorts, we sought to assess what methods are being used, how those methods are being reported, and whether these factors have changed over time. Systematic review of longitudinal observational infectious disease studies pooling individual-level patient data from 2+ studies published in English in 2009, 2014, or 2019. This systematic review protocol is registered with PROSPERO (CRD42020204104). Our search yielded 1,462 unique articles. Of these, 16 were included in the final review. Our analysis showed a lack of causal inference methods and of clear reporting on methods and the required assumptions. There are many approaches to causal inference which may help facilitate accurate inference in the presence of unmeasured and time-varying confounding. In observational ID studies leveraging pooled, longitudinal IPD, the absence of these causal inference methods and gaps in the reporting of key methodological considerations suggests there is ample opportunity to enhance the rigor and reporting of research in this field. Interdisciplinary collaborations between substantive and methodological experts would strengthen future work.

Highlights

Randomized control trials (RCTs) have long been considered the gold standard over observational designs when endeavouring to infer a causal relationship in medicine
We propose the development and implementation of guidelines for transparent comprehensive reporting of harmonization and statistical methods applied to infectious disease (ID) focused studies that pool individual participant data (IPD) from two or more studies, including a) harmonization efforts, b) approaches to account for clustering and heterogeneity, c) approaches to account for missing data, d) approaches to account for data quality, e) justification of methods used, and f) explicit discussion of assessment of testable assumptions and evaluation of untestable assumptions
In reviewing the methods that authors used in their analysis to produce what we identified as causal claims, none of the studies used the modern causal methods we were looking for; 15 implemented traditional regression-based adjustment, and one used bivariate statistical tests

Summary

Introduction

Randomized control trials (RCTs) have long been considered the gold standard over observational designs when endeavouring to infer a causal relationship in medicine. RCTs may not be feasible or ethical, as in public health emergencies or when considering adverse exposures, making observational designs beneficial and necessary. Huge advances in aetiology were made with Rubin’s extension of counterfactuals to include observational studies [4], as well as Robins’ extension that demonstrated timevarying confounding created by time-dependent covariates which can both be confounders and intermediate variables [5], and the introduction of directed acyclic graphs (DAGs) [6,7,8]. Regression adjustment may not appropriately account for certain types of confounding in longitudinal analyses, as with time-dependent confounders impacted by prior treatment [9]. For many longitudinal ID cohorts, conventional regression adjustment may not be the most suitable analytic tool

Objectives

Methods

Results

Conclusion