Methodological evaluation of bias in observational coronavirus disease 2019 studies on drug effectiveness

Abstract

Background and objectiveObservational studies may provide valuable evidence on real-world causal effects of drug effectiveness in patients with coronavirus disease 2019 (COVID-19). As patients are usually observed from hospital admission to discharge and drug initiation starts during hospitalization, advanced statistical methods are needed to account for time-dependent drug exposure, confounding and competing events. Our objective is to evaluate the observational studies on the three common methodological pitfalls in time-to-event analyses: immortal time bias, confounding bias and competing risk bias. MethodsWe performed a systematic literature search on 23 October 2020, in the PubMed database to identify observational cohort studies that evaluated drug effectiveness in hospitalized patients with COVID-19. We included articles published in four journals: British Medical Journal, New England Journal of Medicine, Journal of the American Medical Association and The Lancet as well as their sub-journals. ResultsOverall, out of 255 articles screened, 11 observational cohort studies on treatment effectiveness with drug exposure–outcome associations were evaluated. All studies were susceptible to one or more types of bias in the primary study analysis. Eight studies had a time-dependent treatment. However, the hazard ratios were not adjusted for immortal time in the primary analysis. Even though confounders presented at baseline have been addressed in nine studies, time-varying confounding caused by time-varying treatment exposure and clinical variables was less recognized. Only one out of 11 studies addressed competing event bias by extending follow-up beyond patient discharge. ConclusionsIn the observational cohort studies on drug effectiveness for treatment of COVID-19 published in four high-impact journals, the methodological biases were concerningly common. Appropriate statistical tools are essential to avoid misleading conclusions and to obtain a better understanding of potential treatment effects.