Totality of Evidence of the effectiveness of repurposed therapies for COVID-19: Can we use Real-World Studies alongside Randomised Controlled Trials?

Abstract

Rapid and robust strategies to evaluate the efficacy and effectiveness of novel and existing pharmacotherapeutic interventions (repurposed treatments) in future pandemics are required. Observational 'Real-World Studies' (RWS) can report more quickly than randomised controlled trials (RCTs) and would have value were they to yield reliable results. Both RCTs and RWS were deployed during the COVID-19 pandemic. Comparing results between them offers a unique opportunity to determine the potential value and contribution of each. A learning review of these parallel evidence channels in COVID19, based on quantitative modelling, can help improve speed and reliability in the evaluation of repurposed therapeutics in a future pandemic. Analysis of all-cause mortality data from 249 observational RWS and RCTs across eight treatment regimens for COVID-19 showed that RWS yield more heterogeneous results, and generally over-estimate the effect size subsequently seen in RCTs. This is explained in part by a few study factors: the presence of RWS that are imbalanced for age, gender and disease severity, and those reporting mortality at 2 weeks or less. Smaller studies of either type contributed negligibly. Analysis of evidence generated sequentially during the pandemic indicated that larger RCTs drive our ability to make conclusive decisions regarding clinical benefit of each treatment, with limited inference drawn from RWS. These results suggest that when evaluating therapies in future pandemics, 1) large RCTs, especially platform studies, be deployed early; 2) any RWS should be large and should have adequate matching of known confounders and long follow-up; 3) reporting standards and data standards for primary endpoints, explanatory factors and key subgroups should be improved. In addition, 4) appropriate incentives should be in place to enable access to patient-level data; and 5) an overall aggregate view of all available results should be available at any given time.