Characteristics of non-randomised studies of drug treatments: cross sectional study

Abstract

ObjectiveTo examine the characteristics of comparative non-randomised studies that assess the effectiveness or safety, or both, of drug treatments.DesignCross sectional study.Data sourcesMedline (Ovid), for reports published from 1 June 2022 to 31 August 2022.Eligibility criteria for selecting studiesReports of comparative non-randomised studies that assessed the effectiveness or safety, or both, of drug treatments were included. A randomly ordered sample was screened until 200 eligible reports were found.Data on general characteristics, reporting characteristics, and time point alignment were extracted, and possible related biases,witha piloted form inspired by reporting guidelines and the target trial emulation framework.ResultsOf 462 reports of non-randomised studies identified, 262 studies were excluded (32% had no comparator and 25% did not account for confounding factors). To assess time point alignment and possible related biases, three study time points were considered: eligibility, treatment assignment, and start of follow-up. Of the 200 included reports, 70% had one possible bias, related to: inclusion of prevalent users in 24%, post-treatment eligibility criteria in 32%, immortal time periods in 42%, and classification of treatment in 23%. Reporting was incomplete, and only 2% reported all six of the key elements considered: eligibility criteria (87%), description of treatment (46%), deviations in treatment (27%), causal contrast (11%), primary outcomes (90%), and confounding factors (88%). Most studies used routinely collected data (67%), but only 7% reported using validation studies of the codes or algorithms applied to select the population. Only 7% of reports mentioned registration on a trial registry and 3% had an available protocol.ConclusionsThe findings of the study suggest that although access to real world evidence could be valuable, the robustness and transparency of non-randomised studies need to be improved.