Data Sharing and Reanalyses Among Randomized Clinical Trials Published in Surgical Journals Before and After Adoption of a Data Availability and Reproducibility Policy

Abstract

Clinical trial data sharing holds promise for maximizing the value of clinical research. The International Committee of Medical Journal Editors (ICMJE) adopted a policy promoting data sharing in July 2018. To evaluate the association of the ICMJE data sharing policy with data availability and reproducibility of main conclusions among leading surgical journals. This cross-sectional study, conducted in October 2021, examined randomized clinical trials (RCTs) in 10 leading surgical journals before and after the implementation of the ICMJE data sharing policy in July 2018. Implementation of the ICMJE data sharing policy. To demonstrate a pre-post increase in data availability from 5% to 25% (α = .05; β = 0.1), 65 RCTs published before and 65 RCTs published after the policy was issued were included, and their data were requested. The primary outcome was data availability (ie, the receipt of sufficient data to enable reanalysis of the primary outcome). When data sharing was available, the primary outcomes reported in the journal articles were reanalyzed to explore reproducibility. The reproducibility features of these studies were detailed. Data were available for 2 of 65 RCTs (3.1%) published before the ICMJE policy and for 2 of 65 RCTs (3.1%) published after the policy was issued (odds ratio, 1.00; 95% CI, 0.07-14.19; P > .99). A data sharing statement was observed in 11 of 65 RCTs (16.9%) published after the policy vs none before the policy (risk ratio, 2.20; 95% CI, 1.81-2.68; P = .001). Data obtained for reanalysis (n = 4) were not from RCTs published with a data sharing statement. Of the 4 RCTs with available data, all of them had primary outcomes that were fully reproduced. However, discrepancies or inaccuracies that were not associated with study conclusions were identified in 3 RCTs. These concerned the number of patients included in 1 RCT, the management of missing values in another RCT, and discrepant timing for the principal outcome declared in the study registration and reported in the third RCT. This cross-sectional study suggests that data sharing practices are rare in surgical journals despite the ICMJE policy and that most RCTs published in these journals lack transparency. The results of these studies may not be reproducible by external researchers.