NEgatiVE results in Radiomics research (NEVER): A meta-research study of publication bias in leading radiology journals

Abstract

PurposeThe purpose of this study was to conduct a meta-research of radiomics-related articles for the publication of negative results, with a focus on the leading clinical radiology journals due to their purportedly high editorial standards. MethodsA literature search was performed in PubMed to identify original research studies on radiomics (last search date: August 16th, 2022). The search was restricted to studies published in Q1 clinical radiology journals indexed by Scopus and Web of Science. Following an a priori power analysis based on our null hypothesis, a random sampling of the published literature was conducted. Besides the six baseline study characteristics, a total of three items about publication bias were evaluated. Agreement between raters was analyzed. Disagreements were resolved through consensus. Statistical synthesis of the qualitative evaluations was presented. ResultsFollowing a priori power analysis, we included a random sample of 149 publications in this study. Most of the publications were retrospective (95%; 142/149), based on private data (91%; 136/149), centered on a single institution (75%; 111/149), and lacked external validation (81%; 121/149). Slightly fewer than half (44%; 66/149) made no comparison to non-radiomic approaches. Overall, only one study (1%; 1/149) reported negative results for radiomics, yielding a statistically significant binomial test (p < 0.0001). ConclusionThe top clinical radiology journals almost never publish negative results, having a strong bias toward publishing positive results. Almost half of the publications did not even compare their approach with a non-radiomic method.