Evaluating the quality of clinical evidence in gastrointestinal cancers PubMed searches: How relevant are the results?

Abstract

11037 Background: Bibliographic repositories have become increasingly important as clinical evidence is more accessible on the internet. Often, oncologists rely on published literature to guide patient care. However, with the proliferation of journals and increasing number of peer reviewed papers, current search strategies have the potential to retrieve large numbers of irrelevant or misleading articles. Moreover, access to current best evidence may require paid subscriptions. Here, we assessed the quality of retrieved medical literature pertaining to treatment of gastrointestinal cancers using the PubMed database. Methods: We interrogated 6 focused-therapy questions in 3 categories: medical oncology (MedOnc), surgical oncology (SurgOnc), and alternative medicine (AltMed). We extracted journal metrics of the first two pages (40 results) from each search. We defined the composite outcome “relevant result” as the product of removing results from 1) predatory journals (Beall’s list), 2) non-English language, and 3) no free access publications. As a sensitivity analysis, we defined 2007 as a cutoff for “relevant” publications and Impact Factor (IF) > 3 or H-index > 50. Results: Two hundred and forty results were retrieved (80 per search type). Forty-eight percent of the journals were European (n = 115), 40% US-based (n = 96), and 94% were in English (n = 225). Most journals (n = 170; 70.8%) had an IF between 1-10, followed by ̃21% with an IF > 10 (n = 51); yet ̃45% (n = 107) were in the Clarivate Analytics top quartile. Sixty percent (n = 139) of articles were free to access. The articles had a median H-index of 117 [IR 59, 168]. When modelling the multivariate association with “relevant result”, year of publication after 2007 had an OR = 1.07 (95% CI = 1.01-1.14; p< 0.02) and availability through GOLD Open Access by Clarivate Analytics had an OR = 1.09 (95% CI = 1.03-1.15; p< 0.0008). MedOnc retrieved more papers published in journals with IF > 10 (n = 31; 38.8%) than SurgOnc searches (n = 12; 15%; p< 0.001). Only the AltMed searches included non-peer reviewed publications (n = 4; 5%) and 4 results were from “predatory journals”, all in MedOnc. Conclusions: Articles had a 7% higher chance of being considered a “relevant result” if they were published after 2007 and a 9% higher chance if available under GOLD Open Access. Publications identified as “relevant results” with an IF or H-index above the established cutoffs, had a 6% higher chance to be in the top 40 PubMed search results. In the sensitivity analysis the results remain virtually unchanged. Importantly, nearly 40% of papers indexed in PubMed do not have free full-text articles, making them unavailable to oncologists without access to a medical library or paid subscription. These results highlight the imperative to deliver relevant, freely accessible information that can impact the care of the cancer patients.