In search of better science: on the epistemic costs of systematic reviews and the need for a pluralistic stance to literature search

Abstract

This paper reviews the current status of academic search engines and emerging trends in scientific information retrieval and argues for two key claims. First, since systematic searches rely on the widespread use of academic search engines and the latter are generally not powered by cutting-edge Artificial Intelligence (AI) and not well-positioned to further the goals of findability and discoverability, there are some non-trivial epistemic costs associated with the tradition of systematic search. Second, while narrative reviews are typically criticized because of their lack of transparency, accountability, and reproducibility, they do deserve a place in scientific research. Specifically, once narrative reviews are properly understood as enabled by modern tools such as non-academic search engines, AI-powered recommender systems and academic social networks, it is possible to appreciate how these can indeed further the goal of literature discoverability. The upshot of this piece is that there are multiple goals and trade-offs involved in the process of scientific document search and that we should acknowledge virtues and limitations of different approaches to information retrieval and be prepared to welcome their combined use.