An evaluation of Web of Science, Scopus and Google Scholar citations in operations management

Abstract

Purpose Ongoing deliberation about how research productivity should be measured is exacerbated by extensive disparity between the number of citations for scholarly works reported by commercial academic search engines and Google Scholar (GS), the premier web crawling service for discovering research citations. Disparities identified in citation comparison studies have also led to disagreement about the value of the higher number of citations for social sciences and business scholarly articles consistently reported by GS. The purpose of this paper is to extend previous database citation comparison studies by manually analyzing a sample of unique GS citations to a leading operations management journal (i.e. citations found only in GS and not the commercial search engines) to reveal just where these additional citations are coming from. Design/methodology/approach In addition to comparing citation counts for the three databases, unique GS citation data for the sample of journal articles was manually captured and reviewed. The authors’ approach provides a much more in-depth examination of the provenance of GS citations than is found in previous studies. Findings The findings suggest that concerns about the value of unique GS citations may not be warranted since the document types for the unique GS citing documents identified in the analysis are dominated by familiar scholarly formats. Predominantly authentic and validated journal publications, dissertations, conference papers, and book and book chapters accounted for the large majority of the unique GS citations analyzed. Practical implications The study lends further credence to contentions that the use of citations reported in GS is appropriate for evaluating research impact in disciplines where other formats beyond the English-language journal article are valued. Originality/value Developing a more informed understanding of the provenance of unique GS citations in the authors’ field is important because many scholars not only aspire to publish in elite journals with high impact factors based on citation counts provided by commercial databases to demonstrate quality, but also report the larger number of citations for their publications that are reported by GS to demonstrate impact. The in-depth manual analysis suggests that GS provides a more nuanced and comprehensive representation of research impact and international scope than the commercial databases.