Document type assignment by Web of Science, Scopus, PubMed, and publishers to “Top 100” papers

Abstract

Document type (DT) assignment is an important feature from literature databases. This work evaluated how the literature databases and the publisher websites labeled “Top 100” (T100) papers, a recurring act for which researchers identify and analyze the 100 most cited entities (e.g. articles) within a pre-defined literature set. T100 papers concurrently indexed in the Web of Science (WoS), Scopus and PubMed databses were identified. Among the 248 T100 papers analyzed, no general consensus or consistent pattern was found for labeling T100 papers by the three data sources and the publishers’ websites. All four sources labeled between 30–40% of the T100 papers as reviews. However, PubMed mostly did not give DT labels to the rest of the papers whereas WOS, Scopus, and publisher websites labeled them as articles. The inter-rater agreement was only fair; the decision seemed to be influenced by whether the authors mentioned the word “review” suggestive of the publication/document type in the title, abstract or keywords.