Analysis of journal evaluation indicators: an experimental study based on unsupervised Laplacian score

Abstract

Academic journal rankings have always been a hot topic because the journals ranking will be significantly affected by using different ranking systems or bibliometrics indicators. Many studies have shown that feature selection methods are beneficial to categorize and rank journals according to the different evaluation indicators. However, such methods are limited to analyzing the journals published in a single discipline. In this article, we conduct an experiment based on unsupervised Laplacian score to analyze journal evaluation indicators between three different disciplines (Biology, Artificial Intelligence, and Mathematics). The journal’s features considered in the study include: JIF, 5-Year JIF, CiteScore, SJR, SNIP, and H-index. Based on the results of the feature selection, we first use the spectral clustering technique to verify the impact of different feature combinations on journals categorization. Furthermore, k-Nearest Neighbor (k-NN) and Back Propagation Neural Network classifiers are trained to measure the classification performance by using each single feature or feature subsets. The experimental results demonstrate that the two classifiers achieve the superior classification performance when the input of the classifier is the set of JIF, CiteScore, and H-index. Finally, we design a subjective questionnaire to collect the opinion of the volunteers with regard to the six evaluation indiactors in terms of familiarity, complexity, and usability. According to the vote results of a seven-point Likert scale, JIF and H-index receive the relatively higher ratings in comparison to other bibliometric indicators.