Revalidation of the applicability of Altmetrics indicators in article-level evaluation: An empirical analysis of papers of different types of citation trajectories

Abstract

While providers try to control the quality of the data, the applicability of Altmetrics indicators to the assessment of scientific papers remains an open question. One important reason is that the citation counts used to explain and evaluate the applicability of Altmetrics in this regard do not directly and completely reflect the impact and quality of papers. In view of the fact that the introduction of citation trajectory helps to enrich our understanding of the impact and quality of papers, this study first discusses the correlation between citation counts and Altmetrics indicators of papers under different citation trajectory types on the basis of dividing five citation trajectory types and considering possible influences such as field and publication year. Then, after controlling the relevant variables, we construct a multinomial logistic regression with the citation trajectory type as the dependent variable to analyze the possible relationship between Altmetrics and the citation trajectory type of papers. Finally, we construct a decision tree model and a regression model after mixed sampling to verify the robustness of the regression results. The findings reveal that there were significant differences in the performance of Altmetrics indicators among papers with different citation trajectory types. The applicability of Altmetrics for evaluating papers with different citation trajectory types should be judged carefully. At the same time, it is suggested that robust Altmetrics (such as save) can be applied to assess the quality of papers and characterize the citation life cycle.