Can social media usage of scientific literature predict journal indices of AJG, SNIP and JCR? An altmetric study of economics

Abstract

Altmetrics are often praised as an alternative or complement to classic bibliometric metrics, especially in the social sciences discipline. However, empirical investigations of altmetrics concerning the social sciences are scarce. This study investigates the extent to which economic research is shared on social media platforms with an emphasis on mentions in policy documents in addition to other mentions such as Twitter or Facebook. Moreover, this study explores machine learning models to predict the likelihood of a research article being classified into the top-quality tier of a journal ranking based on the altmetric mentions. The included journal rankings are the academic journal guide (AJG), source normalized impact per paper (SNIP) and journal citation reports (JCR). The investigated journals have been selected based on the AJG list and extracted from Altmetric.com data. After applying extensive data cleaning on the extracted data, a final set of 55,560 journal article records is obtained. The results indicate that the average number of policy mentions of the publications of economics journals is higher than the other subject areas included in the AJG list. Moreover, the publications in top-ranking economic journals are more likely to have a higher average number of policy mentions. Policy and Twitter mentions are presented as the most significant and informative social media mentions in demonstrating the broader impact and dissemination of Economics discipline followed by Blogs, Facebook, Wikipedia, and News. The results show that Support Vector Machine and Logistic Regression performed best in classifying the journal ranking tiers i.e. SNIP-based with 77% accuracy, JCR-based with 71% accuracy, and AJG-based with 66% accuracy. The models classified the ranking tier AJG18 with lower accuracy than SNIP and JCR. This might be because the AJG18 rankings are based on expert opinion, whereas SNIP and JCR are based on citations.