How accurate are Twitter and Facebook altmetrics data? A comparative content analysis

Abstract

Data accuracy is essential for reliable and valid altmetrics analysis. Although Twitter and Facebook altmetrics data are widely used for scholarly communication and scientific evaluation, few studies have tapped into their accuracy issue. Based on content analysis of random sample records over two phases, this study has investigated and compared the accuracy of Twitter and Facebook altmetrics data. Major conclusions are drawn as follows. (1) Three error types were identified from the altmetric data provider and six error types were identified from the altmetric data aggregator. Twitter and Facebook have shared most of the error types except for minor differences in the sub-categories. (2) The overall error rate is substantially high, being 17% and 32% for Twitter and Facebook respectively in April, 2019. However, except for publication date error and posting date error, the percentage of the other error types is relatively low (being around 3%). (3) The percentage of error types related to the dynamic nature of Twitter and Facebook is increasing over time, while percentage of error types concerning the bibliographic data is decreasing over time. (4) The error types are either “high seriousness low percentage” or “low seriousness high percentage”, therefore, they would probably not bring significant negative influence. (5) Underlying reasons of these error types are various. They could be attributable to the Twitter (or Facebook) user, Twitter (or Facebook) platform, altmetric database, as well as the third-party data provider. These results suggest that Twitter and Facebook altmetrics data in the Altmetric database are reliable on the whole, although there is still space for further improvement.