A comparison of misinformation feature effectiveness across issues and time on Chinese social media

Abstract

Misinformation on social media is a nonnegligible phenomenon that causes successive adverse impacts. Numerous scholarly efforts have been devoted to automatic misinformation detection to address this problem. The effective feature is the key to achieving high identification performance. However, the effectiveness of the feature may change in different issues and time considering the manifold social contextual reasons. Most extant literature on misinformation detection does not differentiate between topics, issues or domains. Although some research compares detection across domains, they concentrate on the model's overall performance, neglecting the effectiveness of individual features. Furthermore, the comparison studies mainly incorporate single-domain issues rather than issues that cover multiple domains. It is still difficult to determine which domain's misinformation characteristics will match those of multi-dimensional issues. Since the misinformation nowadays covers multiple domains, finding robust features in misinformation detection over issues and time is an urgent research agenda. In this study, we collected datasets of two issues, climate change and genetically modified organisms (GMOs), between January 1st, 2010 and December 31st, 2020 on Weibo, manually annotated the veracity status of the posts, and compared the performance of the proposed features in identifying misinformation by applying logistic regression. The results demonstrate that (1) the predicting power of content-based features, including topic and sentiment, is relatively robust compared to user-based and propagation-based features across issues and time. (2) The feature effectiveness varied at different time points. Our findings imply that future research could consider focusing more on content-based features, especially implicit features from the content in misinformation detection. Moreover, researchers should evaluate the feature effectiveness at different time stages to improve the efficiency of misinformation detection.