Multilevel Attention Residual Neural Network for Multimodal Online Social Network Rumor Detection

Zhuang Wang,Jie Sui

doi:10.3389/fphy.2021.711221

Abstract

In recent years, with the rapid rise of social networks, such as Weibo and Twitter, multimodal social network rumors have also spread. Unlike traditional unimodal rumor detection, the main difficulty of multimodal rumor detection is in avoiding the generation of noise information while using the complementarity of different modal features. In this article, we propose a multimodal online social network rumor detection model based on the multilevel attention residual neural network (MARN). First, the features of text and image are extracted by Bert and ResNet-18, respectively, and the cross-attention residual mechanism is used to enhance the representation of images with a text vector. Second, the enhanced image vector and text vector are concatenated and fused by the self-attention residual mechanism. Finally, the fused image–text vectors are classified into two categories. Among them, the attention mechanism can effectively enhance the image representation and further improve the fusion effect between the image and the text, while the residual mechanism retains the unique attributes of each original modal feature while using different modal features. To assess the performance of the MARN model, we conduct experiments on the Weibo dataset, and the results show that the MARN model outperforms the state-of-the-art models in terms of accuracy and F1 value.

Highlights

Since the beginning of the 21st century, with the rapid development of the Internet technology and the gradual popularization of computers and other network terminal equipment, the dissemination speed of all kinds of news has been a qualitative leap, which has changed the inherent living habits of human beings to a certain extent
We propose a multimodal social network rumor detection model based on the multilevel attention residual neural network
We propose a multimodal online social network rumor detection model based on the multilevel attention residual mechanism

Summary

INTRODUCTION

Since the beginning of the 21st century, with the rapid development of the Internet technology and the gradual popularization of computers and other network terminal equipment, the dissemination speed of all kinds of news has been a qualitative leap, which has changed the inherent living habits of human beings to a certain extent. Compared with the traditional news industry, social media have a lower release threshold, a faster spread, and a wider range of influence. These network rumors reduce the quality of people’s access to information and seriously endanger the security of the whole society and even at the national level. 2) The noise information generated by the fusion is large, which affects the final classification results To solve these problems, we propose a multimodal social network rumor detection model based on the multilevel attention residual neural network. The contribution of this article can be summarized as the following three points: 1) This article proposes a multimodal social network rumor detection model based on the multilevel attention residual neural network. 3) The experimental results on the real Weibo dataset show that the accuracy and F1 value of the MARN model are higher than those of the current mainstream multimodal rumor detection models

RELATED WORK

EXPERIMENTS

CONCLUSION

Findings

DATA AVAILABILITY STATEMENT