ETMA: Efficient Transformer-Based Multilevel Attention Framework for Multimodal Fake News Detection

Abstract

In this new digital era, social media has created a severe impact on the lives of people. In recent times, fake news content on social media has become one of the major challenging problems for society. The dissemination of fabricated and false news articles includes multimodal data in the form of text and images. The previous methods have mainly focused on unimodal analysis. Moreover, for multimodal analysis, researchers fail to keep the unique characteristics corresponding to each modality. This article aims to overcome these limitations by proposing an efficient transformer-based multilevel attention (ETMA) framework for multimodal fake news detection, which comprises the following components: a visual attention-based encoder, a textual attention-based encoder, and joint attention-based learning. Each component utilizes different forms of attention mechanisms and uniquely deals with multimodal data to detect fraudulent content. The efficacy of the proposed network is validated by conducting several experiments on four real-world fake news datasets: Twitter, Jruvika fake news dataset, Pontes fake news dataset, and Risdal fake news dataset using multiple evaluation metrics. The results show that the proposed method outperforms the baseline methods on all four datasets. Furthermore, the computation time of the model is also lower than the state-of-the-art methods.