Evidence-Aware Multilingual Fake News Detection

Abstract

Due to the rapid growth of the Internet and the subsequent rise of social media users, sharing information has become more flexible than ever before. This unrestricted freedom has also led to an increase in fake news. During the Covid-19 outbreak, fake news spread globally, negatively affecting authorities’ decisions and the health of individuals. As a result, governments, media agencies, and academics have established fact-checking units and developed automatic detection systems. Research approaches focused largely on writing styles, propagation patterns, and building knowledge bases that serve as a reference for fact checking. However, little work has been done to assess the credibility of the source of the claim to be checked. This paper proposes a general framework for detecting fake news that uses external evidence to verify the veracity of online news in a multilingual setting. Search results from Google are used as evidence, and the claim is cross-checked with the top five search results. Additionally, we associate a vector of credibility scores with each evidence source based on the domain name and website reputation metrics. All of these components are combined to derive better predictions of the veracity of claims. Further, we analyze the claim-evidence entailment relationship and select supporting and refuting evidence to cross-check with the claim. The selection free approach yields better detection performance. In this work, we consider as a case study Covid-19 related news. Our framework achieves an F1-score of 0.85 and 0.97 in distinguishing fake from true news on XFact and Constraint datasets respectively. With the achieved results, the proposed framework present a promising automatic fact checker for both early and late detection.