AI vs linguistic-based human judgement: Bridging the gap in pursuit of truth for fake news detection

Abstract

One of the negative aspects of the world becoming more digitized has been fake news, i.e., online disinformation – false, often fabricated reports of events, written and read on websites. The term has already entered collective consciousness and become an inseparable element of scientific discourse. Once a piece of news goes online, stopping it from spreading may become a complicated matter. Literature suggests that the two main pillars of the effective fight against fake news are education and detection. Thus, this paper describes a multidisciplinary study performed by a group of scientists representing two distinct fields - AI and linguistics. In their joint study, they compared, formally evaluated and explored the intersection between two approaches to fake news detection, i.e., the automated one, using a machine-learning-based tool, and the linguistic-based human judgement, using the data from two disinformation campaigns, sourced from two open benchmark fake news datasets. The study focused on the news' headlines as an effective proxy for the identification of fake news. In accordance with the achieved results, the paper argues that in the fight against fake news, the two approaches have the potential of augmenting and enhancing each other, utilizing the state-of-the-art technologies and linguistic knowledge. In addition, this paper provides a list of the linguistic features characteristic of possible disinformation, which is the most comprehensive collection of this kind in the subject literature to date.