Misinformation Detection Research Articles

Depth, breadth and structural virality: the influence of emotion, topic, authority and richness on misinformation spread

PurposeThis study aims to investigate the mechanism of the misinformation spread based on the elaboration likelihood model and the effects of four factors – emotion, topic, authority and richness – on the depth, breadth and structural virality of misinformation spread.Design/methodology/approachThe authors collected 2,514 misinformation microblogs and 142,006 reposts from Weibo, used deep learning methods to identify the emotions and topics of misinformation and extracted the structural characteristics of the spreading network using the network analysis method.FindingsResults show that misinformation has a smaller spread size and breadth than true news but has a similar spread depth and structural virality. The differential influence of emotions on the structural characteristics of misinformation propagation was found: sadness can promote the breadth of misinformation spread, anger can promote depth and disgust can promote depth and structural virality. In addition, the international topic, the number of followers, images and videos can significantly and positively influence the misinformation's spread size, depth, breadth and structural virality.Originality/valueThe influencing factors of the structural characteristics of misinformation propagation are clarified, which is helpful for the detection and management of misinformation.

Adversarial NLP for Social Network Applications: Attacks, Defenses, and Research Directions

The growing use of media has led to the development of several machine learning (ML) and natural language processing (NLP) tools to process the unprecedented amount of social media content to make actionable decisions. However, these ML and NLP algorithms have been widely shown to be vulnerable to adversarial attacks. These vulnerabilities allow adversaries to launch a diversified set of adversarial attacks on these algorithms in different applications of social media text processing. In this article, we provide a comprehensive review of the main approaches for adversarial attacks and defenses in the context of social media applications with a particular focus on key challenges and future research directions. In detail, we cover literature on six key applications: 1) rumors detection; 2) satires detection; 3) clickbaits and spams identification; 4) hate speech detection; 5) misinformation detection; and 6) sentiment analysis. We then highlight the concurrent and anticipated future research questions and provide recommendations and directions for future work.

Exploiting sparsity and statistical dependence in multivariate data fusion: an application to misinformation detection for high-impact events

Exploiting sparsity and statistical dependence in multivariate data fusion: an application to misinformation detection for high-impact events

ACOVMD: Automatic COVID‐19 misinformation detection in Twitter using self‐trained semi‐supervised hybrid deep learning model

AbstractDuring the COVID‐19 pandemic, online social networks are extensively utilized, more than ever before by 8.4%, resulting in the propagation of false information related to COVID‐19. Despite the existence of many fake news detection models; annotation inconsistency, memory consumption, accurate and self‐trained efficient algorithms for detecting the emerging COVID‐19 misinformation tweets are still challenging. Hence, the main aim of this work is to come up with a self‐trained semi‐supervised model that accurately and automatically detects the reliability of emerging COVID‐19 tweets without delay. In this work, COVID‐19 tweet dataset is created in English Language from the period January 2020 to January 2022 as a ground truth database. Then self‐trained semi‐supervised hybrid deep learning model is proposed to train both supervised and unsupervised components simultaneously using the created dataset. The proposed model is self‐trained repeatedly and the model gets updated to predict the reliability of upcoming COVID‐19 tweets that differ from training tweets. We performed experiments multiple times by limiting the percentage amount of labelled tweets shown to the model, namely 80%, 50%, 40%, 30%, 20% and 10% labelled tweets, respectively. Experimental results show that the proposed model achieves 80.92% accuracy and 98.15% accuracy in the 10% and 80% label‐seen experiments, respectively. This shows a clear rising trend in the performance curve. Therefore, this technique will be useful for effectively classifying voluminous amounts of emerging tweets generated as part of the COVID‐19 infodemic. The proposed model may efficiently use a huge amount of unlabelled tweets and enhance the model's generalization performance.

Evidence and gap map of infodemic management interventions in emergencies: a case of COVID-19

Abstract Background Because infodemic management (IM) is an evolving field that is bridging science into practice, it is often conducted in the field in an ad-hoc manner, focusing on implementation to address immediate challenges, and is less evaluated or reported in the scientific literature. This project used the Evidence and Gap Map (EGM) as a tool to provide an overview of investments in research and evidence for IM interventions. Methods A WHO working group advised development of protocol and of taxonomy of IM interventions and outcome measures based on literature review and expert discussion. Search was conducted on electronic databases and grey literature. Screening/coding of the studies was carried out independently by reviewers in two stages: the title and abstract and the full-text screening stage. Data extraction/critical appraisal was carried out for the included studies. Results 983 studies were included from 29,117 search records. Among impact studies that measured and reported outcomes, the reported interventions were on ‘science & health communication’ (n = 30), ‘education & training’ (n = 30), and ‘engagement’ (n = 10). ‘Individual knowledge, attitude and behaviors’ was the most frequently measured outcome. In descriptive studies not measuring outcomes, majority reported an intervention of ‘monitoring/detection of narratives, questions, concerns, and misinformation in the information ecosystem’ (580 studies), by ‘science + health communication’ (n = 258), ‘education + training’ (n = 86), ‘engagement’ (n = 86) and ‘laws, regulation + ethics’ (n = 60). Conclusions The EGM reviewed the evidence base on IM interventions. The analysis shows scarcity of impact evaluation studies in this area. Moreover, the map reflects concentrations of research in social media analytics studies, model studies for misinformation detection, individual level interventions. Major gaps exist in systems level interventions that address the information environment, and in broader impact measures. Key messages • Research funders should prioritize impact evaluation studies for systemic interventions to address infodemic to support solutions in design and implementation of such complex studies. • More research on infodemic management interventions is needed in LMICs and at health system level.

Role of Statistics in Detecting Misinformation: A Review of the State of the Art, Open Issues, and Future Research Directions

With the evolution of social media, cyberspace has become the default medium for social media users to communicate, especially during high-impact events such as pandemics, natural disasters, terrorist attacks, and periods of political unrest. However, during such events, misinformation can spread rapidly on social media, affecting decision-making and creating social unrest. Identifying and curtailing the spread of misinformation during high-impact events are significant data challenges given the scarcity and variety of the data, the speed by which misinformation can propagate, and the fairness aspects associated with this societal problem. Recent statistical machine learning advances have shown promise for misinformation detection; however, key limitations still make this a significant challenge. These limitations relate to using representative and bias-free multimodal data and to the explainability, fairness, and reliable performance of a system that detects misinformation. In this article, we critically discuss the current state-of-the-art approaches that attempt to respond to these complex requirements and present major unsolved issues; future research directions; and the synergies among statistics, data science, and other sciences for detecting misinformation.

A novel hybrid approach for text encoding: Cognitive Attention To Syntax model to detect online misinformation

Most approaches for text encoding rely on the attention mechanism, at the core of the transformers architecture and large language models. The understanding of this mechanism is still limited and present inconvenients such as lack of interpretability, large requirements of data and low generalization. Based on current understanding of the attention mechanism, we propose CATS (Cognitive Attention To Syntax), a neurosymbolic attention encoding approach based on the syntactic understanding of texts. This approach has on-par to better performance compared to classical attention and displays expected advantages of neurosymbolic AI such as better functioning with little data and better explainability. This layer has been tested on the task of misinformation detection but is general and could be used in any task involving natural language processing.

Review of Existing Methods and Technologies for Detection and Mitigation of Misinformation and Distorted Data in Social Networks

Review of Existing Methods and Technologies for Detection and Mitigation of Misinformation and Distorted Data in Social Networks

Gamified inoculation interventions do not improve discrimination between true and fake news: Reanalyzing existing research with receiver operating characteristic analysis.

Gamified inoculation interventions designed to improve the detection of online misinformation are becoming increasingly prevalent. Two of the most notable interventions of this kind are Bad News and Go Viral!. To assess their efficacy, prior research has typically used pre-post designs in which participants rated the reliability or manipulativeness of true and fake news items before and after playing these games, while most of the time also including a control group who played an irrelevant game (Tetris) or did nothing at all. Mean ratings were then compared between pre-tests and post-tests and/or between the control and experimental conditions. Critically, these prior studies have not separated response bias effects (overall tendency to respond "true" or "fake") from discrimination (ability to distinguish between true and fake news, commonly dubbed discernment). We reanalyzed the results from five prior studies using receiver operating characteristic (ROC) curves, a method common to signal detection theory that allows for discrimination to be measured free from response bias. Across the studies, when comparable true and fake news items were used, Bad News and Go Viral! did not improve discrimination, but rather elicited more "false" responses to all news items (more conservative responding). These novel findings suggest that the current gamified inoculation interventions designed to improve fake news detection are not as effective as previously thought and may even be counterproductive. They also demonstrate the usefulness of ROC analysis, a largely unexploited method in this setting, for assessing the effectiveness of any intervention designed to improve fake news detection. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Advanced Misinformation Detection: A Bi-LSTM Model Optimized by Genetic Algorithms

The proliferation of misinformation, as insidious and pervasive as water, presents an unprecedented challenge to public discourse and comprehension. Often propagated to further specific ideologies or political objectives, misinformation not only misleads the populace but also fuels online advertising revenue generation. As such, the urgent need to pinpoint and eliminate misinformation from digital platforms has never been more critical. In response to this dilemma, this paper proposes a solution built on the backbone of massive data generation in today’s digital landscape. By leveraging advanced technologies, such as AI-driven systems with deep learning models and natural language processing capabilities, we can monitor and analyze an extensive scope of social media data. This, in turn, facilitates the identification of misinformation across multiple platforms and alerts users to potential propaganda. Central to our study is the development of misinformation classifiers based on a deep bi-directional long short-term memory (Bi-LSTM) model. This model is further enhanced by employing a genetic algorithm (GA), which automates the search for an optimal neural architecture, thereby significantly impacting the training behavior of the deep learning algorithm and the performance of the model being trained. To validate our approach, we compared the efficacy of our proposed model with nine traditional machine learning algorithms and a deep learning model rooted in long short-term memory (LSTM). The results affirmed the superiority of our GA-tuned Bi-LSTM model, which outperformed all other models in detecting misinformation with remarkable accuracy. Our intention with this paper is not to present our model as a comprehensive solution to misinformation but rather as a technological tool that can aid in the process, supplementing and bolstering the existing methodologies in the field of misinformation detection.

Rapid Review on Publicly Available Datasets for Health Misinformation Detection.

The proliferation of health misinformation in recent years has prompted the development of various methods for detecting and combatting this issue. This review aims to provide an overview of the implementation strategies and characteristics of publicly available datasets that can be used for health misinformation detection. Since 2020, a large number of such datasets have emerged, half of which are focused on COVID-19. Most of the datasets are based on fact-checkable websites, while only a few are annotated by experts. Furthermore, some datasets provide additional information such as social engagement and explanations, which can be utilized to study the spread of misinformation. Overall, these datasets offer a valuable resource for researchers working to combat the spread and consequences of health misinformation.

HealthE: Recognizing Health Advice & Entities in Online Health Communities

The task of extracting and classifying entities is at the core of important Health-NLP systems such as misinformation detection, medical dialogue modeling, and patient-centric information tools. Granular knowledge of textual entities allows these systems to utilize knowledge bases, retrieve relevant information, and build graphical representations of texts. Unfortunately, most existing works on health entity recognition are trained on clinical notes, which are both lexically and semantically different from public health information found in online health resources or social media. In other words, existing health entity recognizers vastly under-represent the entities relevant to public health data, such as those provided by sites like WebMD. It is crucial that future Health-NLP systems be able to model such information, as people rely on online health advice for personal health management and clinically relevant decision making. In this work, we release a new annotated dataset, HealthE, which facilitates the large-scale analysis of online textual health advice. HealthE consists of 3,400 health advice statements with token-level entity annotations. Additionally, we release 2,256 health statements which are not health advice to facilitate health advice mining. HealthE is the first dataset with an entity-recognition label space designed for the modeling of online health advice. We motivate the need for HealthE by demonstrating the limitations of five widely-used health entity recognizers on HealthE, such as those offered by Google and Amazon. We additionally benchmark three pre-trained language models on our dataset as reference for future research. All data is made publicly available.

Towards Cross-Lingual Multi-Modal Misinformation Detection for E-Commerce Management

The misinformation detection systems are increasingly important in E-commerce management, which detect misinformation on the commodity display page. Misinformation in E-commerce is usually presented as a mismatch between multi-modal information, the detection systems need to find the misinformation across the multi-model information. Furthermore, with the development of E-commerce globalization, we hope to deploy the system in the international E-commerce platform, which may face the difficulty caused by multi-lingual data. To this end, we propose a Cross-lingual Multi-modal Misinformation Detection (CMMD) framework for E-commerce management. The CMMD framework includes a word alignment network that embeds information from different languages into the same feature space and a multimodal fusion structure that fuses text representations and image representations through two self-supervised tasks. With the cooperation of these two modules, the CMMD model could extract rich cross-lingual multi-modal features to achieve accurate misinformation detection. We conduct experiments on the public multi-modal dataset and further apply the proposed CMMD to the real-world E-commerce international platform. The experimental results show that the proposed framework CMMD achieves better performance on the public dataset than some benchmarks and gets satisfactory results in real-world E-commerce management.

Automatic detection of health misinformation: a systematic review

The spread of health misinformation has the potential to cause serious harm to public health, from leading to vaccine hesitancy to adoption of unproven disease treatments. In addition, it could have other effects on society such as an increase in hate speech towards ethnic groups or medical experts. To counteract the sheer amount of misinformation, there is a need to use automatic detection methods. In this paper we conduct a systematic review of the computer science literature exploring text mining techniques and machine learning methods to detect health misinformation. To organize the reviewed papers, we propose a taxonomy, examine publicly available datasets, and conduct a content-based analysis to investigate analogies and differences among Covid-19 datasets and datasets related to other health domains. Finally, we describe open challenges and conclude with future directions.

Does blatantly contradictory information reduce the misinformation effect? A Registered Report replication of Loftus (1979)

AbstractPurposeLoftus (1979, Reactions to blatantly contradictory information) demonstrated that participants who received a piece of blatantly contradictory information were not susceptible to it (the boundary condition effect). In addition, participants who had received the blatant misinformation were also less susceptible to the more subtle pieces of misinformation compared to the participants who did not receive the blatant misinformation (the inoculation effect). Researchers often cited this finding as evidence for the idea that detecting a piece of blatant misinformation is similar to being given a warning. However, to our knowledge, the inoculation effect has not been replicated.MethodsExperiment 1 was a direct replication. Participants either encountered four pieces of nonblatant misinformation (subtle condition) or the same four pieces of nonblatant misinformation and an additional piece of blatant misinformation (blatant condition). Experiment 2 was a conceptual replication in which we also manipulated the presence or absence of initial testing. Furthermore, we presented the slides as a video and presented four neutral and four misleading details in the narrative.ResultsIn two high‐powered experiments, we failed to replicate both the inoculation effect and the boundary condition effect. Using a multi‐verse analysis approach, we failed to observe a significant advantage from the inclusion of a piece of blatant misinformation in most (but not all) analyses.ConclusionWe conclude that encountering a piece of blatant misinformation was not sufficient to inoculate witnesses against other nonblatant pieces of misinformation. We further discuss our results in the context of misinformation, warning, and discrepancy detection.

Medical Relevancy of Cancer-Related Tweets and Its Relation to Misinformation

Social media is one of the most dominant ways of spreading information. Still, unfortunately, these open platforms provide ways to spreading misinformation which can be extremely dangerous, especially when relevant to sensitive issues such as health-related information. Hence such platforms require an effective autonomous misinformation detection mechanism. Understanding the data is one of the necessary artifacts for building such a mechanism. In this work, we attempted to determine the medical relevancy of cancer-related tweets and explore whether they contain misinformation. We created a dataset of roughly 500 tweets and labeled them according to their medical relevance: medically relevant, not medically relevant, or unrelated to cancer. We ran logistic regression and support vector machine models on them. The highest proportion of correctly identified “medically relevant” tweets, i.e., accuracy, was 0.795. Our analysis hints at some features and factors that can automatically improve cancer-relevant and non-relevant tweet detection.

Listen to What They Say: Better Understand and Detect Online Misinformation with User Feedback


 
 
 Social media users who report content are key allies in the management of online misinformation; however, no research has been conducted yet to understand their role and the different trends underlying their reporting activity. We suggest an original approach to studying misinformation: examining it from the reporting users’ perspective at the content-level and comparatively across regions and platforms. We propose the first classification of reported content pieces, resulting from a human review of c. 9,000 items reported on Facebook and Instagram in France, the UK, and the US in June 2020. This allows us to observe meaningful distinctions regarding misinformation propagation between countries and platforms as it significantly varies in volume, type, topic, and manipulation technique. Examining six of these techniques, we identify a novel one that is specific to Instagram US and significantly more sophisticated than others, presenting a concrete challenge for algorithmic detection and human moderation. We also identify four reporting behaviours, from which we derive four types of noise capable of explaining 55% of the inaccuracy in misinformation reporting. We finally show that breaking down the user reporting signal into a plurality of behaviours allows us to build a simple classifier trained on a small dataset with a combination of basic users-reports capable of identifying these different types of noise, thus improving the quality of the reporting signal for misinformation detection.
 
 

Wisdom of Two Crowds: Misinformation Moderation on Reddit and How to Improve this Process---A Case Study of COVID-19

Past work has explored various ways for online platforms to leverage crowd wisdom for misinformation detection and moderation. Yet, platforms often relegate governance to their communities, and limited research has been done from the perspective of these communities and their moderators. How is misinformation currently moderated in online communities that are heavily self-governed? What role does the crowd play in this process, and how can this process be improved? In this study, we answer these questions through semi-structured interviews with Reddit moderators. We focus on a case study of COVID-19 misinformation. First, our analysis identifies a general moderation workflow model encompassing various processes participants use for handling COVID-19 misinformation. Further, we show that the moderation workflow revolves around three elements: content facticity, user intent, and perceived harm. Next, our interviews reveal that Reddit moderators rely on two types of crowd wisdom for misinformation detection. Almost all participants are heavily reliant on reports from crowds of ordinary users to identify potential misinformation. A second crowd--participants' own moderation teams and expert moderators of other communities--provide support when participants encounter difficult, ambiguous cases. Finally, we use design probes to better understand how different types of crowd signals---from ordinary users and moderators---readily available on Reddit can assist moderators with identifying misinformation. We observe that nearly half of all participants preferred these cues over labels from expert fact-checkers because these cues can help them discern user intent. Additionally, a quarter of the participants distrust professional fact-checkers, raising important concerns about misinformation moderation.

The Covid-19 Disinformation Detection on Social Media Using the NLP Approaches

Artificial intelligence has emerged with big data technologies in natural language processing and been applied to creative solutions for overload information especially around the time of the COVID-19 epidemic. This paper provides a comprehensive review of research dedicated to applications of artificial intelligence in misinformation detection. This work organizes the necessary background material for COVID-19-related misinformation detection in NLP, concentrating on the transfer learning technique. Database, data preparation, and modeling make up the major body of information. In the part of modeling, it will merge the attributes of the pre-trained model with the specifical task scenario to explain and present pertinent comments on the future model's improvement under the task scenario. This research will benefit the decision-making and information screen for people's inability to distinguish truth from fiction during the COVID-19 pandemic.

Realtime user ratings as a strategy for combatting misinformation: an experimental study

Because fact-checking takes time, verdicts are usually reached after a message has gone viral and interventions can have only limited effect. A new approach recently proposed in scholarship and piloted on online platforms is to harness the wisdom of the crowd by enabling recipients of an online message to attach veracity assessments to it. The intention is to allow poor initial crowd reception to temper belief in and further spread of misinformation. We study this approach by letting 4000 subjects in 80 experimental bipartisan communities sequentially rate the veracity of informational messages. We find that in well-mixed communities, the public display of earlier veracity ratings indeed enhances the correct classification of true and false messages by subsequent users. However, crowd intelligence backfires when false information is sequentially rated in ideologically segregated communities. This happens because early raters’ ideological bias, which is aligned with a message, influences later raters’ assessments away from the truth. These results suggest that network segregation poses an important problem for community misinformation detection systems that must be accounted for in the design of such systems.