Spread Of Misinformation Research Articles

InfoQ: Certifying credible providers of digital health information

Abstract Background Access to accurate health information is crucial. Despite the abundance of information online, sorting through it can be overwhelming, especially with the spread of misinformation. This poses a threat to public health and challenges health literacy. Efforts to combat misinformation are struggling due to the rapid growth of available information. We need complementary strategies, such as promoting trustworthy information sources. An initiative by Bertelsmann Stiftung, aims to counter misinformation by promoting health information from credible sources. Methods In 2021, the U.S. National Academy of Medicine outlined principles for identifying credible health information sources, endorsed globally by the WHO and embraced by the social media industry. However, these principles lack operationalization, leaving indicators for credibility undefined. To address this gap, Bertelsmann Stiftung, in collaboration with German experts, conducted workshops in 2023 to draft evaluation criteria. Their ongoing work, alongside initiatives by the Careum Foundation and Gesundheit Österreich GmbH will be presented. Results The criteria focus on structure (e.g., the expertise of providers) and process (e.g., methods for identifying relevant evidence) rather than outcome (i.e., the quality of each piece of information), and are based on the domains science-basis, transparency and legitimacy. Mandatory and complementary criteria allow a two-step approach to identify health information providers of good quality (meeting all mandatory criteria) and excellent quality (meeting all criteria). Conclusions These criteria build the basis of the establishment of an international certification system and represent a crucial measure to combat misinformation and promote population health literacy. Various international partners and social platforms have already reached a strong consensus on the relevance of the project goals and expressed their willingness to deepen their efforts in this regard.

The impact of misinformation on social media in the context of natural disasters

Abstract Background Rapid dissemination of information during natural disasters plays a crucial role in managing public responses and ensuring safety. Social media can be a tool to disseminate lifesaving information but is also a ground for the propagation of misinformation. This study examines the impact of misinformation spread through social media in the context of natural disasters and explores related dynamics, effects, and countermeasures. Methods Cochrane and PRISMA recommendations were used for protocol development and sound reporting. Key terms related to misinformation, social media and natural disasters were used, with the search conducted in PubMed, and without time limitation for adequate capture. Only research papers in English were eligible for inclusion. Journal and article quality were considered. No restriction was imposed on country or age of target populations. Results Findings reveal significant influence of misinformation on public behaviour during disasters, often leading to panic and hindering effective disaster management. Specific characteristics of misinformation were identified, incl. its origins and spread mechanisms; proximity -both spatial and social- was found to be a critical factor in terms of spread and reach. Emerging evidence highlights the importance of trusted sources to effectively counter misinformation, yet response time remains a critical challenge. Technical detection of misinformation can help combat it but requires further enhancements for real-time application in diverse disaster scenarios. Conclusions Misinformation on social media poses complex challenges to disaster management, affecting both individual and societal resilience. Proactive communication strategies are critical, as is the development of advanced verification tools, and public education strategies to combat misinformation. Future research should focus on refining strategies to enhance disaster preparedness and response, ensuring a resilient and well-informed public.

Digital Communication Technology is not the Only Cause of Rapid Spread of Misinformation in Society

While achieving technological development in media communication, it is important to recognize the dual nature of technological advancement. This rapid development has led to a great threat of spreading misinformation, where ideas are no longer geographically restricted and spread globally at an unprecedented rate (Wang et al., 2019). Statistics from the Central Statistical Office (CSO) 2021 show that 62% of Internet users have seen untrue or suspicious content online Georgiev (2024). Many people believe that digital communication technologies are the only reason for the accelerated spread of misinformation. On the contrary, I believe that this view ignores the more complex political motivations, social, cultural, and psychological underpinnings. While technology itself is an important agent in the process of spreading misinformation, other factors are also integral. Therefore, I do not agree that the cause lies solely in digital communication technology, and this paper will illustrate how technological and social factors interact with each other through examples.

Mitigating adversarial manipulation in LLMs: a prompt-based approach to counter Jailbreak attacks (Prompt-G)

Large language models (LLMs) have become transformative tools in areas like text generation, natural language processing, and conversational AI. However, their widespread use introduces security risks, such as jailbreak attacks, which exploit LLM’s vulnerabilities to manipulate outputs or extract sensitive information. Malicious actors can use LLMs to spread misinformation, manipulate public opinion, and promote harmful ideologies, raising ethical concerns. Balancing safety and accuracy require carefully weighing potential risks against benefits. Prompt Guarding (Prompt-G) addresses these challenges by using vector databases and embedding techniques to assess the credibility of generated text, enabling real-time detection and filtering of malicious content. We collected and analyzed a dataset of Self Reminder attacks to identify and mitigate jailbreak attacks, ensuring that the LLM generates safe and accurate responses. In various attack scenarios, Prompt-G significantly reduced jailbreak success rates and effectively identified prompts that caused confusion or distraction in the LLM. Integrating our model with Llama 2 13B chat reduced the attack success rate (ASR) to 2.08%. The source code is available at: https://doi.org/10.5281/zenodo.13501821.

REGULAÇÃO LEGAL DAS REDES SOCIAIS: PROTEÇÃO DE DADOS E LIBERDADE DE EXPRESSÃO

The legal regulation of social networks is a field in constant evolution, which seeks to balance two fundamental pillars in the digital era: data protection and freedom of expression. Social networks manage a huge amount of personal data, which makes the existence of legal frameworks that protect users’ privacy imperative. Regulations such as the General Data Protection Regulation (GDPR) in the European Union and the General Data Protection Law (LGPD) in Brazil establish strict standards on how platforms must manage personal information. These laws not only allow users to access, correct or delete their data, but also impose sanctions on companies that do not comply with these requirements, ensuring a high level of security and transparency. At the same time, freedom of expression is a fundamental right that must be protected in the digital environment. However, exercising this right on social media often conflicts with the need to moderate content that may be considered offensive, dangerous or even illegal. Platforms implement moderation policies to mitigate the spread of hate speech, misinformation, and other harmful content. This, however, creates significant challenges in relation to censorship and the delimitation of permitted speech, which can affect the free expression of ideas and opinions. The challenge lies in finding an appropriate balance between protecting users’ privacy and data and enabling robust freedom of expression. An excessive focus on data protection can limit users’ ability to express themselves freely, while unrestricted freedom of expression can put individuals’ security and privacy at risk. As social media becomes more deeply integrated into everyday life and becomes a key platform for communication and the exchange of ideas, it is essential that regulations continue to adapt to address these dilemmas in a balanced and fair way. The future of regulation in this field will depend on the ability of legislators and platforms to adapt to rapid technological and social changes. ensuring that both rights are protected effectively and equitably.

ADA-UDA: A transferable transformer framework for rumor detection using Adversarial Domain Alignment within Unsupervised Domain Adaptation

The rapid spread of misinformation on social media has posed significant challenges, particularly in the early detection of rumors, which is critical to mitigating their negative impact. However, the limited data available during the initial stages of trending topics poses significant challenges, including analysis constraints and model overfitting. Therefore, the adoption of specific methods are needed to identify patterns from historical sample data and enable effective knowledge transfer, facilitating inductive reasoning with limited data on new topics. To address these challenges, we propose an Adversarial Domain Alignment Unsupervised Domain Adaptation (ADA-UDA) method based on the Transformer architecture. Our approach leverages labeled historical data from the source domain alongside a limited quantity of unlabeled data from new trending topics in the target domain. At the heart of our method is the Parameter Transferable Module (PTM), which guides the Transformer to focus on transferable and distinguishable features, thereby enhancing the model’s ability to perform effective inductive reasoning with limited data. We conducted extensive experiments to evaluate our method, benchmarking it against current mainstream rumor detection techniques. The findings indicate that our ADA-UDA method outperforms existing approaches, underscoring its potential for early and accurate rumor detection in emerging topics.

Detection of news written by the ChatGPT through authorship attribution performed by a bidirectional LSTM model

The large language based-model chatbot ChatGPT gained a lot of popularity since its launch and has been used in a wide range of situations. This research centers around a particular situation, when the ChatGPT is used to produce news that will be consumed by the population, causing the facilitation in the production of fake news, spread of misinformation and lack of trust in news sources. Aware of these problems, this research aims to build an artificial intelligence model capable of performing authorship attribution on news articles, identifying the ones written by the ChatGPT. To achieve this goal, a dataset containing equal amounts of human and ChatGPT written news was assembled and different natural processing language techniques were used to extract features from it that were used to train, validate and test three models built with different techniques. The best performance was produced by the Bidirectional Long Short Term Memory (LSTM) Neural Network model, achieving 91.57% accuracy when tested against the data from the testing set.

The Relationship Between Misinformation And Social Noise And Their Impact On The Information Ecosystem

Social noise and social entropy are new concepts modeled after Shannon’s information and communication theory, in which the interference of noise between the sender and receiver is measured using entropy. Social noise in the context of social media plays a vital role in magnifying and spreading misinformation, which in turn impacts the overall information ecosystem. Ecosystems are made of interconnected and integrated parts that rely on one another to maintain balance and survive. Studies related to social noise and misinformation have shown that social noise can contribute significantly to spreading misinformation and potentially alter the original intended message (Alsaid & Pampapura, 2022); Alsaid et al. (2024). paper investigates methods of quantifying social noise using entropy to minimize the spread of misinformation on social media, particularly X. Using a combination of Uncertainty Reduction Theory (URT) and Social Entropy, data analysis was performed using one million tweets harvested from #Ukraine. Data analysis involved several methods: sentiment analyses, term association, network maps, and entropy computation. Results have shown a direct relationship between social noise and social entropy as a measure of uncertainty. Also, social noise and uncertainty decrease with the use of URLs and rich content. It is evident from the results that the entropy value is influenced by the accuracy of keywords identified using topic modeling as descriptive of social noise constructs. Semantic analysis of tweets can help improve the definition of social noise constructs, leading to enhanced and more accurate entropy calculation. Future studies may consider advanced machine learning and AI

The Dual Nature of AI in Information Dissemination: Ethical Considerations.

Infodemics pose significant dangers to public health and to the societal fabric, as the spread of misinformation can have far-reaching consequences. While artificial intelligence (AI) systems have the potential to craft compelling and valuable information campaigns with positive repercussions for public health and democracy, concerns have arisen regarding the potential use of AI systems to generate convincing disinformation. The consequences of this dual nature of AI, capable of both illuminating and obscuring the information landscape, are complex and multifaceted. We contend that the rapid integration of AI into society demands a comprehensive understanding of its ethical implications and the development of strategies to harness its potential for the greater good while mitigating harm. Thus, in this paper we explore the ethical dimensions of AI's role in information dissemination and impact on public health, arguing that potential strategies to deal with AI and disinformation encompass generating regulated and transparent data sets used to train AI models, regulating content outputs, and promoting information literacy.

The impact of confirmation bias awareness on mitigating susceptibility to misinformation

IntroductionIn the current digital age, the proliferation of misinformation presents a formidable challenge to a democratic society. False narratives surrounding vaccination efforts pose a significant public health risk. Understanding the role of cognitive biases in susceptibility to misinformation is crucial in addressing this challenge. Confirmation bias, characterized by the tendency to favor information that aligns with pre-existing beliefs or attitudes, can exacerbate the spread of false narratives.MethodsThis study investigates the effect of confirmation bias awareness on susceptibility to general misinformation. For this, a sample of 1,479 participants was recruited, ensuring diverse representation across attitudes towards vaccination. Half of the participants received targeted information about confirmation bias, aimed at increasing awareness of this bias and its potential impact on cognitive processing of information. The other half did not receive this information.ResultsResults from the study indicated that participants exposed to an intervention aimed at inducing awareness of confirmation bias demonstrated reduced susceptibility to misinformation and increased ability to general discernment of veracity. Notably, these effects were only pronounced among individuals who initially were most negative towards COVID-19 vaccines.DiscussionThese insights provide a foundation for developing targeted strategies to promote informed decision-making and mitigate the spread of misinformation, particularly in the context of public health crises. Further research is warranted to explore the underlying mechanisms driving these effects and to refine intervention approaches for diverse populations and contexts.

Stark Decline in Journalists’ Use of Preprints Postpandemic

The COVID-19 pandemic accelerated the use of preprints, aiding rapid research dissemination but also facilitating the spread of misinformation. This study analyzes media coverage of preprints from 2014 to 2023, revealing a significant postpandemic decline. Our findings suggest that heightened awareness of the risks associated with preprints has led to more cautious media practices. While the decline in preprint coverage may mitigate concerns about premature media exposure, it also raises questions about the future role of preprints in science communication, especially during emergencies. Balanced policies based on up-to-date evidence are needed to address this shift.

Applied Ethical and Explainable AI in Adversarial Deepfake Detection: From Theory to Real-World Systems

Deepfake technology is advancing by the minute. This gives rise to increased privacy, trust and security risks. Such technology can be used for malicious activities like manipulating public opinion and spreading misinformation using social media. Adversarial machine learning techniques seem to be a strong defense in detecting and flagging deepfake content. But the challenge with practical use of many deepfake detection models is that they operate as black-boxes with little transparency or accountability in their decisions. This paper proposes a framework and guidelines to integrate ethical AI and explainable AI (XAI) - specifically techniques like SHAP and LIME, to make deepfake detection systems more transparent and trustworthy. We will propose guidelines to incorporate techniques which will make deepfake detection systems more accountable and explainable such that they make deepfake detection systems seamlessly deployable in the real world.

Ethical Implications of Deepfake Technology

The ethical implications of deepfake technology are profound and demand our immediate attention. As technology rapidly evolves, the risks associated with the misuse and manipulation of digital content become increasingly apparent. Deepfake technology, capable of producing incredibly convincing fake videos and audio recordings, raises alarming concerns about privacy violations, the spread of misinformation, and the potential for malicious exploitation. It is imperative for researchers and policymakers to confront these ethical challenges head-on and implement robust measures to safeguard individuals and society as a whole.

Design Insights for Developing Deepfakes Awareness Games for Adolescents

Around a decade ago, significant advancements in the deep learning domain like Generative Adversarial Networks (GANs) started to facilitate the creation of new synthetical content. The development of deepfake algorithms gained momentum due to the increase in computing power and open-source available resources which facilitated broader access to advanced deep learning methods. While initially deepfakes were created by creative pursuits in various media and marketing campaigns, they started to be misused driven by malicious purposes in various social media manipulation campaigns. These campaigns imply spreading disinformation and misinformation for discrediting individuals and creating confusion in relation to specific topics and events and are meant to altering users’ beliefs and behaviours. Such facts raised important ethical and societal concerns. Although deepfakes still represent a recent research area, a large body of studies is dedicated to, on the one hand, generation, and detection of deepfakes, and on the other hand, to understanding their implications and consequences for society. At the same time, governmental and practitioner efforts are devoted to containing their use and impact on society by proposing various strategies and programs. Nevertheless, both academic and societal efforts are in an incipient stage and often adopt a generalist perspective. This represents a crucial point in relation to the fact that many users in various social media platforms like TikTok is represented by adolescents that are known to be unaware and more vulnerable to media content in general, and deepfakes content, in particular. This represents the knowledge gap that this research aims to tackle by synthesizing design insights as requirements and guidelines that can be used when developing and deploying deepfakes awareness games for producing and/or enhancing awareness of adolescents. Accordingly, a systematic literature review is conducted and merged with previous experience in this domain by adopting a transdisciplinary research approach that merges methods and techniques from the deepfakes, AI, social media, cyber security, and gamification domains.

Fighting Misinformation on Social Media: An Empirical Investigation of the Impact of Prominence Reduction Policies

Misinformation has dire implications for both public welfare and the operational aims of user-generated content platforms. As a result, platforms have adopted various content moderation policies aimed at decreasing the volume and impact of misinformation. However, implementing new platform policies runs the risk of decreasing user contribution and alienating core users, and results regarding the efficacy of such policies are mixed. Herein, we empirically assess a prominence reduction policy applied to a problematic group that is high in misinformation. The goal of this policy is to reduce the visibility of misinformation on the platform (rather than deleting misinformation or banning users). The results show that while prominence reduction diminishes misinformation dissemination in the focal group, this method also results in a spillover of misinformation to topically related spaces. This spillover is short-lived and driven primarily by a small set of problematic users. As misinformation is not contagious, we find that this spillover to external groups diminishes over time. Finally, prominence reduction is found to have no impact on non-misinformation contribution on the studied platform. The findings of this study have important implications for platform operations and provide useful recommendations for managers regarding effective ways to reduce the spread of misinformation.

Misinformation spreading on activity-driven networks with heterogeneous spreading rates.

The spread of misinformation on social media is inextricably related to each user's forwarding habits. In this paper, given that users have heterogeneous forwarding probabilities to their neighbors with varied relationships when they receive misinformation, we present a novel ignorant-spreader-refractory (ISR) spreading model with heterogeneous spreading rates on activity-driven networks with various types of links that encode these differential relationships. More exactly, in this model, the same type of links has an identical spreading rate, while different types of links have distinct ones. Using a mean-field approach and Monte Carlo simulations, we investigate how the heterogeneity of spreading rates affects the outbreak threshold and final prevalence of misinformation. It is demonstrated that the heterogeneity of spreading rates has no effect on the threshold when the type of link follows a uniform distribution. However, it has a significant impact on the threshold for non-uniform distributions. For example, the heterogeneity of spreading rates increases the threshold for normal distribution while it lowers the threshold for an exponent distribution. In comparison to the situation of a homogeneous spreading rate, whether the heterogeneity of spreading rates improves or decreases the final prevalence of misinformation is also determined by the distributions of the type of links.

Türkiye’de Covid-19 Salgınında Halkla İlişkiler Kampanyaları ve Gebelerin Aşı Olma Tutumları

Vaccines have been produced and released in different countries to prevent the risk of transmission of Covid-19, which is shown as the biggest global pandemic of the era, and to eliminate the pandemic situation. Public relations campaigns to promote vaccines to the global public is an issue that deserves academic scrutiny. The possible side effects of vaccines according to the age and health status of individuals and the fact that they have not been tried before have brought along doubts in the public opinion. The misinformation spread through communication tools about these doubts has also caused fear and panic in the society. The intensive production of content on social media during the pandemic period has made the dissemination of misinformation and distortion of information on the subject prominent. Although scientific data and findings have shown that vaccination is important, public panic has been tried to be dampened by public relations campaigns. Considering that vaccines are seen as a threat as much as the pandemic, especially for pregnant women, the importance of examining the role of public relations campaigns in accurate information and crisis management can be understood. For this purpose, it is aimed to reveal how public relations campaigns are perceived in individuals' decision-making and vaccination tendencies. In the study, qualitative design method was preferred and data were collected through semi-structured in-depth interviews. For this purpose, the tendency of pregnant women to be vaccinated was questioned in interviews with 5 gynecologists from different districts of Ankara. Taking regional differences into account, the perspectives of health personnel on public relations campaigns were tried to be revealed. The study was conducted between March and June 2023. The study focuses on the concept of opinion leader of the two-stage flow model used in communication studies and focuses on how ideas about vaccination spread from opinion leaders to the society. In addition, the study utilizes social identification theory, which conducts research to understand how social norms are formed, change and function.

Medical misinformation and fear of recurrence in patients with breast cancer.

354 Background: Fear of recurrence is a common yet underappreciated burden patients experience during and after cancer treatment. Medical misinformation about factors that influence risk of cancer progression/recurrence is also common. Little is known about the spread of medical misinformation and how it affects fear of cancer recurrence. Methods: From 7/2023 to 8/2023 patients completed an anonymous online survey administered by Breastcancer.org. Surveys were available in English and Spanish with eligibility: US resident, age ≥18 years, and breast cancer diagnosis within 10 years. Surveys assessed patient awareness of various factors purported to increase or decrease risk of cancer progression/recurrence and fear of recurrence (via validated FCRI-SF, scored 0-36, higher scores indicate greater fear, score ≥22 indicate clinical relevance). Multivariate analysis (MVA) assessed awareness of misinformation after controlling for age, race and ethnicity, marital status, and level of education. Chi-square assessed association between misinformation exposure and fear of recurrence. Results: Overall, 997 completed the survey. Median age was 62 (IQR 53-69), 86% were White, 5% Black, 4% Hispanic, and 3% Asian. 76% were married and 77% had completed a college or advanced degree. 52% were undergoing active cancer treatment at the time of assessment. Most (65%) reported hearing misinformation regarding factors that may INCREASE risk of cancer progression/recurrence. Specifically, patients heard that sugar (61%), deodorant (22%), vaccines (8%), cell phones (8%), and bra type (8%) could increase risk. Half (54%) reported hearing misinformation regarding factors that may DECREASE risk of cancer progression/recurrence. Specifically, patients heard that eating organic food (41%), oral vitamins/supplements (29%), alkaline diet/alkalized water (12%), vitamin infusions (7%), cleanses (5%), essential oils (5%), and oxygen therapy (4%) could decrease risk. Overall, 76% had reported at least one incorrect factor influencing risk of cancer progression/recurrence. On MVA, younger age (OR 0.99/year of age, 95%CI 0.97-1.00, p=0.044) and Hispanic ethnicity (OR 2.98, 95%CI 1.02-8.25, p=0.047) were associated with hearing misinformation. Fear of recurrence score was available for 695 (70%) patients. Median score was 19 (IQR 13-24), 38% had a clinically significant level of fear (≥22). Awareness of medical misinformation was not associated with clinically significant fear (38% vs. 35%, χ 2 0.53, p=0.469). Conclusions: In a large cohort of patients with breast cancer, medical misinformation about risk of progression/recurrence was common. Fear of recurrence was high in over a third of surveyed patients, but was not associated with awareness of false information. Further understanding of how patients perceive and process medical misinformation is essential, especially in populations at highest risk for misinformation spread.

Combating Misinformation on Social Media Using Social Noise and Social Entropy as a Measure of Uncertainty

ABSTRACTRecent events around the world, including the war in Ukraine and the conflict in Gaza have highlighted the effective use of social media as a tool to voice concerns about social issues to create awareness. At the same time, social media has become a fertile ground for spreading misinformation, fake news, and conspiracy theories. Misinformation and conspiracy theories have existed since the existence of mankind. What is new today is the speed by which misinformation can be created, magnified and spread using social media. Efforts to regulate social media and control the widespread spread of misinformation are still lacking due to rapid advances in technology and concerns regarding free speech. One approach to minimizing the impact of misinformation is to focus on social noise as an important factor in magnifying and spreading misinformation. In this paper, we investigate methods of identifying and quantifying social noise using social entropy as a measure of uncertainty and topic modeling. Results from the study have shown a direct relationship between social noise and social entropy. The results have also shown that social noise and social entropy decrease with the use of URLs and rich content (sematic information). Further studies will include the use of machine learning and AI techniques to improve the definition of social news and social entropy.

DSViT: An Enhanced Transformer Model for Deepfake Detection

The rapid development of artificial intelligence and deep learning models has enabled the creation of highly realistic fake images and videos, posing significant threats to information security and safety. Accurate detection of these forged contents is crucial to prevent the spread of misinformation and to protect the integrity of digital media. Although several advanced studies in this field, such as Vision Transformer (ViT) and Convolutional Vision Transformer (CViT), have been conducted, there remain limitations that need to be addressed. In this paper, we introduce a novel model, improved from CViT, designed to optimize the process of deepfake detection, named DSViT (Deepfake Detection with SC-based Convolutional Vision Transformer). This model judiciously integrates Convolutions and a SCConvolution block with the ViT architecture. We conducted experiments on the Deepfake Detection Challenge (DFDC) dataset and compared the results with the CViT model to demonstrate the effectiveness of the proposed model