Extremist Content Research Articles

The Private Life of QAnon: A Mixed Methods Investigation of Americans' Exposure to QAnon Content on the Web

The QAnon movement has been credited with spreading disinformation and fueling online radicalization in the United States and around the globe. While some research has documented publicly-visible communications and engagements with the QAnon movement, little work has examined individuals' actual exposure to QAnon content. In this paper, we investigate the extent to which Americans are exposed to QAnon websites, in what contexts, and to what effect. We employ a mixed methods review of 21 million website visits collected from a nationally representative sample of 1,238 American adults across laptops, smartphones, and tablets during the 2020 U.S. presidential election. Quantitative techniques reveal overall levels of exposure to QAnon and who is more likely to be exposed, and qualitative techniques provide rich information about how participants came to be exposed to QAnon and how it fit within their broader media diets. We find that: (1) exposure to QAnon websites is limited and stratified by political ideology and news consumption; (2) exposure tends to occur within right-wing media ecosystems that align with QAnon beliefs; and (3) mixed methods approaches to analyzing digital trace data can provide rich insights that contextualize quantitative techniques. We discuss the implications of our findings for the design of interventions to lessen exposure to problematic material online and future research on the spread of disinformation and extremist content.

The Role of Religious Education in Dealing with the Issue of Radicalization among Students

Radicalization has become a serious threat in the world of education, especially among students who are in the developing stage of self-identity search, where curiosity and emotional instability make them vulnerable to the influence of radical ideology. Religious education, in this context, plays a central role in shaping students' character by instilling values of moderation, tolerance, and respect for diversity. This study aims to explore the role of religious education in preventing radicalization, using a descriptive qualitative method with data collection techniques through in-depth observation and interviews involving teachers, students, principals, and parents. The focus of the research is on the implementation of religious moderation values in schools as an effort to build students' critical attitudes towards extremist ideologies. The results of the study show that inclusive, interactive, and collaborative teaching approaches in religious education, both in the classroom and by involving the external community, play an important role in fostering tolerant characters and being able to reject radical narratives. The study highlights the importance of collaboration between schools and families, as well as the need for digital literacy education to strengthen students' resilience to exposure to extremist content on social media. The implications of this study include the need to integrate materials relevant to actual issues in the religious education curriculum, as well as the development of community-based educational programs that involve families and communities as additional bulwarks of protection for students.

Unveiling Extremism: Leveraging Digital Data Mining Strategies

In the theoretical exploration of "Unveiling Extremism: Leveraging Digital Data Mining Strategies," the intricate web of extremist behavior is dissected through the lens of digital data mining. By utilizing advanced computational techniques, this study delves into the depths of online platforms, analyzing patterns, sentiments, and interactions to uncover the underlying mechanisms of extremism. The focus lies not only on identifying extremist content but also on understanding the processes that lead individuals towards radicalization. Through theoretical modeling and simulation, this research seeks to map out the pathways of radicalization, shedding light on the factors that contribute to the formation and spread of extremist ideologies. Moreover, the study examines the efficacy of various digital data mining strategies in detecting and countering extremism, proposing innovative approaches to enhance the effectiveness of online monitoring and intervention. Ultimately, this theoretical exploration serves as a foundation for developing proactive measures to combat extremism in the digital age, offering insights that can inform policy-making, intervention programs, and the design of online platforms.

LINGUISTIC MARKERS OF EXTREMISM IN ISLAMIST TEXTS OF THE RELIGIOUS GENRE “SERMON”: METHODOLOGICAL ASPECT

Linguistic examination of texts with allegedly extremist content is carried out using special methods. The most popular today are lexical-semantic, semantic-syntactic methods, as well as the method of discourse anal-ysis, which allow an expert linguist to make an objective expert opinion on the issues raised. An expert linguist conduc ting an examination of texts with allegedly extremist content has to identify linguistic markers of extrem-ism that are implemented at all language levels. The linguistic examination of Islamist discourse texts involves taking into account genre features of a religious text, having an impact on linguistic markers. Islamist discourse is understood as texts related to political radical trends of Islamism. The article presents fundamental diff erenc-es between the terms “Islam” as a world religion and “Islamism” as a political trend. We propose an algorithm for the me thodology of analyzing texts of allegedly extremist content based on the material of an Islamist text of the “sermon” genre, the main function of which is to instruct believers. Linguistic markers of extremism in the sermon genre are identifi ed during the analysis and are classifi ed according to their main functions: motivation and appeal, negation, demonstration of the superiority of one person or group of persons over other people based on nationality, origin and attitude to religion. These linguistic markers are implemented using the following lin-guistic means: morphological (verbs of obligation, verbs of the 2nd person singular or plural in the imperative mood, the form of the imperative of the 1st person plural); syntactic (particles in imperative sentences, complex sentences with subordinate clauses, addresses in combination with verbs in the imperative mood of the 2nd per-son plural); rhetorical means (rhetorical questions, antithesis); lexical and semantic means (nouns, verbs and oth-er parts of speech with negative connotations); morphemic and word-formation means (comparative, superlative forms). The identifi ed linguistic markers realize the function of demonstrating superiority, appeal, negativity in relation to “strangers”, “others”.

Prevalence, Viewpoints, and Encounters with Cyberterrorism among College Students

This research aims to describe prevalence, perspective on cyberextremism and encounters of cyberextremism among college students. A sample of 1000 Tafila Technical University (TTU) students ) representing 12 Jordanian governates, found that 45.4% were males and 54.6% were females from 12 Jordanian governorates. Science colleges and arts and social colleges split the sample almost equally (49.6% vs. 50.4%), respectively. All students are using the internet, and most of them use it intensively (73%), compared to regular use (27%). The study found that less than half of the sample described online extremism as hate speech, violence, cyberbullying, sexual pornography, indirect hate speech, assaults, and post-support extremism. Students perceived several procedures to encounter cyberextremism, including closing websites, fines, criminalizing content, holding websites responsible for compensation, establishing a minimum age for viewing extremist content, removing hardening material, and compensation. Males had a higher mean of cyberextremism than females. ANOVA analysis showed significant differences between external attribution and cyberextremism, internal attributions, and no significant differences between males and females.

Angry, anxious, and online: The role of reinforcement sensitivity theory and trait aggression on the impact of exposure to violent extremist content online.

Angry, anxious, and online: The role of reinforcement sensitivity theory and trait aggression on the impact of exposure to violent extremist content online.

Supplemental Material for Angry, Anxious, and Online: The Role of Reinforcement Sensitivity Theory and Trait Aggression on the Impact of Exposure to Violent Extremist Content Online

Supplemental Material for Angry, Anxious, and Online: The Role of Reinforcement Sensitivity Theory and Trait Aggression on the Impact of Exposure to Violent Extremist Content Online

Method for Constructing Neural Network Means for Recognizing Scenes of Political Extremism in Graphic Materials of Online Social Networks

Countering the spread of calls for political extremism through graphic content on online social networks is becoming an increasingly pressing problem that requires the development of new technological solutions, since traditional approaches to countering are based on the results of recognizing destructive content only in text messages. Since in modern conditions neural network tools for analyzing graphic information are considered the most effective, it is assumed that it is advisable to use such tools for analyzing images and video materials in online social networks, taking into account the need to adapt them to the expected conditions of use, which are determined by the wide variability in the size of graphic content, the presence of typical interference, limited computing resources of recognition tools. Using this thesis, a method has been proposed that makes it possible to implement the construction of neural network recognition tools adapted to the specified conditions. For recognition, the author's neural network model was used, which, due to the reasonable determination of the architectural parameters of the low-resource convolutional neural network of the MobileNetV2 type and the recurrent neural network of the LSTM type, which makes up its structure, ensures high accuracy of recognition of scenes of political extremism both in static images and in video materials under limited computing conditions resources. A mechanism was used to adapt the input field of the neural network model to the variability of the size of graphic resources, which provides for scaling within acceptable limits of the input graphic resource and, if necessary, filling the input field with zeros. Levelling out typical noise is ensured by using advanced solutions in the method for correcting brightness, contrast and eliminating blur of local areas in images of online social networks. Neural network tools developed on the basis of the proposed method for recognizing scenes of political extremism in graphic materials of online social networks demonstrate recognition accuracy at the level of the most well-known neural network models, while ensuring a reduction in resource intensity by more than 10 times. This allows the use of less powerful equipment, increases the speed of content analysis, and also opens up prospects for the development of easily scalable recognition tools, which ultimately ensures an increase in security and a reduction in the spread of extremist content on online social networks. It is advisable to correlate the paths for further research with the introduction of the Attention mechanism into the neural network model used in the method, which will make it possible to increase the efficiency of neural network analysis of video materials.

Break the Hate: A Survey of Youth Experiences with Hate and Violent Extremism Online

As the most prolific users of the Internet, youth are exposed to a diverse array of harmful content and experiences, including cyberbullying and sexual exploitation. What is less well understood is the impact of hate and violent extremism on youth in these online spaces. This study surveyed over 800 youth from Alberta, Canada, to identify where they most frequently encountered hateful and extremist content online, how they react to it, and what they believed were the most appropriate responses to these problems. This study adds to a growing literature which takes youth perspectives seriously in the study of this problem. Our study found that more than three-quarters of youth surveyed reported encountering hateful content, while more than two-thirds reported encountering extremist content. Our findings add to a growing debate on the relationship between identity factors and exposure. While our results indicate respondents who identify as female are more likely to report encountering extremist and hateful content than males, intersectionality factors shed new light on the patterns of online exposure among youth. Specifically, we found that the effect of gender is mediated by other identity factors, like being a visible minority or identifying as 2SLGBTQ+.

Digital Space as a Scene of Extremist Crimes

Innovations in digital communications determined a trend for a sustainable causal link between technological diversity and qualitative changes in the mechanism of extremist crimes. Essentially, the processes of global informatization led to a paradigm shift, and the methods of transforming social behavior based on digital communications and social engineering technologies became some of the key elements in influencing the minds of the objects of extremist indoctrination. Due to this, it is necessary to offer a forensically grounded definition of digital space as a scene where extremist crimes are committed with the purpose of developing recommendation for identifying, capturing and systematizing evidentiary and other forensically relevant information, developing a forensic characteristic and then using it to understand the mechanism of suchlike criminal acts. The methodological basis for the research is the general dialectic method of research cognition. Besides, some general and special research methods were used: systemic-structural, formal-logical, statistical, specific-sociological, analysis and synthesis, induction and deduction, analogy. Data consolidation and modelling were also carried out. According to the author, in order to understand the mechanism of trace formation in the digital environment, it is necessary to identify changes resulting from the electromagnetic interactions between the signal and the environment in which the signal was processed, and to understand what factors influenced the process of electromagnetic interactions. Consequently, in order to identify, capture and establish a link between digital traces and electromagnetic interactions, it is necessary to establish a crossover mutual relationship between a computer device, a digital infrastructure element, as well as an event of interest to the subject of the investigation or an action that has left its reflection. For the extremist crimes committed in the digital space, it is expedient that the crime scene should be interpreted as the place where key criminal extremist actions were committed. Essentially, it is the place where extremist content was uploaded, the information and telecommunication network of the digital space was entered, and the criminal action of extremist nature was then committed, regardless of the place where its consequences happened.

Psychology of the Network Crowd as a Subject of Political Extremism

The purpose of the article is to describe the psychological characteristics of the network crowd and their use to disseminate the ideology of political extremism, as well as to propose recommendations on countering these technologies. Opening the resources of foreign social networks for its citizens, Russia has become a hostage to unspoken rules. The problem with stopping the wave of extremist content demonstrates a serious threat to ensuring information sovereignty and emphasizes the relevance and significance of studying this problem. The methods of interdisciplinary theoretical analysis, synthesis, comparison, generalization and conceptualization of the scientific ideas of the use of the psychological effects of the crowd in the network space, the content of the analysis of the posts of participants in telegram channels during protest actions were used. The article develops an author’s approach to understanding the phenomenon of the network crowd as a specific hybrid that has arisen on the basis of social networks, embodying elements of an organized nucleus and spontaneous periphery. The idea of the mechanism of control of an organized network crowd as a function from the composition of the content to create emotional resonance, the activities of groups that disperse information on social networks and infection leaders, involving participants in social networks with their emotional response, is justified. Conclusions are drawn about the conditions under which the network crowd turns into a threat. Recommendations are proposed to counteract the indication of participants in social networks by the ideology of extremism.

Alt‐health influencers and the threat of social media deplatforming

AbstractThe threat of deplatforming as a content moderation strategy works to reduce harmful anti‐vaccine content and health mis/disinformation on particular social media platforms. However, through an analysis of alt‐health influencers, I show how the perpetual threat of being deplatformed allows them to strategically grow their audiences while funneling users to fringe platforms where the alt‐health influencers share anti‐vaccine and extremist content. Alt‐health influencers are thus able to “launder” their reputations and their content through mainstream social media platforms, acting in accordance with each platform's specific content moderation policies, while persuasively deploying “deep stories” and developing parasocial relationships across mainstream and fringe platforms, building an alt‐health influence network across platforms, and using threats of “censorship” on one platform to sell and legitimize their statuses as truth‐tellers and informational authorities on another. The threat of deplatforming aids in the spreading of health mis/disinformation across social media platforms, demonstrating the necessity of addressing well‐known alt‐health influencers who are detrimental to the health of the overall information ecosystem through decisive, consistent, and multi‐platform‐coordinated deplatforming.

Monitoring and Sorting of Radical and Extremist Content in Social Network Messages using Regression Learning-Based Bayesian Regularized ANN

Monitoring and Sorting of Radical and Extremist Content in Social Network Messages using Regression Learning-Based Bayesian Regularized ANN

Taking it to the extreme: prevalence and nature of extremist sentiment in games.

More than half of all game players report experiencing some form of hate, harassment or abuse within gaming spaces. While prevalence assessments of these actions in digital gaming spaces are ongoing, little remains known about the more extreme forms of these behaviors. Specifically, experiences of extremism. This paper addresses the gap in research knowledge around the expression of extremist sentiment in games by evaluating their prevalence, location, and nature, and impact. Assessing experiences via an online survey, game players (n = 423) reported an alarmingly high rate of frequency for being the direct target of, as well as a witness to, all forms of extremist content. Most of these experiences were text-based, reported to be happening in-game. Most players endorsed statements relating to a normalization of extreme ideologies within gaming cultures. It is promising that reporting these behaviors was the primary action taken by players for most of the players; however, "ignoring" these actions was also a common strategy. It is possible that player inaction reflects the embeddedness and normalization of these actions in gaming spaces and/or a lack of trust in moderation systems to be responsive. The prevalence of extreme sentiment in gaming cultures should raise concern from game makers, members of the gaming community, parents, and policy makers alike.

Unleashing Transformative Potential of Artificial Intelligence (AI) in Countering Terrorism, Online Radicalisation, Extremism, and Possible Recruitment

This research explores the implications of Artificial Intelligence (AI) in deciphering the multifarious manifestations of online Radicalisation, Extremism, and possible recruitment. It analyses AI's capabilities in disseminating positive representations and counter-narratives, its instrumental role in analysing positive imagery, optimising platform reach, and its pivotal contribution to global efforts to combat online extremism. However, AI's dualistic nature underscores the need for its ethical and responsible utilisation to curb extremist content effectively. It necessitates the development of balanced, nuanced, and coherent approaches, amalgamating enhanced analytical paradigms, collaborative efforts, ethical frameworks, strategic insight, and diversified applications. The formulation of such multifaceted approaches and cohesive strategies is paramount in navigating the intricate terrains of online extremism and in enhancing societal resilience against the proliferation of extremist ideologies. The study concludes that the strategic and ethical deployment of AI technologies is pivotal in reshaping digital discourse and in the collective endeavour to counterbalance extremist ideologies.

Watering down the wine: European Union regulation of violent right‐wing extremism content and the securitisation of new online spaces

AbstractThe regulation of violent right‐wing extremist online content in the European Union is facing a conundrum. On the one hand, pressure to respond to concern over violent right‐wing extremism (or VRWE, a term used by the European Union to refer to ‘far right’) has led to a push to develop a coherent European‐wide response. On the one hand, this has led to a watering down or ‘loosening’ of the concept and implementation, to come to policy agreements between Member States. On the other hand, we have also seen a broadening and expansion of key concepts around VRWE to include more online practices. We argue that this combination of ‘loosening’ the definition of far right, whilst simultaneously ‘broadening’ what constitutes far‐right content, risks the securitisation of vast online practices ‐ muddying the means for developing coherent and effective responses to VRWE in online spheres. This paper aims to explore the framing and securitisation of violent right‐wing extremist content, by tracking the discursive construction of European content moderation policy tools over time.

Moderating borderline content while respecting fundamental values

AbstractAs efforts to identify and remove online terrorist and violent extremist content have intensified, concern has also grown about so‐called lawful but awful content. Various options have been touted for reducing the visibility of this borderline content, including removing it from search and recommendation algorithms, downranking it and redirecting those who search for it. This article contributes to this discussion by considering the moderation of such content, in terms of three sets of values. First, definitional clarity. This is necessary to provide users with fair warning of what content is liable to moderation and to place limits on the discretion of content moderators. Yet, at present, definitions of borderline content are vague and imprecise. Second, necessity and proportionality. While downranking and removal from search and recommender algorithms should be distinguished from deplatforming, tech companies' efforts to deamplify borderline content give rise to many of the same concerns as content removal and account shutdowns. Third, transparency. While a number of platforms now publish their content moderation policies and transparency data reports, these largely focus on violative, not borderline content. Moreover, there remain questions around access to data for independent researchers and transparency at the level of the individual user.

Recommending instead of taking down: YouTube hyperpartisan content promotion amid the Brazilian general elections

AbstractYouTube released a global commitment to reduce the spread of problematic content by actively recommending “trusted” news sources on their platform but did not disclose the criteria used to classify the reputation of channels and the list of trusted news sources favored by the company. We aim to understand the patterns of recommendation by YouTube's algorithm and identify what the platform classifies as “reliable informative sources.” In our experiment, we performed 205 tests on YouTube's recommendation system, simulating the behavior of new Brazilian users with no historical data on the platform looking for reliable news throughout the elections. We documented the Brazilian media groups that were suggested on the homepage, further analyzing the presence, position and content of Jovem Pan videos, Brazil's flagship conservative media group. We identified that, far from reigning in extremist content, YouTube systematically prioritized Jovem Pan and did not actively take down toxic Jovem Pan content. Our findings indicate that YouTube magnified the imbalance between candidates' portrayals, exposing the regulatory asymmetry between broadcast media and online platforms in the country. In spite of the opaque commercial agreements and business strategies, the platform's recommendation decisions may have an “advertising effect” of sponsorship and endorsement on users.

A Hybrid Technique for Detecting Extremism in Arabic Social Media Texts

Today, social media sites like Twitter provide effective platforms to share opinions and thoughts in public with millions of other users. These opinions shared on such sites influence a large number of people who may easily retweet them and accelerate their spread. Unfortunately, some of these opinions were expressed by extremists who promoted hateful content. Since Arabic is one of the most spoken languages, it is crucial to automate the process of monitoring Arabic content published on social sites. Therefore, this study aims to propose a hybrid technique to detect extremism in Arabic social media texts and articles to monitor the situation of published extremist content. The proposed technique combines the lexicon-based approach with the rough set theory approach. The rough set theory is employed with two approximation strategies: lower approximation and accuracy approximation. The hybrid technique used the rough set theory as a classifier and the lexicon-based as a vector. Furthermore, this study built three types of corpuses (V1, V2, and V3) collected from Twitter. The experimental findings show that among the proposed hybrid methods, the accuracy approximation was superior to the lower approximation with seed vector. It was also revealed that hybrid methods outperformed machine learning techniques in terms of efficiency. Moreover, the study recommends using an accuracy approximation method with seed vector to identify the polarity of the text.

Web-ресурстардағы экстремисттік мәліметтерді анықтауға арналған машиналық әдістерді оқыту және сынау үшін қазақ тіліндегі мәтіндер корпусын құру

The article deals with the issue of creating a corpus in the Kazakh language, which is necessary to create a semantic model for identifying extremist information on web resources. As a result of the study, text data was collected from closed groups of the VKontakte social network, YouTube comments and news sites using previously developed analytics. The texts were classified into two classes: «extremist» and «neutral». A corpus for further training of models was created. The total number of extremist messages included in the corpus is about 1,200, and the total number of words in the corpus is about 140,000 words. The distribution of extremist and neutral texts was carried out, the analysis of the corpus was carried out using a word cloud. The Python 3.7 programming language with pandas, numpy, matplotlib, plotly, bokeh, cufflinks, spacy, googletrans packages was used as the main calculation and visualization libraries for data analysis. Signs of texts of extremist content were identified.