Hate Speech Detection Research Articles

EnML: Multi-label Ensemble Learning for Urdu Text Classification

Exponential growth of electronic data requires advanced multi-label classification approaches for the development of natural language processing (NLP) applications such as recommendation systems, drug reaction detection, hate speech detection, and opinion recognition/mining. To date, several machine and deep learning–based multi-label classification methodologies have been proposed for English, French, German, Chinese, Arabic, and other developed languages. Urdu is the 11th largest language in the world and has no computer-aided multi-label textual news classification approach. Unlike other languages, Urdu is lacking multi-label text classification datasets that can be used to benchmark the performance of existing machine and deep learning methodologies. With an aim to accelerate and expedite research for the development of Urdu multi-label text classification–based applications, this article provides multiple contributions as follows: First, it provides a manually annotated multi-label textual news classification dataset for the Urdu language. Second, it benchmarks the performance of traditional machine learning approaches particularly by adapting three data transformation approaches along with three top-performing machine learning classifiers and four algorithm adaptation-based approaches. Third, it benchmarks performance of 16 existing deep learning approaches and the four most widely used language models. Finally, it provides an ensemble approach that reaps the benefits of three different deep learning architectures to precisely predict different classes associated with a particular Urdu textual document. Experimental results reveal that proposed ensemble approach performance values (87% accuracy, 92% F1-score, and 8% hamming loss) are significantly higher than adapted machine and deep learning–based approaches.

T5 for Hate Speech, Augmented Data, and Ensemble

We conduct relatively extensive investigations of automatic hate speech (HS) detection using different State-of-The-Art (SoTA) baselines across 11 subtasks spanning six different datasets. Our motivation is to determine which of the recent SoTA models is best for automatic hate speech detection and what advantage methods, such as data augmentation and ensemble, may have on the best model, if any. We carry out six cross-task investigations. We achieve new SoTA results on two subtasks—macro F1 scores of 91.73% and 53.21% for subtasks A and B of the HASOC 2020 dataset, surpassing previous SoTA scores of 51.52% and 26.52%, respectively. We achieve near-SoTA results on two others—macro F1 scores of 81.66% for subtask A of the OLID 2019 and 82.54% for subtask A of the HASOC 2021, in comparison to SoTA results of 82.9% and 83.05%, respectively. We perform error analysis and use two eXplainable Artificial Intelligence (XAI) algorithms (Integrated Gradient (IG) and SHapley Additive exPlanations (SHAP)) to reveal how two of the models (Bi-Directional Long Short-Term Memory Network (Bi-LSTM) and Text-to-Text-Transfer Transformer (T5)) make the predictions they do by using examples. Other contributions of this work are: (1) the introduction of a simple, novel mechanism for correcting Out-of-Class (OoC) predictions in T5, (2) a detailed description of the data augmentation methods, and (3) the revelation of the poor data annotations in the HASOC 2021 dataset by using several examples and XAI (buttressing the need for better quality control). We publicly release our model checkpoints and codes to foster transparency.

Detection of Hateful Social Media Content for Arabic Language

Social media is a common medium for expression of views, discussion, sharing of content, and promotion of products and ideas. These views are either polite or obscene. The growth of hate speech is one of the negative aspects of the medium and its emergence poses risk factors for society at various levels. Although there are rules and laws for these platforms, they cannot oversee and control all types of content. Thus, there is an urgent need to develop modern algorithms to automatically detect hateful content on social media. Arab society is not isolated from the world, and the usage of social media by its members has highlighted the importance of automated systems that help build an electronic society free of hate and aggression. This article aims to detect hate speech based on Arabic context over the Twitter platform by proposing different novel deep learning architectures in order to provide a thorough analytical study. Also, a comparative study is presented with a different well-known machine learning algorithm, as well as other state-of-the-art algorithms from the literature to be used as a beacon for interested researchers. These models have been applied to the Arabic tweets dataset, which included 15K tweets and 14 features. After training these models, the results obtained for the top two models included an improved bidirectional long short-term memory with an accuracy of 92.20% and a macro F1-score of 92% and a modified convolutional neural network with an accuracy of 92.10% and a macro F1-score of 91%. The results also showed the superiority of the performance of the deep learning models over other models in terms of accuracy.

MetaHate: AI‐based hate speech detection for secured online gaming in metaverse using blockchain

AbstractThe emergence of Web 3.0, blockchain technology (BC), and artificial intelligence (AI) are transforming multiplayer online gaming in the metaverse. This development has its concerns about safety and inclusivity. Hate speech, in particular, poses a significant threat to the harmony of these online communities. Traditional moderation methods struggle to cope with the immense volume of user‐generated content, necessitating innovative solutions. This article proposes a novel framework, MetaHate, that employs AI and BC to detect and combat hate speech in online gaming environments within the metaverse. Various machine learning (ML) models are applied to analyze Hindi–English code mixed datasets, with gradient boosting proving the most effective, achieving 86.01% accuracy. AI algorithms are instrumental in identifying harmful language patterns, while BC technology ensures transparency and user accountability. Moreover, a BC‐based smart contract is proposed to support the moderation of hate speech in the game chat. Integrating AI and BC can significantly enhance the safety and inclusivity of the metaverse, underscoring the importance of these technologies in the ongoing battle against hate speech and in bolstering user engagement. This research emphasizes the potential of AI and BC synergy in creating a safer metaverse, highlighting the need for continuous refinement and deployment of these technologies.

LITHUANIAN HATE SPEECH CLASSIFICATION USING DEEP LEARNING METHODS

The ever-increasing amount of online content and the opportunity for everyone to express their opinions online leads to frequent encounters with social problems: bullying, insults, and hate speech. Some online portals are taking steps to stop this, such as no longer allowing user-generated comments to be made anonymously, removing the possibility to comment under the articles, and some portals employ moderators who identify and eliminate hate speech. However, given the large number of comments, an appropriately large number of people are required to do this work. The rapid development of artificial intelligence in the language technology area may be the solution to this problem. Automated hate speech detection would allow to manage the ever-increasing amount of online content, therefore we report hate speech classification for Lithuanian language by application of deep learning.

Bibliometric Analysis of Deep Learning for Social Media Hate Speech Detection

Social media has become an important web technology for creating and sharing information plus enhancing business reputations worldwide. However, the anonymity accorded by social media platforms has been cryptically vituperated to spread horrendous content such as hate speech. Recently, researchers have been progressively gravitating towards the use of deep learning techniques to address the problem of social media hate speech detection. This study provides bibliometric analysis and mapping of the existing literature on hate speech detection using deep learning algorithms. The study used articles published between 2016 and 2022 from the Scopus database, while Vos Viewer, Biblioshiny, and Panda’s software tools were employed for the bibliometric analysis. The research explored the yearly trajectory of recent publications, dominant countries, collaborative institutions, sources of primary studies that have employed deep learning for hate speech detection, and the intellectual and social structures of the research constituents. It has been observed that the literature on hate speech detection is rapidly growing, but research output and collaborations from the developing countries of the world are still limited. The findings of this study provide insights into the intellectual structure and advancements in deep learning applications for hate speech detection while identifying research gaps for future work.

Indonesian Hate Speech Detection Using IndoBERTweet and BiLSTM on Twitter

Hate speech is an act of speech to spread hate to other people. In this digital era where everyone connects with social media, hate speech is growing rapidly and uncontrollably. Many people do not realize they are giving hate speech when critics something on social media due to a lack of awareness of the difference between hate speech and free speech. The results make victims feel alienated from society, and the people who spread it would often face the law. Detection in the sentences to identify whether it contains hate speech is essential to counter people's ignorance. For detecting such sentences, a machine learning algorithm is widely used to help identify each sentence. In this paper, we used a subset from machine learning named deep learning with the latest IndoBERT model named IndoBERTweet and combined it with RNN layer named BiLSTM. The appearance of IndoBERTweet opened more chances to further improve text classification performance with the addition of BiLSTM layer. The model first made a token representative from the sentence, then calculated it to analyze and made the classification based on the calculation. For this model to be effective, we trained our model with the labeled public dataset retrieved from Twitter. These datasets are classified into hate speech and non-hate speech, and these labels are applied to the models. We evaluated our model and achieved an accuracy of 93.7%, an improvement for classifying hate speech sentences from previous research.

Application of LSTM and GloVe Word Embedding for Hate Speech Detection in Indonesian Twitter Data

Application of LSTM and GloVe Word Embedding for Hate Speech Detection in Indonesian Twitter Data

Detection of Indonesian Hate Speech in the Comments Column of Indone-sian Artists' Instagram Using the RoBERTa Method

This study detects hate speech comments from Instagram post comments where the method used is RoBERTa. Roberta's model was chosen based on the consideration that this model has a high level of accuracy in classifying text in English compared to other models, and possibly has good potential in detecting Indonesian as used in this research. There are two test scenarios namely full-preprocessing and non full-preprocessing where the experimental results show that non full-preprocessing has an average value of accuracy higher than full-preprocessing, and the average value of non full-preprocessing accuracy is 85.09%. Full-preprocessing includes several preprocessing stages, namely cleansing, case folding, normalization, tokenization, and stemming. While non full-preprocessing includes all processes in preprocessing except the stemming process. This shows that RoBERTa predicts comments well when not using full-preprocessing.

Artificial intelligence applied in the detection of hate speech in the social political context in early 2023 in Peru

The rapid spread of hate speech through social networks to a large number of people has generated destabilization in a social and economic context at the beginning of the year 2023. The purpose of the research is to detect hate speech through Artificial Intelligence with Machine Learning techniques in a social political context. In addition, the research is of an applied type in which an exploratory search was carried out and the population is comprised of 1,970 tweets in the central and southern regions of Peru. The sampling is probabilistic, and 1,970 tweets from the regions involved were reviewed and manually labeled. Likewise, the methodology applied was SEMMA, as it allows to ensure that the models are accurate and fit the data of this study. A comparison of the respective models was performed to achieve an optimal result, of which the main indicator was the average accuracy in which the Multinomial Naive Bayes model was the best model with a score of 73.87%. Finally, it is concluded that this study allows accelerating the process of detecting verbal hate speech in social networks for the analysis of public opinion and the detection of suspected hate inciters.

EnsMulHateCyb: Multilingual hate speech and cyberbully detection in online social media

Nowadays, users across the globe interact with one another for information exchange, communication, and association on various online social media. However, some individuals exploit these venues for malicious practices like hate speech and cyberbully. In this paper, we present an improved multilingual hate speech and cyberbully detection model using bagging-stacking based hybrid ensemble deep learning techniques. The proposed model utilizes Bi-directional Long Short-Term Memory (BiLSTM), Bi-directional Gated Recurrent Unit (Bi-GRU), Convolutional Neural Network (CNN), and Long Short-Term Memory (LSTM) techniques to enhance the overall performance. We first preprocess the multilingual data streams followed by adoption of Global vectors for word Representation (GloVe) embeddings to convert words to a vector representation in parallel enabling the data streams for binary classification task. In order to construct an architecture for the detection of hate speech and cyberbully, we introduce a heterogeneous fusion of multiple effective models in a unique approach such that CNN-LSTM utilizes a stacking approach with stochastic gradient descent to achieve optimal weights, whereas all the base learners used bagging ensemble approach with cross-validation to reach optimal weights. The final output layer of the proposed ensemble deep learning architecture is achieved using a super learner approach on base learners. To show the efficacy of the proposed model, we conduct the simulation on a total of nine real-world social media datasets in different languages and compared the results with other contemporary hate speech and cyberbully detection methods. The collected findings show that the proposed model outperforms other models on considered datasets and shows an improvement of at least 4.44% in F1 scores.

Multimodal hate speech detection via multi-scale visual kernels and knowledge distillation architecture

People increasingly use social media platforms to express themselves by posting visuals and texts. As a result, hate content is on the rise, necessitating practical visual caption analysis. Thus, the relationship between image and caption modalities is crucial in visual caption analysis. Contrarily, most methods combine features from the image and caption modalities using deep learning architectures with millions of parameters already trained without integrating a specialized attention module, resulting in less desirable outcomes. This paper suggests a novel multi-modal architecture for identifying hateful memetic information in response to the above observation. The proposed architecture contains a novel "multi-scale kernel attentive visual" (MSKAV) module that uses an efficient multi-branch structure to extract discriminative visual features. Additionally, MSKAV utilizes an adaptive receptive field using multi-scale kernels. MSKAV also incorporates a multi-directional visual attention module to highlight spatial regions of importance. The proposed model also contains a novel "knowledge distillation-based attentional caption" (KDAC) module. It uses a transformer-based self-attentive block to extract discriminative features from meme captions. Thorough experimentation on multi-modal hate speech benchmarks MultiOff, Hateful Memes, and MMHS150K datasets achieved accuracy scores of 0.6250, 0.8750, and 0.8078, respectively. It also reaches impressive AUC scores of 0.6557, 0.8363, and 0.7665 on the three datasets, respectively, beating SOTA multi-modal hate speech identification models.

Enhancing hate speech detection with user characteristics

Enhancing hate speech detection with user characteristics

Improving hate speech detection using Cross-Lingual Learning

The growth of social media worldwide has brought social benefits and challenges. One problem we highlight is the proliferation of hate speech on social media. We propose a novel method for detecting hate speech in texts using Cross-Lingual Learning. Our approach uses transfer learning from Pre-Trained Language Models (PTLM) with large corpora available to solve problems in languages with fewer resources for the specific task. The proposed methodology comprises four stages: corpora acquisition, the PTLM definition, training strategies, and evaluation. We carried out experiments using Pre-Trained Language Models in English, Italian, and Portuguese (BERT and XLM-R) to verify which best suited the proposed method. We used corpora in English (WH) and Italian (Evalita 2018) as the source language and the OffComBr-2 corpus in Portuguese (the target language). The results of the experiments showed that the proposed methodology is promising: for the OffComBr-2 corpus, the best state-of-the-art result was obtained (F1-measure = 92%).

What do Twitter comments tell about news article bias? Assessing the impact of news article bias on its perception on Twitter

News stories circulating online, especially on social media platforms, are nowadays a primary source of information. Given the nature of social media, news no longer are just news, but they are embedded in the conversations of users interacting with them. This is particularly relevant for inaccurate information or even outright misinformation because user interaction has a crucial impact on whether information is uncritically disseminated or not. Biased coverage has been shown to affect personal decision-making. Still, it remains an open question whether users are aware of the biased reporting they encounter and how they react to it. The latter is particularly relevant given that user reactions help contextualize reporting for other users and can thus help mitigate but may also exacerbate the impact of biased media coverage.This paper approaches the question from a measurement point of view, examining whether reactions to news articles on Twitter can serve as bias indicators, i.e., whether how users comment on a given article relates to its actual level of bias. We first give an overview of research on media bias before discussing key concepts related to how individuals engage with online content, focusing on the sentiment (or valance) of comments and on outright hate speech. We then present the first dataset connecting reliable human-made media bias classifications of news articles with the reactions these articles received on Twitter. We call our dataset BAT - Bias And Twitter. BAT covers 2,800 (bias-rated) news articles from 255 English-speaking news outlets. Additionally, BAT includes 175,807 comments and retweets referring to the articles.Based on BAT, we conduct a multi-feature analysis to identify comment characteristics and analyze whether Twitter reactions correlate with an article’s bias. First, we fine-tune and apply two XLNet-based classifiers for hate speech detection and sentiment analysis. Second, we relate the results of the classifiers to the article bias annotations within a multi-level regression. The results show that Twitter reactions to an article indicate its bias, and vice-versa. With a regression coefficient of 0.703 (p<0.01), we specifically present evidence that Twitter reactions to biased articles are significantly more hateful. Our analysis shows that the news outlet’s individual stance reinforces the hate-bias relationship. In future work, we will extend the dataset and analysis, including additional concepts related to media bias.

A comparative study of the characteristics of hate speech propagators and their behaviours over Twitter social media platform

The internet and social media have facilitated diverse communication genres, enabling widespread and rapid opinions-sharing. However, hate speech imposes a contemporary challenge on individuals and communities, given the user anonymity, freedom, and inadequate regulation. Therefore, it is imperative to identify the perpetrators responsible for spreading hate content and examine their behaviour to prevent and mitigate the negative impact. This study aimed to compare the characteristics of hate speech propagators and their behaviour with non-hate users on Twitter for the first time in Sri Lanka. The intrinsic and extrinsic profile features were extensively analyzed, employing Sinhala and English text analysis techniques. A corpus of 102882 posts from 530 hate and non-hate Twitter user profiles was selected for the study. This study investigates the unique characteristics of hate speech propagators and non-hate users by examining their profile self-presentation, conducting social network analysis, and analyzing sentiment and emotion through linguistic analysis. Hate users often refrained from expression, with infrequent account verification and geotagging. They tend to have a higher follower and following counts and more favourites, group memberships, and statuses than non-hate users. However, general Twitter user engagement with hate users was significantly low, with fewer likes, retweets, and replies. The limited involvement of normal users with hate content indicates that audiences can be effectively utilized to combat hate speech. The sentiment analysis between languages showed polarisation of negative tweets towards Sinhala, with the synergistic effect of English language users using positive sentiment to spread hate content. The novel findings shed light on the characteristics of hate users, facilitating their early detection and moderation of hate speech and aiding in developing algorithms to rank and categorize hate users using artificial intelligence. Moreover, it can be used for policy reforms, awareness programmes, and building social cohesion while combating hate speech.

A Supervised Classification Approach for Detecting Hate Speech in English Tweets

As social concerns about threats of hatred and harassment have grown on the internet, there has been a lot of attention paid to detecting hate speech. This research looks at how well SGD classifiers with hyper-parameter tuning perform at detecting hate speech in tweets. It describes the categorization of English tweets with stochastic gradient descent (SGD) classifiers. The categorization of text documents depends on their content, which is divided into groups based on predefined categories. The Term-Frequency (TF) and Inverse-Document Frequency (IDF) parameters are implemented in the proposed system. A Stochastic Gradient Descent method (SGD) is used to generate classifiers that learn independent features, and performance is assessed using Accuracy and F1-score.

Hate Speech Detection in Indonesia Twitter Comments Using Convolutional Neural Network (CNN) and FastText Word Embedding

Hate Speech Detection in Indonesia Twitter Comments Using Convolutional Neural Network (CNN) and FastText Word Embedding

Data Augmentation for Improving Explainability of Hate Speech Detection

Data Augmentation for Improving Explainability of Hate Speech Detection

Hate Speech Detection for Banjarese Languages on Instagram Using Machine Learning Methods

Hate speech refers to verbal expression or communication that aims to provoke or discriminate against individuals. The Ministry of Communication and Information of Indonesia has encountered and dealt with 3,640 cases of hate speech transmitted through digital channels between 2018 and 2021. Particularly in South Kalimantan, hate speech in the local language, Banjarese has become increasingly prevalent in recent years. Surprisingly, there is a lack of research on using machine learning to detect hate speech in the Banjarese language, specifically on Instagram. Therefore, this study aimed to address this gap by constructing a dataset of Banjarese language hate speech and comparing various feature extraction and machine learning models to detect Banjarese language hate speech effectively. Thisresearch used several feature extraction techniques and machine learning methods to detect Banjareselanguage hate speech. The feature extraction methods used were Word N-Gram, Term Frequency- Inverse Document Frequency (TF-IDF), a combination of Word N-Gram and TF-IDF, Word2Vec, and Glove, while the machine learning methods used were Support Vector Machine (SVM), Na¨ıve Bayes, and Decision Tree. The results of this study revealed that the combination of TF-IDF for feature extraction and SVM as the model achieves exceptional performance. The average Recall, Precision, Accuracy, and F1-Score score exceeded 90%, demonstrating the model’s ability to identify Banjarese hate speech accurately.