Code-mixed Text Research Articles

A survey on NLP tasks, resources and techniques for low-resource Telugu-English code-mixed text

With the proliferation of informal content on various social media platforms in the form of posts, comments, and feedback, the importance of analyzing text in code-mixed form is gaining importance. Telugu, a low-resource Indian language, has a lot of online content being generated in code-mixed form. However, the lack of large corpora, annotated data and Natural Language Processing (NLP) resources are impeding research on Telugu-English code-mixed data. This paper provides a survey of existing literature on Telugu-English code-mixed text in the areas of resources, POS tagging, Named Entity Recognition, language identification, sentiment analysis, application tasks, dialog systems, and Question-Answering. Various datasets being used by the researchers in the field, along with methods applied to them are detailed. Research gaps are identified to provide future direction for researchers working in this field.

Use of prompt-based learning for code-mixed and code-switched text classification

Code-mixing and code-switching (CMCS) are prevalent phenomena observed in social media conversations and various other modes of communication. When developing applications such as sentiment analysers and hate-speech detectors that operate on this social media data, CMCS text poses challenges. Recent studies have demonstrated that prompt-based learning of pre-trained language models outperforms full fine-tuning across various tasks. Despite the growing interest in classifying CMCS text, the effectiveness of prompt-based learning for the task remains unexplored. This paper presents an extensive exploration of prompt-based learning for CMCS text classification and the first comprehensive analysis of the impact of the script on classifying CMCS text. Our study reveals that the performance in classifying CMCS text is significantly influenced by the inclusion of multiple scripts and the intensity of code-mixing. In response, we introduce a novel method, Dynamic+AdapterPrompt, which employs distinct models for each script, integrated with adapters. While DynamicPrompt captures the script-specific representation of the text, AdapterPrompt emphasizes capturing the task-oriented functionality. Our experiments on Sinhala-English, Kannada-English, and Hindi-English datasets for sentiment classification, hate-speech detection, and humour detection tasks show that our method outperforms strong fine-tuning baselines and basic prompting strategies.

Predicting multi-label emojis, emotions, and sentiments in code-mixed texts using an emojifying sentiments framework

In the era of social media, the use of emojis and code-mixed language has become essential in online communication. However, selecting the appropriate emoji that matches a particular sentiment or emotion in the code-mixed text can be difficult. This paper presents a novel task of predicting multiple emojis in English-Hindi code-mixed sentences and proposes a new dataset called SENTIMOJI, which extends the SemEval 2020 Task 9 SentiMix dataset. Our approach is based on exploiting the relationship between emotion, sentiment, and emojis to build an end-to-end framework. We replace the self-attention sublayers in the transformer encoder with simple linear transformations and use the RMS-layer norm instead of the normal layer norm. Moreover, we employ Gated Linear Unit and Fully Connected layers to predict emojis and identify the emotion and sentiment of a tweet. Our experimental results on the SENTIMOJI dataset demonstrate that the proposed multi-task framework outperforms the single-task framework. We also show that emojis are strongly linked to sentiment and emotion and that identifying sentiment and emotion can aid in accurately predicting the most suitable emoji. Our work contributes to the field of natural language processing and can help in the development of more effective tools for sentiment analysis and emotion recognition in code-mixed languages. The codes and data will be available at https://www.iitp.ac.in/~ai-nlp-ml/resources.html#SENTIMOJI to facilitate research.

Abusive Comment Detection in Tamil Code-Mixed Data by Adjusting Class Weights and Refining Features

In recent years, a significant portion of the content on various platforms on the internet has been found to be offensive or abusive. Abusive comment detection can go a long way in preventing internet users from facing the adverse effects of coming in contact with abusive language. This problem is particularly challenging when the comments are found in low-resource languages like Tamil or Tamil-English code-mixed text. So far, there has not been any substantial work on abusive comment detection using imbalanced datasets. Furthermore, significant work has not been performed, especially for Tamil code-mixed data, that involves analysing the dataset for classification and accordingly creating a custom vocabulary for preprocessing. This paper proposes a novel approach to classify abusive comments from an imbalanced dataset using a customised training vocabulary and a combination of statistical feature selection with language-agnostic feature selection while making use of explainable AI for feature refinement. Our model achieved an accuracy of 74% and a macro F1-score of 0.46.

Share What You Already Know: Cross-Language-Script Transfer and Alignment for Sentiment Detection in Code-Mixed Data

Code-switching entails mixing multiple languages. It is an increasingly occurring phenomenon in social media texts. Usually, code-mixed texts are written in a single script, even though the languages involved have different scripts. Pre-trained multilingual models primarily utilize the data in the native script of the language. In existing studies, the code-switched texts are utilized as they are. However, using the native script for each language can generate better representations of the text owing to the pre-trained knowledge. Therefore, a cross-language-script knowledge sharing architecture utilizing the cross attention and alignment of the representations of text in individual language scripts was proposed in this study. Experimental results on two different datasets containing Nepali-English and Hindi-English code-switched texts, demonstrate the effectiveness of the proposed method. The interpretation of the model using model explainability technique illustrates the sharing of language-specific knowledge between language-specific representations.

Faux Hate: unravelling the web of fake narratives in spreading hateful stories: a multi-label and multi-class dataset in cross-lingual Hindi-English code-mixed text

Faux Hate: unravelling the web of fake narratives in spreading hateful stories: a multi-label and multi-class dataset in cross-lingual Hindi-English code-mixed text

Augmenting sentiment prediction capabilities for code-mixed tweets with multilingual transformers

People in the modern digital era are increasingly embracing social media platforms to express their concerns and emotions in the form of reviews or comments. While positive interactions within diverse communities can considerably enhance confidence, it is critical to recognize that negative comments can hurt people’s reputations and well-being. Currently, individuals tend to express their thoughts in their native languages on these platforms, which is quite challenging due to potential syntactic ambiguity in these languages. Most of the research has been conducted for resource-aware languages like English. However, low-resource languages such as Urdu, Arabic, and Hindi present challenges due to limited linguistic resources, making information extraction labor-intensive. This study concentrates on code-mixed languages, including three types of text: English, Roman Urdu, and their combination. This study introduces robust transformer-based algorithms to enhance sentiment prediction in code-mixed text, which is a combination of Roman Urdu and English in the same context. Unlike conventional deep learning-based models, transformers are adept at handling syntactic ambiguity, facilitating the interpretation of semantics across various languages. We used state-of-the-art transformer-based models like Electra, code-mixed BERT (cm-BERT), and Multilingual Bidirectional and Auto-Regressive Transformers (mBART) to address sentiment prediction challenges in code-mixed tweets. Furthermore, results reveal that mBART outperformed the Electra and cm-BERT models for sentiment prediction in code-mixed text with an overall F1-score of 0.73. In addition to this, we also perform topic modeling to uncover shared characteristics within the corpus and reveal patterns and commonalities across different classes.

Consensus-Based Machine Translation for Code-Mixed Texts

Multilingualism in India is widespread due to its long history of foreign acquaintances. This leads to the presence of an audience familiar with conversing using more than one language. Additionally, due to the social media boom, the usage of multiple languages to communicate has become extensive. Hence, the need for a translation system that can serve the novice and monolingual user is the need of the hour. Such translation systems can be developed by methods such as statistical machine translation and neural machine translation, where each approach has its advantages as well as disadvantages. In addition, the parallel corpus needed to build a translation system, with code-mixed data, is not readily available. In the present work, we present two translation frameworks that can leverage the individual advantages of these pre-existing approaches by building an ensemble model that takes a consensus of the final outputs of the preceding approaches and generates the target output. The developed models were used for translating English-Bengali code-mixed data (written in Roman script) into their equivalent monolingual Bengali instances. A code-mixed to monolingual parallel corpus was also developed to train the preceding systems. Empirical results show improved BLEU and TER scores of 17.23 and 53.18 and 19.12 and 51.29, respectively, for the developed frameworks.

Sentiment Analysis of Code-Mixed Text: A Comprehensive Review

Sentiment Analysis is the task of identifying and extracting the opinion expressed in a text to determine the writer's perception of an entity. Due to globalization, people often mix two or more languages and use phonetic typing and lexical borrowing in web communication. This concept is known as code-mixing. Although extracting the opinion of text written in monolingual languages is simple and straightforward, Sentiment Analysis of code-mixed text is challenging. Classifiers fail within the context of the code-mixed text as text may consist of creative writing, spelling variations, grammatical errors, and different word orders. Hence, SA of code-mixed text is an interesting, challenging, and popular research area. This paper presents the state-of-the-art in Sentiment Analysis of code-mixed text by discussing each concept in detail. The paper also discusses the focused areas, techniques used, limitations, and performances of the studies related to code-mixing.

Language Identification and Transliteration approaches for Code-Mixed Text

Language Identification and Transliteration approaches for Code-Mixed Text

Toxic comment classification and rationale extraction in code-mixed text leveraging co-attentive multi-task learning

Toxic comment classification and rationale extraction in code-mixed text leveraging co-attentive multi-task learning

Word Level Language Identification in Indonesian-Javanese-English Code-Mixed Text

Code-mixing is a prevalent phenomenon in contemporary communication, particularly in social media interactions. In Indonesia, code-mixing is common, with languages such as Indonesian, Javanese, and English frequently mixed in everyday conversation. Accurate language identification at the word level is important for various downstream natural language processing tasks, such as sentiment analysis and translation. This study investigates machine learning techniques for word-level language identification in Indonesian-Javanese-English code-mixed texts. We compared three categories of machine learning models: traditional machine learning, BLSTM-based architectures, and transformer-based pre-trained models. Our experiments demonstrate that fine-tuning pre-trained models achieves strong performance, with XLM-RoBERTa and IndoBERTweet achieving the best F1 scores of 93.69% and 93.63%, respectively. Our results highlight the effectiveness of fine-tuning pre-trained models for language identification tasks, demonstrating their ability to comprehend sentence contexts and produce accurate label predictions.

English to Arabic Braille Neural Machine Translation Through Corpus Augmentation

In this paper we have shown the development of English to Arabic Braille Neural Machine Translation (NMT) System. For our experiments we have developed two NMT systems. The first was the baseline NMT system which was trained only on the English-Arabic parallel corpus whereas the second NMT system has some sub modules for handling syntax transfer of English to Arabic and translation/transliteration of English Named Entities into Arabic equivalents. Then the system used this code-mixed text to train the NMT model. The English-Arabic parallel corpus was taken from Opus parallel corpus repository. For preprocessing of the English text Stanford's Stanza NLP Library was used. The trained NMT models were evaluated using standard automatic MT evaluation metrics and their results were also corelated with human judgements. It was found that the corpus augmented NMT system performed better than the baseline NMT system.

Sarcasm Detection in Indonesian-English Code-Mixed Text Using Multihead Attention-Based Convolutional and Bi-Directional GRU

Sarcasm Detection in Indonesian-English Code-Mixed Text Using Multihead Attention-Based Convolutional and Bi-Directional GRU

Analysing Code-Mixed Text in Programming Instruction Through Machine Learning for Feature Extraction

Analysing Code-Mixed Text in Programming Instruction Through Machine Learning for Feature Extraction

A feature fusion and detection approach using deep learning for sentimental analysis and offensive text detection from code-mix Malayalam language

Social media has efficiently turned out to be a major source of digital marketing and communication. As social media platforms set up free manifestation of facts and thoughts in images, video, and text, there is a widespread requirement to screen them to secure groups and individuals from being exposed to offensive content that specifically targets them. In social media communications, limitless practice of vocabulary complicates the code-mixed text processing. The most important aspiration of this research is to analyze the sentiments and classify the code-mixed social media posts/comments in the Dravidian language: Malayalam using deep learning. In this study, initial phase is the ALBERT tokenization that splits the words in the comments. Subsequently, to extract significant features from the dataset, feature extraction is carried out. Following feature extraction, feature fusion takes place using the Shannon entropy and Recurrent Neural Network (RNN) model. Finally, sentimental analysis and offensive text identification are conducted using the Hierarchical Attention Network (HAN). The experimentation of the proposed Feature fusion + HAN technique is performed by the metrics, namely accuracy, sensitivity, and specificity, where the accuracy attained by the proposed model is 0.956, the sensitivity is 0.939, and the specificity is 0.978.

Language augmentation approach for code-mixed text classification

The usage of more than one language in the same text is referred to as Code Mixed. It is evident that there is a growing degree of adaption of the use of code-mixed data, especially English with a regional language, on social media platforms. Existing deep-learning models do not take advantage of the implicit language information in the code-mixed text. Our study aims to improve BERT-based models’ performance on low-resource Code-Mixed Hindi–English Datasets by experimenting with language augmentation approaches. For language augmentation in BERT models, we explore word-level interleaving and post-sentence placement of language information. We have examined the performance of vanilla BERT-based models and their code-mixed HingBERT counterparts on respective benchmark datasets, comparing their results with and without using word-level language information. The models were evaluated using metrics such as accuracy, precision, recall, and F1 score. Our findings show that the proposed language augmentation approaches work well across different BERT models. We demonstrate the importance of augmenting code-mixed text with language information on five different code-mixed Hindi–English classification datasets based on sentiment analysis, hate speech detection, and emotion detection.

AdapterFusion-based multi-task learning for code-mixed and code-switched text classification

Social media text can be classified in different ways, viz sentiment analysis, humour detection, hate speech detection and hope speech detection. Multitask learning (MTL) models built on Large Language Models (LLMs) eliminate the need to build separate models for each of these tasks. However, building MTL models by fully fine-tuning the LLM has limitations such as catastrophic forgetting and requiring complete retraining to add a new task. AdapterFusion was introduced to address these limitations. However, existing AdapterFusion techniques have not been experimented with code-mixed or code-switched text. Moreover, they only considered task-based AdapterFusion on monolingual LLMs. However, using monolingual LLMs is sub-optimal in classifying code-mixed or code-switched text. A better alternative is multilingual LLMs. In this paper, we present an MTL model that combines task AdapterFusion with language adapters on top of a multilingual LLM. We combine language adapters sequentially, in parallel, and as a fusion with task adapters to capture cross-lingual knowledge in code-mixed and code-switched text. We believe that this is the first research to introduce language-based AdapterFusion.

The Impact of Data Pre-Processing on Hate Speech Detection in a Mix of English and Hindi–English (Code-Mixed) Tweets

Due to the increasing reliance on social network platforms in recent years, hate speech has risen significantly among online users. Government and social media platforms face the challenging responsibility of controlling, detecting, and removing massively growing hateful content as early as possible to prevent future criminal acts, such as cyberviolence and real-life hate crimes. Twitter is used globally by people from various backgrounds and nationalities; it contains tweets posted in different languages, including code-mixed language, such as Hindi–English. Due to the informal format of tweets with variations in spelling and grammar, hate speech detection is especially challenging in code-mixed text. In this paper, we tackle the critical issue of hate speech detection on social media, with a focus on a mix of English and Hindi–English (code-mixed) text messages on Twitter. More specifically, we aim to evaluate the impact of data pre-processing on hate speech detection. Our method first performs 10-step data cleansing; then, it builds a detection method based on two architectures, namely a convolutional neural network (CNN) and a combination of CNN and long short-term Memory (LSTM) algorithms. We tune the hyperparameters of the proposed model architectures and conduct extensive experimental analysis on real-life tweets to evaluate the performance of the models in terms of accuracy, efficiency, and scalability. Moreover, we compare our method with a closely related hate speech detection method from the literature. The experimental results suggest that our method results in an improved accuracy and a significantly improved runtime. Among our best-performing models, CNN-LSTM improved accuracy by nearly 2% and decreased the runtime by almost half.

Sentimental analysis & Hate speech detection on English and German text collected from social media platforms using optimal feature extraction and hybrid diagonal gated recurrent neural network

Social networking platforms allow users to ask questions, exchange information, opinions, ideas, and promote companies. Social media's massive user-generated text is useful for study, translation, and analysis. Due of their quick spread and detrimental impact, offensive remarks, hate speech, and harassment must be discovered and deleted immediately. In non-English main language environments, code-mixed text makes hate speech detection difficult. Hate speech's thematic emphasis and target-oriented orientation are typically ignored in binary categorization methods. These methods also fail in code-mixing multilingual contexts. In this study, we propose an optimal feature extraction and hybrid diagonal gated recurrent neural network (FE-DGRNN) for hate speech detection and sentiment analysis on code-mixed texts in multiple languages. Our FE-DGRNN technique consists of three processes: preprocessing, improved seagull optimization (ISO) for feature extraction, and hybrid diagonal gated recurrent neural network (Hyb-DGRNN) for hate speech detection and sentiment analysis. We evaluate the performance of our proposed technique using the HASOC 2019 dataset, focusing on English and German. The results demonstrate high accuracy of 95%, 96%, and 92% for English tasks, with Precision and F-measure of 94%, 96%, and 91%. For German tasks, the accuracy is 92% and 93%, with Precision and F-measure of 90% and 91%.