Low-resource Languages Research Articles

Efficient Data Augmentation via lexical matching for boosting performance on Statistical Machine Translation for Indic and a Low-resource language

Efficient Data Augmentation via lexical matching for boosting performance on Statistical Machine Translation for Indic and a Low-resource language

Deep Ensemble Network for Sentiment Analysis in Bi-lingual Low-resource Languages

Sentiment analysis (SA) is the systematic identification, extraction, quantification, and study of affective states and subjective information using natural language processing. It is widely used for analyzing users’ feedback, such as reviews or social posts. Recently, SA has been one of the favorite research domains in NLP due to their wide range of applications, including E-commerce, healthcare, hotel business, and others. Many machine learning and deep learning-based models exist to predict the sentiment of the user’s post. However, the sentiment analysis in low-resource languages such as Kannada, Malayalam, Telugu, and Tamil received less attention due to language complexity and the low availability of required resources. This research fills the gap by proposing an ensemble model for predicting the sentiment of code-mixed Kannada and Malayalam languages. The ensemble of transformer-based models achieved a promising weighted F 1 -score of 0.66 for Kannada code-mixed language. In contrast, the ensemble model of the deep learning framework performed best by achieving a weighted F 1 -score of 0.72 for the Malayalam dataset, outperforming existing research.

CodeFed: Federated Speech Recognition for Low-Resource Code-Switching Detection

One common constraint in the practical application of speech recognition is Code Switching. The issue of code-switched languages is especially aggravated in the context of Indian languages – since most massively multilingual models are trained on corpora that are not representative of the diverse set of Indian languages. An associated constraint with such systems is the privacy-intrusive nature of the applications that aim to collate such representative data. To collectively mitigate both problems, this work presents CodeFed: A federated learning-based code-switching detection model that can be deployed to collaboratively be trained by leveraging private data from multiple users, without compromising their privacy. Using a representative low-resource Indic dataset, we demonstrate the superior performance of a collaboratively trained global model that is trained using federated learning on three low-resource Indic languages – Gujarati, Tamil and Telugu and draw a comparison of the model with respect to the most current work in the field. Finally, to evaluate the practical realizability of the proposed system, CodeFed also discusses the system overview of the label generation architecture which may accompany CodeFed’s possible real-time deployment.

Switch-Transformer Sentiment Analysis Model for Arabic Dialects That Utilizes a Mixture of Experts Mechanism

In recent years, models such as the transformer have demonstrated impressive capabilities in the realm of natural language processing. However, these models are known for their complexity and the substantial training they require. Furthermore, the self-attention mechanism within the transformer, designed to capture semantic relationships among words in sequences, faces challenges when dealing with short sequences. This limitation hinders its effectiveness in five-polarity Arabic sentiment analysis (SA) tasks. The switch-transformer model has surfaced as a potential substitute. Nevertheless, when employing one-task learning for their training, these models frequently face challenges in presenting exceptional performances and encounter issues when producing resilient latent feature representations, particularly in the context of small-size datasets. This challenge is particularly prominent in the case of the Arabic dialect, which is recognized as a low-resource language. In response to these constraints, this research introduces a novel method for the sentiment analysis of Arabic text. This approach leverages multi-task learning (MTL) in combination with the switch-transformer shared encoder to enhance model adaptability and refine sentence representations. By integrating a mixture of experts (MoE) technique that breaks down the problem into smaller, more manageable sub-problems, the model becomes skilled in managing extended sequences and intricate input–output relationships, thereby benefiting both five-point and three-polarity Arabic sentiment analysis tasks. The proposed model effectively identifies sentiment in Arabic dialect sentences. The empirical results underscore its exceptional performance, with accuracy rates reaching 84.02% for the HARD dataset, 67.89% for the BRAD dataset, and 83.91% for the LABR dataset, as demonstrated by the evaluations conducted on these datasets.

The Task of Post-Editing Machine Translation for the Low-Resource Language

In recent years, machine translation has made significant advancements; however, its effectiveness can vary widely depending on the language pair. Languages with limited resources, such as Kazakh, Uzbek, Kalmyk, Tatar, and others, often encounter challenges in achieving high-quality machine translations. Kazakh is an agglutinative language with complex morphology, making it a low-resource language. This article addresses the task of post-editing machine translation for the Kazakh language. The research begins by discussing the history and evolution of machine translation and how it has developed to meet the unique needs of languages with limited resources. The research resulted in the development of a machine translation post-editing system. The system utilizes modern machine learning methods, starting with neural machine translation using the BRNN model in the initial post-editing stage. Subsequently, the transformer model is applied to further edit the text. Complex structural and grammatical forms are processed, and abbreviations are replaced. Practical experiments were conducted on various texts: news publications, legislative documents, IT sphere, etc. This article serves as a valuable resource for researchers and practitioners in the field of machine translation, shedding light on effective post-editing strategies to enhance translation quality, particularly in scenarios involving languages with limited resources such as Kazakh and Uzbek. The obtained results were tested and evaluated using specialized metrics—BLEU, TER, and WER.

Linguistic disparities in cross-language automatic speech recognition transfer from Arabic to Tashlhiyt

Tashlhiyt is a low-resource language with respect to acoustic databases, language corpora, and speech technology tools, such as Automatic Speech Recognition (ASR) systems. This study investigates whether a method of cross-language re-use of ASR is viable for Tashlhiyt from an existing commercially-available system built for Arabic. The source and target language in this case have similar phonological inventories, but Tashlhiyt permits typologically rare phonological patterns, including vowelless words, while Arabic does not. We find systematic disparities in ASR transfer performance (measured as word error rate (WER) and Levenshtein distance) for Tashlhiyt across word forms and speaking style variation. Overall, performance was worse for casual speaking modes across the board. In clear speech, performance was lower for vowelless than for voweled words. These results highlight systematic speaking mode- and phonotactic-disparities in cross-language ASR transfer. They also indicate that linguistically-informed approaches to ASR re-use can provide more effective ways to adapt existing speech technology tools for low resource languages, especially when they contain typologically rare structures. The study also speaks to issues of linguistic disparities in ASR and speech technology more broadly. It can also contribute to understanding the extent to which machines are similar to, or different from, humans in mapping the acoustic signal to discrete linguistic representations.

Offensive language detection in low resource languages: A use case of Persian language.

THIS ARTICLE USES WORDS OR LANGUAGE THAT IS CONSIDERED PROFANE, VULGAR, OR OFFENSIVE BY SOME READERS. Different types of abusive content such as offensive language, hate speech, aggression, etc. have become prevalent in social media and many efforts have been dedicated to automatically detect this phenomenon in different resource-rich languages such as English. This is mainly due to the comparative lack of annotated data related to offensive language in low-resource languages, especially the ones spoken in Asian countries. To reduce the vulnerability among social media users from these regions, it is crucial to address the problem of offensive language in such low-resource languages. Hence, we present a new corpus of Persian offensive language consisting of 6,000 out of 520,000 randomly sampled micro-blog posts from X (Twitter) to deal with offensive language detection in Persian as a low-resource language in this area. We introduce a method for creating the corpus and annotating it according to the annotation practices of recent efforts for some benchmark datasets in other languages which results in categorizing offensive language and the target of offense as well. We perform extensive experiments with three classifiers in different levels of annotation with a number of classical Machine Learning (ML), Deep learning (DL), and transformer-based neural networks including monolingual and multilingual pre-trained language models. Furthermore, we propose an ensemble model integrating the aforementioned models to boost the performance of our offensive language detection task. Initial results on single models indicate that SVM trained on character or word n-grams are the best performing models accompanying monolingual transformer-based pre-trained language model ParsBERT in identifying offensive vs non-offensive content, targeted vs untargeted offense, and offensive towards individual or group. In addition, the stacking ensemble model outperforms the single models by a substantial margin, obtaining 5% respective macro F1-score improvement for three levels of annotation.

Mathematical Problem Solving in Arabic: Assessing Large Language Models

This paper comprehensively evaluates the efficacy of different large language models (LLMs) in addressing mathematical challenges expressed in natural languages, mainly focusing on low-resource languages like Arabic. The main challenge of this problem is that despite the considerable size and impressive problem-solving capabilities of these models, they still require enhancements to achieve satisfactory performance for mathematical problem-solving. An LLM must be trained or fine-tuned extensively in understanding and solving mathematical problems articulated in natural language contexts. This is further complicated by the scarcity of high-quality, annotated datasets in low-resource languages. To overcome this problem, we introduce a method for generating synthetic data to address the challenge of data scarcity in Arabic. This approach includes using LLMs to create high-quality, synthetic mathematical problem datasets that can be used to augment the training process. We propose the Synthetic Data Augmentation (SDA) framework, an advanced variant of traditional data augmentation techniques explicitly tailored for mathematical problem-solving in low-resource languages. Experimental studies reveal that using synthetic data significantly improves the performance of LLMs in solving mathematical problems in Arabic. This study contributes to the field of natural language processing and provides insights into the potential of synthetic data generation to enhance LLM performance in low-resource languages.

An Intelligent Health Care System for Detecting Drug Abuse in Social Media Platforms Based on Low Resource Language

An Intelligent Health Care System for Detecting Drug Abuse in Social Media Platforms Based on Low Resource Language

Improving Amazigh POS tagging using machine learning

Tamazight, Berber, and Amazigh are the multiple names for the same language. It covers a great geographical area including the north of Africa, Sahara Sahel. It is spread principally in Morocco, Algeria, Tunisia, and Mali. In terms of natural language processing, it is considered a low-resource language. This paper presents multiple applications of different machine learning algorithms for part-of-speech tagging Amazigh for the first time. Those algorithms include trigrams 'n' tags (TnT), Brill tagging, hidden Markov model (HMM), Unigram, Bigram, Unigram + Bigram,and conditional random fields (CRF). Also, we present a part-of-speech tagger using CRF with our function of extracting features from the Amazigh language. The importance of finding a performant POS tagger for the Amazigh is to enrich its corpus, which is a main step for other NLP applications. In this research, we used 60000 tokens of annotated Amazigh corpus with 28 tags, and we realized the necessary processing step on it to be in an adequate form for feeding each model. A detailed comparison of the performance results is presented to establish the best one and the results show that our application of CRF model outperforms other techniques.

Can Pretrained English Language Models Benefit Non-English NLP Systems in Low-Resource Scenarios?

Pretrained language models have achieved great success in a wide range of natural language processing (NLP) problems, because they learn language representations from large-scale text corpora and can adapt to downstream tasks by finetuning them on annotated task data. However, such success relies on both large-scale text and annotated data, so the lack of training data is a major practical problem for many languages, especially low-resource languages. In this paper, we explore whether a pretrained English language model can benefit non-English NLP systems in low-resource scenarios, i.e., with limited text corpora or annotated data. To achieve this, we first propose cross-lingual knowledge transfer methods and then validate our methods in low-resource scenarios. Specifically, our cross-lingual knowledge transfer methods are applied in the training stages of language model pretraining or downstream finetuning. At the two stages, the methods are designed for the transfer of upstream general knowledge or downstream task-specific knowledge, respectively. In the experiments, we perform pretraining and finetuning with limited non-English data to simulate the low-resource scenarios. We evaluate our methods on ten downstream tasks over a wide range of languages, and present systematic comparisons among various knowledge transfer methods. Experimental results show that our methods successfully leverage a pretrained English language model to improve task performance in other languages. Besides, we demonstrate the multilinguality of the English language model in various application scenarios. Our findings imply the possibility to improve low-resource-language NLP systems with large-scale English language models.

Enhancing Model Performance through Translation-based Data Augmentation in the context of Fake News Detection

The rapid development of social media in recent years has encouraged the sharing of vast amounts of data and the propagation of fake news. This has pushed the scientific community to focus on this phenomenon, particularly those working on natural language processing, by developing detection tools to combat fake news. At the same time, most studies have focused on languages with a high resource content (corpus). This paper aims to shed light on low-resource languages, particularly the Algerian dialect, through an experimental study with two objectives. The first one is to verify if the automatic translation from Modern Standard Arabic (MSA) to the Algerian dialect can be considered an approach to increase the resources in the Algerian dialect, especially with the rise of large language models (LLMs). The second is to verify the impact of the translation-based data augmentation method on fake news detection by using transformer-based Arabic pre-trained models in different data augmentation configurations. We have discovered that LLMs can generate translations that closely resemble human translations. In this study, we demonstrate that data augmentation can result in saturation and a decline in model performance due to the introduction of noise and variations in writing styles.

Lexicon-based Language Relatedness Analysis

The field of computational linguistics has been impacting various issues in language disciplines. The enormous growth of machine learning algorithms and Natural Language Processing (NLP) empowers its advancement and brings huge benefits to societies. For instance, machine translation, text summarization, sentence auto-completion, and sentiment analysis are a few of its benefits. However, leveraging this opportunity for low-resourced languages is challenging due to the lack of available electronic datasets. This paper presents a lexicon-based language relatedness analysis on Ethiopian low-resourced languages. The languages Wolaita, Dawuro, Gamo, and Gofa belong to the Ethiopian Omotic language family and share rich linguistic cultures and similarities. However, the extent of their inter-relatedness remains unknown. To address this gap, we collected and prepared novel corpora from the Bible and academic texts. We employed the TF-IDF technique for feature extraction and used the cosine similarity method to measure the similarities among these languages. In addition to cosine similarity, we used Euclidean distance to measure the spatial distances between the languages. The experiment results showed that Wolaita and Gofa exhibited high relatedness (33.4%), while Dawuro and Gamo demonstrated low relatedness (12.1%).

Offensive Language Detection for Low Resource Language Using Deep Sequence Model

Social media platforms are heavily used by people to express their views in their native languages. Besides positive views, people often use abusive or offensive language to express their anger or frustration. Resource-rich languages have offensive language detection systems which automatically monitor and block offensive content, however, they are very rare for low-resourced languages. This is because of the nonavailability of datasets for local languages. This article proposes a model which automatically detects offensive language for a very low-resource language, i.e., Pashto. The Roman Pashto dataset is created by picking 60 thousand comments from different social media and labeling them manually. The proposed model is trained and tested using three different feature extraction approaches, i.e., bag-of-words (BoW), term frequency-inverse document frequency (TF-IDF), and sequence integer encoding. Four traditional classifiers and a deep sequence model are used to train on this task. Experimental result shows that the random forest classifier works best and give 94.07 % testing accuracy on a combination of unigrams, bigrams, and trigrams. The same classifier gives maximum accuracy of 93.90 % with TF-IDF. However, the overall highest testing accuracy of 97.21% is achieved by using bidirectional long short-term memory (BLSTM). The corpus created in this work is made available for the researcher working in this domain.

Comparative Analysis of Transformer Models for Sentiment Analysis in Low-Resource Languages

Comparative Analysis of Transformer Models for Sentiment Analysis in Low-Resource Languages

Towards Cross-Corpora Generalization for Low-Resource Spoken Language Identification

Towards Cross-Corpora Generalization for Low-Resource Spoken Language Identification

Application of Quantum Recurrent Neural Network in Low Resource Language Text Classification

Application of Quantum Recurrent Neural Network in Low Resource Language Text Classification

Boosting English-Amharic machine translation using corpus augmentation and Transformer

The Transformer-based neural machine translation (NMT) model has been very successful in recent years and has become a new mainstream method. However, using them in lowresourced languages requires large amounts of data and efficient model configuration (hyperparameter tuning) mechanisms. The scarcity of parallel texts is a bottleneck for high quality (N) MTs, especially for under resourced languages like Amharic. As a result, this paper presents an attempt to improve English-Amharic MT by introducing three different vanilla Transformer architectures, with different hyper-parameter values. To obtain additional training material, offline token level corpus augmentation was applied to the previously collected English-Amharic parallel corpus. Compared to previous work on Amharic MT, the best of the three Transformer models have achieved state-of-the-art BLEU scores. In fact, we were able to achieve this result by employing corpus augmentation techniques and hyper-parameter tuning.

Automatic Speech Recognition Variance: Consecutive Runs of Low-Resource Languages in Whisper

Automatic Speech Recognition Variance: Consecutive Runs of Low-Resource Languages in Whisper

Dynamic decoding and dual synthetic data for automatic correction of grammar in low-resource scenario.

Grammar error correction systems are pivotal in the field of natural language processing (NLP), with a primary focus on identifying and correcting the grammatical integrity of written text. This is crucial for both language learning and formal communication. Recently, neural machine translation (NMT) has emerged as a promising approach in high demand. However, this approach faces significant challenges, particularly the scarcity of training data and the complexity of grammar error correction (GEC), especially for low-resource languages such as Indonesian. To address these challenges, we propose InSpelPoS, a confusion method that combines two synthetic data generation methods: the Inverted Spellchecker and Patterns+POS. Furthermore, we introduce an adapted seq2seq framework equipped with a dynamic decoding method and state-of-the-art Transformer-based neural language models to enhance the accuracy and efficiency of GEC. The dynamic decoding method is capable of navigating the complexities of GEC and correcting a wide range of errors, including contextual and grammatical errors. The proposed model leverages the contextual information of words and sentences to generate a corrected output. To assess the effectiveness of our proposed framework, we conducted experiments using synthetic data and compared its performance with existing GEC systems. The results demonstrate a significant improvement in the accuracy of Indonesian GEC compared to existing methods.