Low-resource Languages Research Articles

Automatic graph construction and Exploring different types of LSTMs for Asian Hindi languages for Medical review Sentiment Analysis

Sentiment Analysis (SA) of medical reviews is crucial for improving healthcare outcomes. However, analyzing sentiment in low-resource languages such as Asian Hindi presents significant challenges. In this study, we propose an automatic graph construction approach to extract relevant features from medical reviews in Asian Hindi languages. We explore different types of Long Short-Term Memory (LSTMs), including traditional LSTMs, bidirectional LSTMs, and attention-based LSTMs, to classify the sentiment of medical reviews. Our proposed approach uses attention-based LSTM architecture and pre-trained Word2Vec embeddings to achieve high accuracy. We compare the proposed approach with existing models using various evaluation metrics, including accuracy, precision, recall, and F1-score. The results demonstrate that our proposed approach outperforms all existing models in terms of accuracy, achieving an accuracy score of 81%. These findings could have implications for improving healthcare outcomes by enabling better monitoring of patient feedback and identifying areas for improvement in medical services.

Few-shot Named Entity Recognition: Definition, Taxonomy and Research Directions

Recent years have seen an exponential growth (+98% in 2022 w.r.t. the previous year) of the number of research articles in the few-shot learning field, which aims at training machine learning models with extremely limited available data. The research interest toward few-shot learning systems for Named Entity Recognition (NER) is thus at the same time increasing. NER consists in identifying mentions of pre-defined entities from unstructured text, and serves as a fundamental step in many downstream tasks, such as the construction of Knowledge Graphs, or Question Answering. The need for a NER system able to be trained with few-annotated examples comes in all its urgency in domains where the annotation process requires time, knowledge and expertise (e.g., healthcare, finance, legal), and in low-resource languages. In this survey, starting from a clear definition and description of the few-shot NER (FS-NER) problem, we take stock of the current state-of-the-art and propose a taxonomy which divides algorithms in two macro-categories according to the underlying mechanisms: model-centric and data-centric. For each category, we line-up works as a story to show how the field is moving toward new research directions. Eventually, techniques, limitations, and key aspects are deeply analyzed to facilitate future studies.

Speech data collection system for KUI, a Low resourced tribal language

<p>A new generation of speech translation technology is being developed to enable natural cross-language communication. Research efforts must focus on large vocabulary, spontaneous speech, and speaker variances to accommodate the varying demands of speech recognition technologies. These are important issues that need to be resolved for the general application of voice recognition in realistic settings. Most languages with limited resources don’t even have any speech data. Creating speech corpora is extremely difficult and time-consuming. Among all, KUI is regarded as one of the low-resource languages. In this paper, we developed the speech dataset for the KUI language to document and preserve their culture, tradition, and history for future generations. We also discuss the design, data collection procedures, and implementations and outline the different research possibilities using our KUI dataset. This paper mainly describes the GUI and method for the collection of KUI speech more quickly. In this section, the statistics of the people who helped and contributed to the collection of this KUI dataset have been provided. This study details a novel method of gathering data for any speech dataset. Using this process, we collected 60 hours of speech data sampled at 16 kHz by three different devices such as a Zoom recorder, Mobile, and Laptop from 80 different speakers. Each speaker contributed 500 sentences in the KUI language. A GUI application is designed to capture the speeches of numerous speakers in the KUI language. Several guidelines are proposed and used for the collection of the KUI speech dataset. All the guidelines are based on real-time experience gained during the data collection process by our team members.</p>

Metadata for Efficient Management of Digital News Articles in Multilingual News Archives

The digital news preservation and management of low-resource languages are challenging tasks, especially in vast collections. Unique identification of individual digital objects is possible with well-defined attributes to assure efficient management, such as access, retrieval, preservation, usability, and transformability. The metadata element set is required to maximize the available attributes related to the digital objects. To create a comprehensive metadata set that contains all the necessary attributes and data about the digital news objects. It is more challenging and complicated when the archive contains articles from low-resourced and morphologically complex languages like Urdu and Arabic, which is difficult for machines to understand. The study presents challenges in low-resource languages (LRL) and research challenges. This metadata will help to link news articles based on similarity with other news articles stored in the digital news stories archive (DNSA) and ensures accessibility. In this study, we introduced 38 metadata elements set for the digital news stories preservation (DNSP) framework, of which 16 are explicit and 12 are implicit metadata elements. The paper presents how the digital news stories archive (DNSA) is enhanced to a multilingual archive and discusses the digital news stories extractor, which addresses major issues in implementing low-resource languages and facilitates normalized format migration. The extraction results are presented in detail for high-resource languages, that is, English, and low-resource languages (HRL), that is, Urdu and Arabic. The LRL encountered a high error rate during preservation compared to HRL, 10%, and 03%, respectively. The metadata extraction results show that HRL sources support all metadata elements as compared to LRL. The LRL has good support for explicit meta elements and many implicit meta elements with low extraction percentages. The LRL needs a more detailed study for accurate news content extraction and archiving for future access.

A multi-purpose audio-visual corpus for multi-modal Persian speech recognition: The Arman-AV dataset

Automatic lip reading has advanced significantly in recent years. However, these methods need large-scale datasets that are scarce for many low-resource languages. In this paper, we introduce a new multipurpose audio-visual dataset for Persian. The dataset contains approximately 220 h of videos from 1760 speakers. The dataset can be used for multiple tasks, such as lip reading, automatic speech recognition, audio-visual speech recognition, and speaker recognition. It is also the first large-scale lip reading dataset in this language. We provide a baseline method for each task and propose a technique to identify visemes (visual units of speech) in Persian. The visemes obtained by this technique improve the accuracy of the lip reading task by 7% relatively compared to the previously proposed visemes, which can be generalized to other languages as well.

Toward Best Practices for Training Multilingual Dense Retrieval Models

Dense retrieval models using a transformer-based bi-encoder architecture have emerged as an active area of research. In this article, we focus on the task of monolingual retrieval in a variety of typologically diverse languages using such an architecture. Although recent work with multilingual transformers demonstrates that they exhibit strong cross-lingual generalization capabilities, there remain many open research questions, which we tackle here. Our study is organized as a “best practices” guide for training multilingual dense retrieval models, broken down into three main scenarios: when a multilingual transformer is available, but training data in the form of relevance judgments are not available in the language and domain of interest (“have model, no data”); when both models and training data are available (“have model and data”); and when training data are available but not models (“have data, no model”). In considering these scenarios, we gain a better understanding of the role of multi-stage fine-tuning, the strength of cross-lingual transfer under various conditions, the usefulness of out-of-language data, and the advantages of multilingual vs. monolingual transformers. Our recommendations offer a guide for practitioners building search applications, particularly for low-resource languages, and while our work leaves open a number of research questions, we provide a solid foundation for future work.

Common latent representation learning for low-resourced spoken language identification

Common latent representation learning for low-resourced spoken language identification

A Chinese–Kazakh Translation Method That Combines Data Augmentation and R-Drop Regularization

Low-resource languages often face the problem of insufficient data, which leads to poor quality in machine translation. One approach to address this issue is data augmentation. Data augmentation involves creating new data by transforming existing data through methods such as flipping, cropping, rotating, and adding noise. Traditionally, pseudo-parallel corpora are generated by randomly replacing words in low-resource language machine translation. However, this method can introduce ambiguity, as the same word may have different meanings in different contexts. This study proposes a new approach for low-resource language machine translation, which involves generating pseudo-parallel corpora by replacing phrases. The performance of this approach is compared with other data augmentation methods, and it is observed that combining it with other data augmentation methods further improves performance. To enhance the robustness of the model, R-Drop regularization is also used. R-Drop is an effective method for improving the quality of machine translation. The proposed method was tested on Chinese–Kazakh (Arabic script) translation tasks, resulting in performance improvements of 4.99 and 7.7 for Chinese-to-Kazakh and Kazakh-to-Chinese translations, respectively. By combining the generation of pseudo-parallel corpora through phrase replacement with the application of R-Drop regularization, there is a significant advancement in machine translation performance for low-resource languages.

Improving Sequence-to-sequence Tibetan Speech Synthesis with Prosodic Information

There are about 6,000 languages worldwide, most of which are low-resource languages. Although the current speech synthesis (or text-to-speech, TTS) for major languages (e.g., Mandarin, English, French) has achieved good results, the voice quality of TTS for low-resource languages (e.g., Tibetan) still needs to be further improved. Because prosody plays a significant role in natural speech, the article proposes two sequence-to-sequence (seq2seq) Tibetan TTS models with prosodic information fusion to improve the voice quality of synthesized Tibetan speech. We first constructed a large-scale Tibetan corpus for seq2seq TTS. Then we designed a prosody generator to extract prosodic information from the Tibetan sentences. Finally, we trained two seq2seq Tibetan TTS models by fusing prosodic information, including feature-level and model-level prosodic information fusion. The experimental results showed that the proposed two seq2seq Tibetan TTS models, which fuse prosodic information, could effectively improve the voice quality of synthesized speech. Furthermore, the model-level prosodic information fusion only needs 60% ~ 70% of the training data to synthesize a voice similar to the baseline seq2seq Tibetan TTS. Therefore, the proposed prosodic information fusion methods can improve the voice quality of synthesized speech for low-resource languages.

Experimental Study of Morphological Analyzers for Topic Categorization in News Articles

Natural language processing refers to the ability of computers to understand text and spoken words similar to humans. Recently, various machine learning techniques have been used to encode a large amount of text and decode feature vectors of text successfully. However, understanding low-resource languages is in the early stages of research. In particular, Korean, which is an agglutinative language, needs sophisticated preprocessing steps, such as morphological analysis. Since morphological analysis in preprocessing significantly influences classification results, ideal and optimized morphological analyzers must be used. This study explored five state-of-the-art morphological analyzers for Korean news articles and categorized their topics into seven classes using term frequency–inverse document frequency and light gradient boosting machine frameworks. It was found that a morphological analyzer based on unsupervised learning achieved a computation time of 6 s in 500,899 tokens, which is 72 times faster than the slowest analyzer (432 s). In addition, a morphological analyzer using dynamic programming achieved a topic categorization accuracy of 82.5%, which is 9.4% higher than achieve when using the hidden Markov model (73.1%) and 13.4% higher compared to the baseline (69.1%) without any morphological analyzer in news articles. This study can provide insight into how each morphological analyzer extracts morphemes in sentences and affects categorizing topics in news articles.

Toward Better Automated Content Moderation in Low-Resource Languages

Toward Better Automated Content Moderation in Low-Resource Languages

Fake News Detection in Low Resource Languages using SetFit Framework

Social media has become an integral part of people’s lives, resulting in a constant flow of information. However, a concerning trend has emerged with the rapid spread of fake news, attributed to the lack of verification mechanisms. Fake news has far-reaching consequences, influencing public opinion, disrupting democracy, fuelingsocial tensions, and impacting various domains such as health, environment, and the economy. In order to identify fake news with data sparsity, especially with low resources languages such as Arabic and its dialects, we propose a few-shot learning fake news detection model based on sentence transformer fine-tuning, utilizing no crafted prompts and language model with few parameters. The experimental results prove that the proposed method can achieve higher performances with fewer news samples. This approach provided 71% F1 score on the Algerian dialect fake news dataset and 70% F1 score on the Modern Standard Arabic (MSA) version of the same dataset, which proves that the approach can work on the standard Arabic and its dialects. Therefore, the proposed model can identify fake news in several domains concerning the Algerian community such as politics, COVID-19, tourism, e-commerce, sport, accidents, and car prices.

Transfer Learning-Based Neural Machine Translation for Low-Resource Languages

College of Liberal Arts, Ludong University, Yantai 264025, China Chinese Lexicography Research Center, Yantai 264025, China Neural Machine Translation (NMT) improves readability by augmenting sentence suggestions based on the precise likelihood of the words. The word suggestions are trained using learning paradigms through repeated translations, word searches, and user inputs. However, the challenging process is the implication of NMT for low-resource language wherein the chances of false suggestions/ word substitutions are high. So, this article proposes a Likelihood-based Machine Translation Model (LMTM) for low-resource languages. The model uses word frequency and potential substitutions from less-known sentences to identify sentences with high precision. This is achieved through a combination of recurrence and substitutions using transfer learning. The identified high-likelihood words are used for sentence augmentation, and the entire set of words from the generated sentence updates the learning paradigm. The model suggests the highest likelihood words for NMT, preventing sentence falsification and ensuring accurate translations. The proposed model increases likelihood by 9.15%, correctness by 7.46%, and substitutions by 7.7%, respectively. It reduces falsification and time complexity by 9.33% and 8.52%, respectively. Overall, the LMTM improves translation quality for low-resource languages.

Towards better Chinese-centric neural machine translation for low-resource languages

The last decade has witnessed enormous improvements in science and technology, stimulating the growing demand for economic and cultural exchanges in various countries. Building a neural machine translation (NMT) system has become an urgent trend, especially in the low-resource setting. However, recent work tends to study NMT systems for low-resource languages centered on English, while a few studies focus on low-resource NMT systems centered on other languages such as Chinese. To achieve this, the low-resource multilingual translation challenge of the 2021 iFLYTEK AI Developer Competition provides the Chinese-centric multilingual low-resource NMT tasks, where participants are required to build NMT systems based on the provided low-resource samples. In this paper, we present the competition-winning system that leverages monolingual word embeddings data enhancement, bilingual curriculum learning, and contrastive re-ranking. In addition, a new Incomplete-Trust (In-trust) loss function is proposed to replace the traditional cross-entropy loss when training. The extensive experimental results demonstrate that the implementation of these ideas leads to better performance than other state-of-the-art methods. All the experimental codes are released at: https://github.com/WENGSYX/Low-resource-text-translation.

Deep Models for Low-Resourced Speech Recognition: Livvi-Karelian Case

Recently, there has been a growth in the number of studies addressing the automatic processing of low-resource languages. The lack of speech and text data significantly hinders the development of speech technologies for such languages. This paper introduces an automatic speech recognition system for Livvi-Karelian. Acoustic models based on artificial neural networks with time delays and hidden Markov models were trained using a limited speech dataset of 3.5 h. To augment the data, pitch and speech rate perturbation, SpecAugment, and their combinations were employed. Language models based on 3-grams and neural networks were trained using written texts and transcripts. The achieved word error rate metric of 22.80% is comparable to other low-resource languages. To the best of our knowledge, this is the first speech recognition system for Livvi-Karelian. The results obtained can be of a certain significance for development of automatic speech recognition systems not only for Livvi-Karelian, but also for other low-resource languages, including the fields of speech recognition and machine translation systems. Future work includes experiments with Karelian data using techniques such as transfer learning and DNN language models.

A Hybrid Optimization Approach for Neural Machine Translation Using LSTM+RNN with MFO for Under Resource Language (Telugu)

NMT (Neural Machine Translation) is an innovative approach in the field of machine translation, in contrast to SMT (statistical machine translation) and Rule-based techniques which has resulted annotable improvements. This is because NMT is able to overcome many of the shortcomings that are inherent in the traditional approaches. The Development of NMT has grown tremendously in the recent years but NMT performance remain under optimal when applied to low resource language pairs like Telugu, Tamil and Hindi. In this work a proposedmethod fortranslating pairs (Telugu to English) is attempted, an optimal approach which enhancesthe accuracy and execution time period.A hybrid method approach utilizing Long short-term memory (LSTM) and traditional Recurrent Neural Network (RNN) are used for testing and training of the dataset. In the event of long-range dependencies, LSTM will generate more accurate results than a standard RNN would endure and the hybrid technique enhances the performance of LSTM. LSTM is used during the encoding and RNN is used in decoding phases of NMT. Moth Flame Optimization (MFO) is utilized in the proposed system for the purpose of providing the encoder and decoder model with the best ideal points for training the data.

Demographical Based Sentiment Analysis for Detection of Hate Speech Tweets for Low Resource Language

Advancement in IT and communication technology provides the opportunity for social media users to communicate their ideas and thoughts across the globe within no time as well big data promulgated in a result of the communication process itself has immense challenges. Recently, the provision of freedom of speech has witnessed immense promulgation of offensive and hate speech content on the internet aimed the basic human rights violation. The detection of abusive content on social media for rich resource language has become a hot area for researchers in the recent past. However, low-resource languages are underprivileged due to the non-availability of large corpus and its complexity to understand. The proposed methodology mainly has two parts. One is to detect abusive content and the other is to have a demographical analysis of the Indigenously developed dataset. The process starts with the development of a unique unlabeled Urdu dataset of 0.2 M from Twitter through a web scrapper tool named snscraper. The dataset is collected against the 36 districts of Punjab from Pakistan and from the duration 2018- Apr 2022. The dataset is labeled into three target classes Neutral, Offensive, and Hate Speech. After data cleaning, the feature extraction process is achieved with the help of traditional techniques such as Bow and tf-idf with the combination of word and char n-gram and word embedding word2Vec. The dataset is trained on both machine learning algorithms SVM and Logistic regression and deep learning techniques Long Short Term Memory (LSTM) and Convolutional Neural Networks (CNN). The best F score achieved through LSTM on this dataset is 64 and accuracy is 93 through CNN algorithms. A Choropleth map is used for visualization of the dataset distributed among 36 districts of Punjab and a time series plot for time analysis covers five years duration from 2018-Apr to 22.

SentiUrdu-1M: A large-scale tweet dataset for Urdu text sentiment analysis using weakly supervised learning.

Low-resource languages are gaining much-needed attention with the advent of deep learning models and pre-trained word embedding. Though spoken by more than 230 million people worldwide, Urdu is one such low-resource language that has recently gained popularity online and is attracting a lot of attention and support from the research community. One challenge faced by such resource-constrained languages is the scarcity of publicly available large-scale datasets for conducting any meaningful study. In this paper, we address this challenge by collecting the first-ever large-scale Urdu Tweet Dataset for sentiment analysis and emotion recognition. The dataset consists of a staggering number of 1, 140, 821 tweets in the Urdu language. Obviously, manual labeling of such a large number of tweets would have been tedious, error-prone, and humanly impossible; therefore, the paper also proposes a weakly supervised approach to label tweets automatically. Emoticons used within the tweets, in addition to SentiWordNet, are utilized to propose a weakly supervised labeling approach to categorize extracted tweets into positive, negative, and neutral categories. Baseline deep learning models are implemented to compute the accuracy of three labeling approaches, i.e., VADER, TextBlob, and our proposed weakly supervised approach. Unlike the weakly supervised labeling approach, the VADER and TextBlob put most tweets as neutral and show a high correlation between the two. This is largely attributed to the fact that these models do not consider emoticons for assigning polarity.

Multilingual hope speech detection: A Robust framework using transfer learning of fine-tuning RoBERTa model

Hope Speech Detection (HSD) from social media is a new direction for promoting and supporting positive content to encourage harmony and positivity in society. As users of social media belong to different linguistic communities, hope speech detection is rarely studied as a multilingual task considering low-resource languages. Moreover, prior studies explored only monolingual techniques, and the Russian language is not addressed. This study tackles the issue of Multi-lingual Hope Speech Detection (MHSD) in English and Russian languages using the transfer learning paradigm with fine-tuning approach. We explore joint multi-lingual and translation-based approaches to tackle the task of multilingualism, where the latter approach adopts the translation mechanism to transform all content into one language and then classify them. The joint multi-lingual method handles it by designing a universal classifier for various languages. We explore the strengths of the Robustly Optimized BERT Pre-Training Approach (RoBERTa) that showed a benchmark in capturing the semantics and contextual information within the content. The proposed framework consists of several stages: 1) data preprocessing, 2) representation of data using RoBERTa models, 3) fine-tuning phase, and 4) classification of hope speech into two labels. A new Russian corpus for hope speech detection is built, containing YouTube comments. Several experiments are conducted in English and Russian languages by using semi-supervised bilingual English and Russian datasets. The findings show that the proposed framework demonstrated benchmark performance and outperformed the baselines. Furthermore, the translation-based approach (Russian-RoBERTa) offered the best performance by achieving 94% accuracy and 80.24% f1-score.

Detection of offensive terms in resource-poor language using machine learning algorithms.

The use of offensive terms in user-generated content on different social media platforms is one of the major concerns for these platforms. The offensive terms have a negative impact on individuals, which may lead towards the degradation of societal and civilized manners. The immense amount of content generated at a higher speed makes it humanly impossible to categorise and detect offensive terms. Besides, it is an open challenge for natural language processing (NLP) to detect such terminologies automatically. Substantial efforts are made for high-resource languages such as English. However, it becomes more challenging when dealing with resource-poor languages such as Urdu. Because of the lack of standard datasets and pre-processing tools for automatic offensive terms detection. This paper introduces a combinatorial pre-processing approach in developing a classification model for cross-platform (Twitter and YouTube) use. The approach uses datasets from two different platforms (Twitter and YouTube) the training and testing the model, which is trained to apply decision tree, random forest and naive Bayes algorithms. The proposed combinatorial pre-processing approach is applied to check how machine learning models behave with different combinations of standard pre-processing techniques for low-resource language in the cross-platform setting. The experimental results represent the effectiveness of the machine learning model over different subsets of traditional pre-processing approaches in building a classification model for automatic offensive terms detection for a low resource language, i.e., Urdu, in the cross-platform scenario. In the experiments, when dataset D1 is used for training and D2 is applied for testing, the pre-processing approach named Stopword removal produced better results with an accuracy of 83.27%. Whilst, in this case, when dataset D2 is used for training and D1 is applied for testing, stopword removal and punctuation removal were observed as a better preprocessing approach with an accuracy of 74.54%. The combinatorial approach proposed in this paper outperformed the benchmark for the considered datasets using classical as well as ensemble machine learning with an accuracy of 82.9% and 97.2% for dataset D1 and D2, respectively.