High-resource Languages Research Articles

SexWEs: Domain-Aware Word Embeddings via Cross-Lingual Semantic Specialisation for Chinese Sexism Detection in Social Media

The goal of sexism detection is to mitigate negative online content targeting certain gender groups of people. However, the limited availability of labeled sexism-related datasets makes it problematic to identify online sexism for low-resource languages. In this paper, we address the task of automatic sexism detection in social media for one low-resource language -- Chinese. Rather than collecting new sexism data or building cross-lingual transfer learning models, we develop a cross-lingual domain-aware semantic specialisation system in order to make the most of existing data. Semantic specialisation is a technique for retrofitting pre-trained distributional word vectors by integrating external linguistic knowledge (such as lexico-semantic relations) into the specialised feature space. To do this, we leverage semantic resources for sexism from a high-resource language (English) to specialise pre-trained word vectors in the target language (Chinese) to inject domain knowledge. We demonstrate the benefit of our sexist word embeddings (SexWEs) specialised by our framework via intrinsic evaluation of word similarity and extrinsic evaluation of sexism detection. Compared with other specialisation approaches and Chinese baseline word vectors, our SexWEs shows an average score improvement of 0.033 and 0.064 in both intrinsic and extrinsic evaluations, respectively. The ablative results and visualisation of SexWEs also prove the effectiveness of our framework on retrofitting word vectors in low-resource languages.

Cross-Lingual and Cross-Domain Crisis Classification for Low-Resource Scenarios

Social media data has emerged as a useful source of timely information about real-world crisis events. One of the main tasks related to the use of social media for disaster management is the automatic identification of crisis-related messages. Most of the studies on this topic have focused on the analysis of data for a particular type of event in a specific language. This limits the possibility of generalizing existing approaches because models cannot be directly applied to new types of events or other languages. In this work, we study the task of automatically classifying messages that are related to crisis events by leveraging cross-language and cross-domain labeled data. Our goal is to make use of labeled data from high-resource languages to classify messages from other (low-resource) languages and/or of new (previously unseen) types of crisis situations. For our study we consolidated from the literature a large unified dataset containing multiple crisis events and languages. Our empirical findings show that it is indeed possible to leverage data from crisis events in English to classify the same type of event in other languages, such as Spanish and Italian (80.0% F1-score). Furthermore, we achieve good performance for the cross-domain task (80.0% F1-score) in a cross-lingual setting. Overall, our work contributes to improving the data scarcity problem that is so important for multilingual crisis classification. In particular, mitigating cold-start situations in emergency events, when time is of essence.

English–Assamese neural machine translation using prior alignment and pre-trained language model

In a multilingual country like India, automatic natural language translation plays a key role in building a community with different linguistic people. Many researchers have explored and improved the translation process for high-resource languages such as English, German, etc., and achieved state-of-the-art results. However, the unavailability of adequate data is the prime obstacle to automatic natural language translation of low-resource north-eastern Indian languages such as Mizo, Khasi, and Assamese. Though the recent past has witnessed a deluge in several automatic natural language translation systems for low-resource languages, the low values of their evaluation measures indicate the scope for improvement. In the recent past, the neural machine translation approach has significantly improved translation quality, and the credit goes to the availability of a huge amount of data. Subsequently, the neural machine translation approach for low-resource language is underrepresented due to the unavailability of adequate data. In this work, we have considered a low-resource English–Assamese pair using the transformer-based neural machine translation, which leverages the use of prior alignment and a pre-trained language model. To extract alignment information from the source–target sentences, we have used the pre-trained multilingual contextual embeddings-based alignment technique. Also, the transformer-based language model is built using monolingual target sentences. With the use of both prior alignment and a pre-trained language model, the transformer-based neural machine translation model shows improvement, and we have achieved state-of-the-art results for the English-to-Assamese and Assamese-to-English translation, respectively.

Context-aware Emotion Detection from Low-resource Urdu Language Using Deep Neural Network

Emotion detection (ED) plays a vital role in determining individual interest in any field. Humans use gestures, facial expressions, and voice pitch and choose words to describe their emotions. Significant work has been done to detect emotions from the textual data in English, French, Chinese, and other high-resource languages. However, emotion classification has not been well studied in low-resource languages (i.e., Urdu) due to the lack of labeled corpora. This article presents a publicly available Urdu Nastalique Emotions Dataset (UNED) of sentences and paragraphs annotated with different emotions and proposes a deep learning (DL)-based technique for classifying emotions in theUNEDcorpus. Our annotatedUNEDcorpus has six emotions for both paragraphs and sentences. We perform extensive experimentation to evaluate the quality of the corpus and further classify it using machine learning and DL approaches. Experimental results show that the developed DL-based model performs better than generic machine learning approaches with an F1 score of 85% on the UNED sentence-based corpus and 50% on the UNED paragraph-based corpus.

MBERT-GRU multilingual deep learning framework for hate speech detection in social media

One major issue plaguing online social media is hate speech, a complex phenomenon whose identification and target categorization have been studied by the natural language processing community. In recent years, notable studies have been made towards hate speech detection using various mechanisms varying from traditional machine learning to complex deep neural network models. However, these studies mainly focus on high-resource English language. The multilingual societies such as the Indian subcontinent: English, Hindi and Hindi-English code-mixed languages are widespread and convenient for the users. The research works studying hate speech detection in these languages are still very limited. To fill this gap, we propose an mBERT-GRU framework comprising of multilingual BERT embedding and bidirectional GRU layers to learn the cumulative features for hate speech detection and its target categorization. We evaluated our work on three datasets HASOC-2019, HS and HEOT to prove the competitive performance. Our results show that the proposed framework outperformed monolingual and state-of-the-art methods on English, Hindi and Hindi-English code-mixed datasets with Macro-F1 measure values of 0.87, 0.83 and 0.77, respectively.

Filtering and Extended Vocabulary based Translation for Low-resource Language Pair of Sanskrit-Hindi

Neural Machine Translation (NMT) is widely employed for language translation tasks because it performs better than the conventional statistical and phrase-based approaches. However, NMT techniques involve challenges, such as requiring a large and clean corpus of parallel data and the inability to deal with rare words. They need to be faster for real-time applications. More work needs to be done using NMT to address the challenges in translating Sanskrit, one of the oldest and rich languages known to the world, with its morphological richness and limited multilingual parallel corpus. There is usually no similar data between a language pair; hence, no application exists so far that can translate Sanskrit to/from other languages. This study presents an in-depth analysis to address these challenges with the help of a low-resource Sanskrit-Hindi language pair. We employ a novel training corpus filtering with extended vocabulary in a zero-shot transformer architecture. The structure of the Sanskrit language is thoroughly investigated to justify the use of each step. Furthermore, the proposed method is analyzed based on variations in sentence length and also applied to a high-resource language pair in order to demonstrate its efficacy.

CovTiNet: Covid text identification network using attention-based positional embedding feature fusion.

Covid text identification (CTI) is a crucial research concern in natural language processing (NLP). Social and electronic media are simultaneously adding a large volume of Covid-affiliated text on the World Wide Web due to the effortless access to the Internet, electronic gadgets and the Covid outbreak. Most of these texts are uninformative and contain misinformation, disinformation and malinformation that create an infodemic. Thus, Covid text identification is essential for controlling societal distrust and panic. Though very little Covid-related research (such as Covid disinformation, misinformation and fake news) has been reported in high-resource languages (e.g. English), CTI in low-resource languages (like Bengali) is in the preliminary stage to date. However, automatic CTI in Bengali text is challenging due to the deficit of benchmark corpora, complex linguistic constructs, immense verb inflexions and scarcity of NLP tools. On the other hand, the manual processing of Bengali Covid texts is arduous and costly due to their messy or unstructured forms. This research proposes a deep learning-based network (CovTiNet) to identify Covid text in Bengali. The CovTiNet incorporates an attention-based position embedding feature fusion for text-to-feature representation and attention-based CNN for Covid text identification. Experimental results show that the proposed CovTiNet achieved the highest accuracy of 96.61±.001% on the developed dataset (BCovC) compared to the other methods and baselines (i.e. BERT-M, IndicBERT, ELECTRA-Bengali, DistilBERT-M, BiLSTM, DCNN, CNN, LSTM, VDCNN and ACNN).

Learning Student Intents and Named Entities in the Education Domain

Detecting intents and extracting necessary contextual information (aka named entities) in input utterances are two fundamental tasks in understanding what the users say in chatbot systems. While most work in this field has been dedicated to high-resource languages in popular domains like business and home automation, little research has been done for low-resource languages, especially in a less popular domain like education. To narrow this gap, this paper presents the first study on learning to detect student intents and to recognize named entities in the education domain targeted to the Vietnamese language. Specifically, we first introduce a complete corpus consisting of 3690 utterances of students. It was manually annotated with both named entities and intent information at two levels of granularity: the fine-grained and the coarse-grained levels. We then systematically investigate different approaches to deal with the two tasks using not only independent but also joint learning architectures. The experimental results show that the joint architectures based on pre-trained language models are superior in boosting the performance of both tasks. They outperformed the conventional independent learning architectures which looked at the two tasks separately. Moreover, to further enhance the final performance, this paper proposes a technique to enrich the models with more useful linguistic features. Compared to the standard approaches, we achieve considerably better results for two tasks in both architectures. Overall, for the named entity recognition task, the best model yielded an F1 score of 88.61%. For the intent detection task, it yielded F1 scores of 94.36% and 91.62% at the coarse-grained and fine-grained levels, respectively.

Meta-Learning a Cross-lingual Manifold for Semantic Parsing

Abstract Localizing a semantic parser to support new languages requires effective cross-lingual generalization. Recent work has found success with machine-translation or zero-shot methods, although these approaches can struggle to model how native speakers ask questions. We consider how to effectively leverage minimal annotated examples in new languages for few-shot cross-lingual semantic parsing. We introduce a first-order meta-learning algorithm to train a semantic parser with maximal sample efficiency during cross-lingual transfer. Our algorithm uses high-resource languages to train the parser and simultaneously optimizes for cross-lingual generalization to lower-resource languages. Results across six languages on ATIS demonstrate that our combination of generalization steps yields accurate semantic parsers sampling ≤10% of source training data in each new language. Our approach also trains a competitive model on Spider using English with generalization to Chinese similarly sampling ≤10% of training data.1

Zero-shot cross-lingual transfer language selection using linguistic similarity

We study the selection of transfer languages for different Natural Language Processing tasks, specifically sentiment analysis, named entity recognition and dependency parsing. In order to select an optimal transfer language, we propose to utilize different linguistic similarity metrics to measure the distance between languages and make the choice of transfer language based on this information instead of relying on intuition. We demonstrate that linguistic similarity correlates with cross-lingual transfer performance for all of the proposed tasks. We also show that there is a statistically significant difference in choosing the optimal language as the transfer source instead of English. This allows us to select a more suitable transfer language which can be used to better leverage knowledge from high-resource languages in order to improve the performance of language applications lacking data. For the study, we used datasets from eight different languages from three language families.

A Voyage on Neural Machine Translation for Indic Languages

With the invention of deep learning concepts, Machine Translation (MT) migrated towards Neural Machine Translation (NMT) architectures, eventually from Statistical Machine Translation (SMT), which ruled MT for a few decades. Slowly, NMT paved its path into Indian MT research and witnessed many works for various language pairs in this regard. Numerous NMT architectures are floating across the international and national research pool; many claims to be state-of-the-art architectures. Though NMT for Indic languages (ILNMT) is giving better results for majority speaking language pairs, the translation quality is low due to a lack of significant resources. Automated machine translation models are unavailable for some less spoken Indic languages like Kashmiri and Dogri. Hence, there is increasing demand in the research to address the challenges of developing applicable MT models even when minuscule training data is available. Based on the corpus availability, the languages are categorized into High Resource Languages (HRLs), Low Resource Languages (LRLs), and Zero Resource Languages (ZRLs). Many Indic languages are classified into HRLs, LRLs, and ZRLs based on corpus availability. The vision behind this literature survey paper is to make this paper a collective source for all information regarding the predominant ILNMT architectures, the toolkits available for building NMT models, and various pre-trained language models needed by researchers who contribute to the ILNMT research community. In this survey paper, ILNMT architectures for different Indic languages are covered, e.g., Hindi, Tamil (HRLs), Kannada, Marathi (LRLs), Sinhala, and Nepali (ZRLs). There are a few language-specific survey papers on ILNMT, and this is one of the first kinds of survey papers where all the information is gathered under one canopy.

Multilingual Sentiment Analysis for Under-Resourced Languages: A Systematic Review of the Landscape

Sentiment analysis automatically evaluates people’s opinions of products or services. It is an emerging research area with promising advancements in high-resource languages such as Indo-European languages (e.g. English). However, the same cannot be said for languages with limited resources. In this study, we evaluate multilingual sentiment analysis techniques for under-resourced languages and the use of high-resourced languages to develop resources for low-resource languages. The ultimate goal is to identify appropriate strategies for future investigations. We report over 35 studies with different languages demonstrating an interest in developing models for under-resourced languages in a multilingual context. Furthermore, we illustrate the drawbacks of each strategy used for sentiment analysis. Our focus is to critically compare methods, employed datasets and identify research gaps. This study contributes to theoretical literature reviews with complete coverage of multilingual sentiment analysis studies from 2008 to date. Furthermore, we demonstrate how sentiment analysis studies have grown tremendously. Finally, because most studies propose methods based on deep learning approaches, we offer a deep learning framework for multilingual sentiment analysis that does not rely on the machine translation system. According to the meta-analysis protocol of this literature review, we found that, in general, just over 60% of the studies have used deep learning frameworks, which significantly improved the sentiment analysis performance. Therefore, deep learning methods are recommended for the development of multilingual sentiment analysis for under-resourced languages.

Division and the Digital Language Divide: A Critical Perspective on Natural Language Processing Resources for the South and North Korean Languages

Abstract: The digital world is marked by large asymmetries in the volume of content available between different languages. As a direct corollary, this inequality also exists, amplified, in the number of resources (labeled and unlabeled datasets, pretrained models, academic research) available for the computational analysis of these languages or what is generally called natural language processing (NLP). NLP literature divides languages between high- and low-resource languages. Thanks to early private and public investment in the field, the Korean language is generally considered to be a high-resource language. Yet, the good fortunes of Korean in the age of machine learning obscure the divided state of the language, as recensions of available resources and research solely focus on the standard language of South Korea, thus making it the sole representant of an otherwise diverse linguistic family that includes the Northern standard language as well as regional and diasporic dialects. This paper shows that the resources developed for the South Korean language do not necessarily transfer to the North Korean language. However, it also argues that this does not make North Korean a low-resource language. On one hand, South Korean resources can be augmented with North Korean data to achieve better performance. On the other, North Korean has more resources than commonly assumed. Retracing the long history of NLP research in North Korea, the paper shows that a large number of datasets and research exists for the North Korean language even if they are not easily available. The paper concludes by exploring the possibility of "unified" language models and underscoring the need for active NLP research collaboration across the Korean peninsula.

A Survey of Cross-Lingual Text Classification and Its Applications on Fake News Detection

Cross-lingual text classification is a challenging task in natural language processing. The objective is to build accurate text classification models for low-resource languages by transferring the knowledge learned from high-resource languages. The task has been studied since 2003 and has attracted significantly growing attention in the last decade due to the success of deep learning models in natural language processing. Many new methods have been proposed to address the challenges in cross-lingual text classification. Meanwhile, cross-lingual fake news detection is one of the most important applications of cross-lingual text classification. It has already created significant social impacts on alleviating the infodemic problem in low-resource languages. The research works on cross-lingual text classification and cross-lingual fake news detection have been growing rapidly in recent years. Therefore, a comprehensive survey is imperative to summarize existing algorithms for cross-lingual text classification and explain the connections among them. This paper systematically reviews research works on cross-lingual text classifications and their applications in cross-lingual fake news detection. We categorize the evolution of cross-lingual text classification methods into four phases: (1) Traditional text classification models with translation; (2) Cross-lingual word embedding-based methods, (3) Pretraining then finetuning-based methods, and (4) Pretraining then prompting-based methods. We first discuss and analyze the representative methods in each phase in detail. Second, we provide a detailed review of their applications in the emerging fake news detection problem. Finally, we explore the potential issues of this open problem and also discuss possible future directions.

Transfer Learning Based Neural Machine Translation of English-Khasi on Low-Resource Settings

Machine translation for low-resource language can be improved using various techniques. One such technique is the application of knowledge learned by training a model with high-resource language pair to another model with a low-resource language pair. The paper discusses the experiments and improvement of the results of neural machine translation using transfer learning for the English-Khasi language pair. Long short-term memory is used as the backbone architecture for the transfer learning model. The essential technique is the shared vocabulary, constructed utilizing the subword unit of byte pair encoding of the two pairs of languages and the subword unit of byte pair encoded datasets. Analysis and evaluation of the experimental output using human subjective evaluation, statistical evaluation, and automatic evaluation show positive results for the transfer learning system. A thorough analysis of word order agreement and comparisons of the outputs between the baseline and the transfer learning system is made. The analysis and evaluation methods portray that neural machine translation using transfer learning improves the translation accuracy for Khasi, a low-resource language.

Dzongkha Handwritten Digit Recognition using Machine Learning Techniques

Handwritten digit recognition has recently gained importance, attracting many researchers due to its use in various machine learning and computer vision applications. As technology and science progressing, there is a need for a system to recognize the handwritten script in several real-time applications to reduce human effort. There a lot of work has been done on the recognition and generation of handwritten digits of high-resource languages such as English. However, insufficient work has been done on Dzongkha digits recognition, as Dzongkha digits are low-resource and more complex than English patterns. This paper aims to perform handwritten character recognition of Dzongkha digit using several machine learning techniques. The unavailability of the Dzongkha handwritten digit dataset is the prime motivation behind this work. To facilitates the recognition of Dzongkha handwritten digit, we have collected the data of Dzongkha handwritten digit from indigenous and non-indigenous people of Bhutan and provided the dataset for further research. Moreover, we have used several machine algorithms, including a support vector machine, K-nearest neighbor, and decision tree. Among these algorithms, the support vector machine classification algorithm has achieved a remarkable result with an accuracy of 98.29%.

Pradvis vac: A socio-demographic dataset for determining the level of hatred severity in a low-resource Hinglish language

In multilingual societies like India, mixing the native language with English has become common during social media conversations. Further, due to the government’s digitization push, more people from rural India are joining social media platforms, resulting in the exponential growth of native or code-mixed content. The resultant content on social media is available for both positive (also termed as Hope Speech) as well as negative context (also termed as Hate Speech). To keep the social media clean and hate free, it is important to remove the negative content using machine learning filters. Since most of the existing hate content prediction models are trained using high resource language such as English, they fail to work on code-mixed text due to its spelling variance and non-grammatical structure. In addition, the lack of suitable training data could be one reason behind existing models’ poor performance on code-mixed text. To address these issues and promote research in this direction, we developed a manually annotated Hinglish Code-mixed corpus of 9254 comments taken from Twitter handles. We also annotated our data with the target audience and severity level. In each label, we provided a more fine-grained classification with three independent classes, and we built a Multi-label and Multi-class corpus for the severity of hate content prediction in Hinglish code-mixed text. Further, we modeled various supervised classifiers for severity prediction to validate our proposed data. The proposed models employ transformers for feature extraction and different machine learning and RNN (Recurrent neural network) models for classification. According to the experimental results, the target label combined with embeddings from Twitter text using the BiLSTM (a varient of RNN) classifier performed better on severity prediction, attaining an acceptable weighted F1 score.

루간다어-영어 병렬코퍼스 구축과 번역 모델 학습

Neural machine translation (NMT) has achieved great successes with large datasets, so NMT is more premised on high-resource languages. This continuously underpins the low resource languages such as Luganda due to the lack of high-quality parallel corpora, so even 'Google translate' does not serve Luganda at the time of this writing. In this paper, we build a parallel corpus with 41,070 pairwise sentences for Luganda and English which is based on three different open-sourced corpora. Then, we train NMT models with hyper-parameter search on the dataset. Experiments gave us a BLEU score of 21.28 from Luganda to English and 17.47 from English to Luganda. Some translation examples show high quality of the translation. We believe that our model is the first Luganda-English NMT model. The bilingual dataset we built will be available to the public.

Sentence Boundary Disambiguation for Tibetan Based on Attention Mechanism at the Syllable Level

Tibetan is a low-resource language with few existing electronic reference materials. The goal of Tibetan sentence boundary disambiguation (SBD) is to segment long text into sentences, and it is the foundation for downstream tasks corpora building. This study implemented the Tibetan SBD at the syllable level to avoid word segmentation (WS) errors affecting the accuracy of SBD. Specifically, the attention mechanism is introduced based on a recurrent neural network (RNN) to study Tibetan SBD. The primary objective is to determine, using a trained model, whether the shad contained in Tibetan text is the ending of the sentence, and implement experiments on syllable embedding and component embedding to measure the model's performance. The highest accuracy for Tibetan syllable embedding and component embedding is 96.23% and 95.40 %, respectively, and the F1 score reaches 96.23% and 95.37%, respectively. The experimental results demonstrate that the proposed method can achieve better results than the established rule-based and statistical methods without considering various syntactic and part-of-speech (POS) tagging rules. German and English data from the Europarl corpus and Thai data from the IWSLT2015 corpus are validated to prove the models’ reliability and generalizability. The results demonstrate that this method is efficient not only for low-resource languages but also for high-resource languages. More importantly, we can formally apply the experimental results of this study to the research of downstream tasks, such as machine translation and automatic summarization.

Threatening URDU Language Detection from Tweets Using Machine Learning

Technology’s expansion has contributed to the rise in popularity of social media platforms. Twitter is one of the leading social media platforms that people use to share their opinions. Such opinions, sometimes, may contain threatening text, deliberately or non-deliberately, which can be disturbing for other users. Consequently, the detection of threatening content on social media is an important task. Contrary to high-resource languages like English, Dutch, and others that have several such approaches, the low-resource Urdu language does not have such a luxury. Therefore, this study presents an intelligent threatening language detection for the Urdu language. A stacking model is proposed that uses an extra tree (ET) classifier and Bayes theorem-based Bernoulli Naive Bayes (BNB) as the based learners while logistic regression (LR) is employed as the meta learner. A performance analysis is carried out by deploying a support vector classifier, ET, LR, BNB, fully connected network, convolutional neural network, long short-term memory, and gated recurrent unit. Experimental results indicate that the stacked model performs better than both machine learning and deep learning models. With 74.01% accuracy, 70.84% precision, 75.65% recall, and 73.99% F1 score, the model outperforms the existing benchmark study.