Language Identification Research Articles

Towards audio-based identification of Ethio-Semitic languages using recurrent neural network

In recent times, there is an increasing interest in employing technology to process natural language with the aim of providing information that can benefit society. Language identification refers to the process of detecting which speech a speaker appears to be using. This paper presents an audio-based Ethio-semitic language identification system using Recurrent Neural Network. Identifying the features that can accurately differentiate between various languages is a difficult task because of the very high similarity between characters of each language. Recurrent Neural Network (RNN) was used in this paper in relation to the Mel-frequency cepstral coefficients (MFCCs) features to bring out the key features which helps provide good results. The primary goal of this research is to find the best model for the identification of Ethio-semitic languages such as Amharic, Geez, Guragigna, and Tigrigna. The models were tested using an 8-h collection of audio recording. Experiments were carried out using our unique dataset with an extended version of RNN, Long Short Term Memory (LSTM) and Bidirectional Long Short Term Memory (BLSTM), for 5 and 10 s, respectively. According to the results, Bidirectional Long Short Term Memory (BLSTM) with a 5 s delay outperformed Long Short Term Memory (LSTM). The BLSTM model achieved average results of 98.1, 92.9, and 89.9% for training, validation, and testing accuracy, respectively. As a result, we can infer that the best performing method for the selected Ethio-Semitic language dataset was the BLSTM algorithm with MFCCs feature running for 5 s.

Arabic cyberbullying detecting using ensemble deep learning technique

<span lang="EN-US">There has been a huge growth in recent years interest in studies on abusive language and cyberbullying detection due to its effects on both individual victims and societies. Hate speech, bullying, racism, aggressive content, harassment and other forms of abuse have all significantly increased as a result of Facebook, Instagram, and other social media platforms (SMPs). Since there is a significant need to detect, control, and prohibit the circulation of offensive content on social networking sites, we undertook this study to automate the identification of abusive language or cyberbullying. Arabic data set is balanced and will be used in the offensive detection process. Recently, ensemble machine learning has been used to increase the effectiveness of categorization models. Arabic detection is more precise given that each spatial feature text can make references to every other contextual piece of information. The authors utilized a model that merged convolutional neural network (CNN) with bidirectional long short-term memory (Bi-LSTM) and inverse document frequency gated recurrent unit (GRU) in a hybrid fashion without any post-processing. Our work outperformed every other publicly released cutting-edge ensemble model in the specifications of the official deep learning challenge. The findings indicate that the three-layer inverse document frequency long short-term memory (LSTM) classifier surpassed other classifiers in accuracy with a score of 92.75% compared to different algorithms.</span>

OffensEval 2023: Offensive language identification in the age of Large Language Models

AbstractThe OffensEval shared tasks organized as part of SemEval-2019–2020 were very popular, attracting over 1300 participating teams. The two editions of the shared task helped advance the state of the art in offensive language identification by providing the community with benchmark datasets in Arabic, Danish, English, Greek, and Turkish. The datasets were annotated using the OLID hierarchical taxonomy, which since then has become the de facto standard in general offensive language identification research and was widely used beyond OffensEval. We present a survey of OffensEval and related competitions, and we discuss the main lessons learned. We further evaluate the performance of Large Language Models (LLMs), which have recently revolutionalized the field of Natural Language Processing. We use zero-shot prompting with six popular LLMs and zero-shot learning with two task-specific fine-tuned BERT models, and we compare the results against those of the top-performing teams at the OffensEval competitions. Our results show that while some LMMs such as Flan-T5 achieve competitive performance, in general LLMs lag behind the best OffensEval systems.

Cross-data recognition on sign language letters based on convolutional neural networks

With an estimated 1.5 billion hearing-impaired people globally, sign language is a vital means of communication among them. Meanwhile, the complexity and diversity of sign languages across different regions bring challenges to researchers. As existing studies in this field usually lack the implementation of recognition models with multiple datasets, which limits their practical value in real-life scenarios, this study intends to get around this constraint. This work constructs a baseline American Sign Language Letter Recognition model using Convolutional Neural Network (CNN) and then optimizes it to enhance its ability. Finally, cross-data recognition is carried out. In order to train and evaluate the model, this study gathers data from several sources, including the MNIST sign language set and real-life photos. It also investigates how data augmentation affects recognition ability. Consequently, the CNN model is able to recognize hand gestures for different alphabetic letters in solid backgrounds. Its accurate rate of it reaches about 99.83%. For the extra real-life dataset, which contains 96 images, the model could still detect the majority of the images with sign language equivalents, despite some accuracy loss in more crowded backgrounds. This work, in general, concentrates on the potential of CNNs for sign language identification and highlights the significance of cross-data recognition in creating useful recognition models.

Trustful Test Suites for Natural Language Processing

Machine Translation (MT) research has witnessed continuous growth, accompanied by an increasing demand for automated error detection and correction in textual content. In response, Unbabel has developed a hybrid approach that combines machine translation with human editors in post-edition (PE) to provide high-quality translations. To facilitate the tasks of post-editors, Unbabel has created a proprietary error detection tool named Smartcheck, designed to identify errors and provide correction suggestions. Traditionally, the evaluation of translation errors relies on carefully curated annotated texts, categorized based on error types, which serve as the evaluation standard or Test Suites for assessing the accuracy of machine translation systems. However, it is crucial to consider that the effectiveness of evaluation sets can significantly impact the outcomes of evaluations. In fact, if evaluation sets do not accurately represent the content or possess inherent flaws, the decisions made based on such evaluations may inadvertently yield undesired effects. Hence, it is of utmost importance to employ suitable datasets containing representative data of the structures needed for each system, including Smartcheck. In this paper we present the methodology that has been developed and implemented to create reliable and revised Test Suites specifically designed for the evaluation process of MT systems and error detection tools. By using these meticulously curated Test Suites to evaluate proprietary systems and tools, we can ensure the trustworthiness of the conclusions and decisions derived from the evaluations. This methodology accomplished robust identification of problematic error types, grammar-checking rules, and language- and/or register-specific issues, leading to the adoption of effective production measures. With the integration of Smartcheck’s reliable and accurate correction suggestions and the improvements made to the post-edition revision process, the work presented herein led to a noticeable improvement in the translation quality delivered to customers.

Lessons Learned in Transcribing 5000 h of Air Traffic Control Communications for Robust Automatic Speech Understanding

Voice communication between air traffic controllers (ATCos) and pilots is critical for ensuring safe and efficient air traffic control (ATC). The handling of these voice communications requires high levels of awareness from ATCos and can be tedious and error-prone. Recent attempts aim at integrating artificial intelligence (AI) into ATC communications in order to lessen ATCos’s workload. However, the development of data-driven AI systems for understanding of spoken ATC communications demands large-scale annotated datasets, which are currently lacking in the field. This paper explores the lessons learned from the ATCO2 project, which aimed to develop an unique platform to collect, preprocess, and transcribe large amounts of ATC audio data from airspace in real time. This paper reviews (i) robust automatic speech recognition (ASR), (ii) natural language processing, (iii) English language identification, and (iv) contextual ASR biasing with surveillance data. The pipeline developed during the ATCO2 project, along with the open-sourcing of its data, encourages research in the ATC field, while the full corpus can be purchased through ELDA. ATCO2 corpora is suitable for developing ASR systems when little or near to no ATC audio transcribed data are available. For instance, the proposed ASR system trained with ATCO2 reaches as low as 17.9% WER on public ATC datasets which is 6.6% absolute WER better than with “out-of-domain” but gold transcriptions. Finally, the release of 5000 h of ASR transcribed speech—covering more than 10 airports worldwide—is a step forward towards more robust automatic speech understanding systems for ATC communications.

Meta adversarial learning improves low-resource speech recognition

Low-resource automatic speech recognition is a challenging task. To resolve this issue, multilingual meta-learning learns a better model initialization from many source languages, allowing for rapid adaption to target languages. However, differences in data scales and learning difficulties vary greatly from one language to another. As a result, the model favors large-scale and simple source languages. Moreover, the shared semantic space of various languages is difficult to learn due to a lack of restrictions on multilingual pre-training. In this paper, we propose a meta adversarial learning approach to address this problem. The meta-learner will be guided to learn language-independent information by using an adversarial auxiliary objective of language identification, which makes the shared semantic space more compact and improves model generalization. Additionally, we optimize adversarial training using Wasserstein distance and temporal normalization, enabling more stable and simple training. Experiment results on IARPA BABEL and OpenSLR show a significant performance improvement. It also outperforms state-of-the-art results by a large margin in all target languages, and especially in few-shot settings. Finally, we demonstrate how our method is superior by using t-SNE visualization.

Identification of 10 Regional Indonesian Languages Using Machine Learning

Language Identification plays a pivotal role in deciphering the rich tapestry of Indonesia's diverse regional languages, encompassing a wide spectrum of scripts, and spoken forms. Language Identification, an integral component of Natural Language Processing, is frequently addressed through Text Classification. In this study, we embark on the task of identifying 10 Indonesian languages, leveraging the NusaX dataset, with the overarching objective of contextual language determination. To achieve this, we harness a diverse array of machine learning techniques, including Support Vector Machine, Naïve Bayes Classifier, Decision Tree, Rocchio Classification, Logistic Regression, and Random Forest. We complement these methods with two distinct feature extraction approaches: N-gram and TF-IDF. This comprehensive approach enables us to construct robust models for language identification. Our findings unveil the strong efficacy of these models in discerning Indonesian languages, with the Naïve Bayes Classifier emerging as the frontrunner, achieving an impressive accuracy rate of 99.2% with TF-IDF and an even more remarkable 99.4% with N-Gram. To gain deeper insights, we delve into error analysis, revealing that misclassifications often stem from shared words across different languages. This research is underpinned by the necessity for a robust language identification model, underscoring its critical role within the complex linguistic landscape of Indonesian regional languages. These results hold great promise for applications in automated language processing and understanding within this diverse and multifaceted linguistic context.

Pars-OFF: A Benchmark for Offensive Language Detection on Farsi Social Media

With the increasing use of social media with its ability for users to share comments immediately, the extent of a system to identify offensive content has become a necessity in all languages. Due to the lack of publicly available resources on offensive language identification for Farsi, which has more than 110 million speakers, we present Pars-OFF, a three-layered annotated corpus for offensive language detection in Farsi to fill the existing gap. The introduced corpus contains 10,563 data samples. The tweets have been collected with a combination of similarity-based and keyword-based data selection techniques to avoid severe unbalancedness. Additionally, as a baseline, this paper reports the performance of the traditional machine learning approaches and Transformer based models over the Pars-OFF dataset. The best performance was obtained by the BERT+fastText model, yielding the F1-Macro score of 89.57.

The development and productivity of a measure for identifying low language abilities in children aged 24–36 months

BackgroundAccurate early identification of children with low language ability is important but existing measures generally have low sensitivity. This remains an area of concern for preventive and public health services. This study aimed to create and evaluate a measure of child language, communication and related risks which can be used by community health nurses to accurately identify children with low language aged 24–30 months.MethodsThe Early Language Identification Measure (ELIM) was developed and comprised five measurement sections, each measuring different aspects of development combined into a single measure. This was tested blind against a reference standard language measure, the Preschool Language Scale-5 (PLS-5), at the universal 24–30-month health visitor review in England. The threshold for likely low language was the tenth centile or below on the PLS-5. The aim was to ascertain the performance of the five individual sections in the scale, and consider the optimum combination of sections, for predicting low language ability. Specificity, sensitivity, and positive and negative predictive values were reported for each of the five sections of the ELIM alone and in conjunction with each other. The performance for children from monolingual English-speaking families and those who spoke languages other than English were also considered separately.ResultsThree hundred and seventy-six children were assessed on both the ELIM identification measure and the PLS-5 with 362 providing complete data. While each section of the ELIM predicted low language ability, the optimal combination for predicting language outcome was the parent reported vocabulary checklist coupled with the practitioner observation of the child’s communication and related behaviours. This gave a sensitivity of 0·98 with a specificity of 0·63.ConclusionsA novel measure has been developed which accurately identifies children at risk of low language, allowing clinicians to target resources efficiently and intervene early.

Ethio-Semitic language identification using convolutional neural networks with data augmentation

Ethio-Semitic language identification using convolutional neural networks with data augmentation

Common latent representation learning for low-resourced spoken language identification

Common latent representation learning for low-resourced spoken language identification

Classifying cuneiform symbols using machine learning algorithms with unigram features on a balanced dataset

Abstract Problem Recognizing written languages using symbols written in cuneiform is a tough endeavor due to the lack of information and the challenge of the process of tokenization. The Cuneiform Language Identification (CLI) dataset attempts to understand seven cuneiform languages and dialects, including Sumerian and six dialects of the Akkadian language: Old Babylonian, Middle Babylonian Peripheral, Standard Babylonian, Neo-Babylonian, Late Babylonian, and Neo-Assyrian. However, this dataset suffers from the problem of imbalanced categories. Aim Therefore, this article aims to build a system capable of distinguishing between several cuneiform languages and solving the problem of unbalanced categories in the CLI dataset. Methods Oversampling technique was used to balance the dataset, and the performance of machine learning algorithms such as Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Decision Tree (DT), Random Forest (RF), and deep learning such as deep neural networks (DNNs) using the unigram feature extraction method was investigated. Results The proposed method using machine learning algorithms (SVM, KNN, DT, and RF) on a balanced dataset obtained an accuracy of 88.15, 88.14, 94.13, and 95.46%, respectively, while the DNN model got an accuracy of 93%. This proves improved performance compared to related works. Conclusion This proves the improvement of classifiers when working on a balanced dataset. The use of unigram features also showed an improvement in the performance of the classifier as it reduced the size of the data and accelerated the processing process.

Acculturation gaps among Afghan refugee families in Canada: Implications for family relationships, adaptation outcomes, and subjective wellbeing of emerging adults

The present study applied and extended the Acculturation Gap-Distress Model to a sample of Afghan emerging adults and their parents living in Canada. Using two different methods to operationalize the “gap”, the impact of acculturation gaps on family relationships, adaptation outcomes, and subjective wellbeing of Afghan emerging adults was examined. Acculturation was evaluated concerning both Canadian and Afghan cultures, and independently for the language, identity, and behavioural domains of acculturation. The following three themes emerged in the results: a) Emerging adults’ proficiency in the Farsi language and higher identification with Afghan culture are important factors behind their family relationships, adaptation outcomes, and subjective wellbeing. b) Parents’ identification with the Canadian culture is an important factor behind emerging adults’ psychological adaptation and subjective wellbeing. c) The parent-emerging adult gap in Canadian identity acculturation was the only consequential gap associated with emerging adults’ lower reports of family cohesion.

Boosting Low-Resource Speech Recognition in Air Traffic Communication via Pretrained Feature Aggregation and Multi-Task Learning

Developing a robust Automatic Speech Recognition (ASR) system usually requires a large amount of well-annotated samples which is extremely hard to build in the Air Traffic Control (ATC) due to domain-specific knowledge. In this work, we present a novel approach to improve ASR performance in the ATC domain by integrating Self-Supervised Learning and Multi-Task learning into a unified framework. Specifically, the proposed framework follows a two-stage training paradigm, i.e., (a) learning universal acoustic representations by employing the wav2vec 2.0 model and (b) jointly finetuning the model by the ASR, Speaker Role Identification, and Language Identification tasks. To capture the task-specific representations, an attention-guided feature aggregation module is dedicatedly designed to disentangle the discriminative representations from the pretrained features. In addition, the uncertainty-based loss combination strategy is employed to balance the loss weights for each task in a learnable manner. Finally, we conduct experiments to validate the technical improvements in a real-world ATC dataset. Experimental results demonstrated that the proposed framework outperforms competitive baselines among all tasks.

Crossing language identification: Multilingual ASR framework based on semantic dataset creation & Wav2Vec 2.0

This study proposes an innovative methodology to enhance the performance of multilingual Automatic Speech Recognition (ASR) systems by capitalizing on the high semantic similarity between sentences across different languages and eliminating the requirement for Language Identification (LID). To achieve this, special bilingual datasets were created from the Mozzila Common Voices datasets in Spanish, Russian, and Portuguese. The process involves computing sentence embeddings using Language-agnostic BERT and selecting sentence pairs based on high and low cosine similarity. Subsequently, we train the Wav2vec 2.0 XLSR53 model on these datasets and assess its performance utilizing Character Error Rate (CER) and Word Error Rate (WER) metrics. The experimental results indicate that models trained on high-similarity samples consistently surpass their low-similarity counterparts, emphasizing the significance of high semantic similarity data selection for precise and dependable ASR performance. Furthermore, the elimination of LID contributes to a simplified system with reduced computational costs and the capacity for real-time text output. The findings of this research offer valuable insights for the development of more efficient and accurate multilingual ASR systems, particularly in real-time and on-device applications.

A Hybrid Convolutional Bi-Directional Gated Recurrent Unit System for Spoken Languages of JK and Ladakhi

Spoken language identification is the process of recognising language in an audio segment and is the precursor for several technologies such as automatic call routing, language recognition, multilingual conversation, language parsing, and sentimental analysis. Language identification has become a challenging task for low-resource languages like Kashmiri and Ladakhi spoken in the UT’s of Jammu and Kashmir (JK) and Ladakh, India. This is mainly due to speaker variations like duration, moderator, and ambiance particularly when training and testing are done on different datasets whilst analysing the accuracy of language identification system in actual implementation, thus producing low accuracy results. In order to tackle this problem, we propose a hybrid convolutional bi-directional gated recurrent unit (Bi-GRU) utilising the effects of both static and dynamic behaviour of the audio signal in order to achieve better results as compared to state-of-the-art models. The audio signals are first converted into two-dimensional structures called Mel-spectrograms to represent the frequency distribution over time. To investigate the spectral behaviour of audio signals, we employ a convolutional neural network (CNN) that perceives Mel-spectrograms in multiple dimensions. The CNN-learned feature vector serves as input to the Bi-GRU that maintains the dynamic behaviour of the audio signal. Experiments are done on six spoken languages, i.e. Ladakhi, Kashmiri, Hindi, Urdu, English, and Dogri. The data corpora used for experimentation are the International Institute of Information Technology Hyderabad-Indian Language Speech Corpus (IIITH-ILSC) and the self-created data corpus for the Ladakhi language. The model is tested on two datasets, i.e. speaker-dependent and speaker-independent. Results show that when validating the efficiency of our proposed model on both speaker-dependent and speaker-independent datasets, we achieve optimal accuracies of 99% and 91%, respectively, thus achieving promising results in comparison to the state-of-the-art models available.

IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian Languages

A cornerstone in AI research has been the creation and adoption of standardized training and test datasets to earmark the progress of state-of-the-art models. A particularly successful example is the GLUE dataset for training and evaluating Natural Language Understanding (NLU) models for English. The large body of research around self-supervised BERT-based language models revolved around performance improvements on NLU tasks in GLUE. To evaluate language models in other languages, several language-specific GLUE datasets were created. The area of speech language understanding (SLU) has followed a similar trajectory. The success of large self-supervised models such as wav2vec2 enable creation of speech models with relatively easy to access unlabelled data. These models can then be evaluated on SLU tasks, such as the SUPERB benchmark. In this work, we extend this to Indic languages by releasing the IndicSUPERB benchmark. Specifically, we make the following three contributions. (i) We collect Kathbath containing 1,684 hours of labelled speech data across 12 Indian languages from 1,218 contributors located in 203 districts in India. (ii) Using Kathbath, we create benchmarks across 6 speech tasks: Automatic Speech Recognition, Speaker Verification, Speaker Identification (mono/multi), Language Identification, Query By Example, and Keyword Spotting for 12 languages. (iii) On the released benchmarks, we train and evaluate different self-supervised models alongside the a commonly used baseline FBANK. We show that language-specific fine-tuned models are more accurate than baseline on most of the tasks, including a large gap of 76% for Language Identification task. However, for speaker identification, self-supervised models trained on large datasets demonstrate an advantage. We hope IndicSUPERB contributes to the progress of developing speech language understanding models for Indian languages.

Corpus creation and language identification for code-mixed Indonesian-Javanese-English Tweets.

With the massive use of social media today, mixing between languages in social media text is prevalent. In linguistics, the phenomenon of mixing languages is known as code-mixing. The prevalence of code-mixing exposes various concerns and challenges in natural language processing (NLP), including language identification (LID) tasks. This study presents a word-level language identification model for code-mixed Indonesian, Javanese, and English tweets. First, we introduce a code-mixed corpus for Indonesian-Javanese-English language identification (IJELID). To ensure reliable dataset annotation, we provide full details of the data collection and annotation standards construction procedures. Some challenges encountered during corpus creation are also discussed in this paper. Then, we investigate several strategies for developing code-mixed language identification models, such as fine-tuning BERT, BLSTM-based, and CRF. Our results show that fine-tuned IndoBERTweet models can identify languages better than the other techniques. This is the result of BERT's ability to understand each word's context from the given text sequence. Finally, we show that sub-word language representation in BERT models can provide a reliable model for identifying languages in code-mixed texts.

Systematic keyword and bias analyses in hate speech detection

Hate speech detection refers broadly to the automatic identification of language that may be considered discriminatory against certain groups of people. The goal is to help online platforms to identify and remove harmful content. Humans are usually capable of detecting hatred in critical cases, such as when the hatred is non-explicit, but how do computer models address this situation? In this work, we aim to contribute to the understanding of ethical issues related to hate speech by analysing two transformer-based models trained to detect hate speech. Our study focuses on analysing the relationship between these models and a set of hateful keywords extracted from the three well-known datasets. For the extraction of the keywords, we propose a metric that takes into account the division among classes to favour the most common words in hateful contexts. In our experiments, we first compared the overlap between the extracted keywords with the words to which the models pay the most attention in decision-making. On the other hand, we investigate the bias of the models towards the extracted keywords. For the bias analysis, we characterize and use two metrics and evaluate two strategies to try to mitigate the bias. Surprisingly, we show that over 50% of the salient words of the models are not hateful and that there is a higher number of hateful words among the extracted keywords. However, we show that the models appear to be biased towards the extracted keywords. Experimental results suggest that fitting models with hateful texts that do not contain any of the keywords can reduce bias and improve the performance of the models.