Language Identification System Research Articles

Generative adversarial networks for whispered to voiced speech conversion: a comparative study

AbstractGenerative Adversarial Networks (GANs) have demonstrated promising results as end-to-end models for whispered to voiced speech conversion. Leveraging non-autoregressive systems like GANs capable of performing conditional waveform generation eliminates the need for separate models to estimate voiced speech features, and leads to faster inference compared to autoregressive methods. This study aims to identify the optimal GAN architecture for the whispered to voiced speech conversion task by comparing six state-of-the-art models. Furthermore, we present a method for evaluating the preservation of speaker identity and local accent, using embeddings obtained from speaker- and language identification systems. Our experimental results show that building the speech conversion system based on the HiFi-GAN architecture yields the best objective evaluation scores, outperforming the baseline by $$\sim$$ ∼ 9% relative using frequency-weighted Signal-to-Noise Ratio and Log Likelihood Ratio, as well as by $$\sim$$ ∼ 29% relative using Root Mean Squared Error. In subjective tests, HiFi-GAN yielded a mean opinion score of 2.9, significantly outperforming the baseline with a score of 1.4. Furthermore, HiFi-GAN enhanced ASR performance and preserved speaker identity and accent, with correct language detection rates of up to $$\sim$$ ∼ 98%.

Convolutional neural network based language identification system: A spectrogram based approach

Convolutional neural network based language identification system: A spectrogram based approach

Improved Arithmetic Optimization Algorithm with Transfer Learning based Arabic Sign Language Identification System

The Arabic sign language (ArSL) has witnessed ground-breaking research activities to identify hand gestures and signs through the deep learning (DL) model. SL is a unique communication tool that bridges the gaps between people with hearing impairment and ordinary people. The ArSL recognition system is of immense importance for different groups of people since it enables individuals with hearing impairment to communicate effectively. In SLs, signs are characterized by discrepancies in hand positions, shapes, motions, body parts, and facial expressions, posing a crucial threat to visual recognition in computer vision (CV). An automated sign detection technique needs two primary courses of action: the recognition of specific features and the classification of the input dataset. Previously, several approaches for detecting and classifying SLs had been proposed. In this study, an Improved Metaheuristics with Transfer Learning based Arabic Sign Language Identification System (IMTL-ArSL) method is developed. The primary objective of the IMTL-ArSL method is to detect and classify the existence of various signs in the Arabic language. In the IMTL-ArSL model, the bilateral filtering (BF) model is initially used for preprocessing. Besides, the Residual Network (ResNet50v2) model is applied for extracting features, and an improved arithmetic optimization algorithm (IAOA) model is utilized for hyperparameter tuning. Finally, a gated recurrent unit (GRU) network is exploited to identify sign languages. The empirical analysis of the IMTL-ArSL approach is tested using a benchmark dataset. The experimental values of the IMTL-ArSL approach portrayed a superior accuracy value of 93.87% over other techniques.

Implementation of Sibi Sign Language Realtime Detection Program (Case Studi At Sekolah Luar Biasa Negeri 1 Tabanan)

Indonesian deaf people utilize SIBI to communicate using spoken words, gestures, facial expressions, and body language. SIBI, certified for Special Schools (SLB), helps deaf pupils communicate. This project implements SIBI (Indonesian Sign Language System) a real-time detection algorithm at Sekolah Luar Biasa Negeri 1 Tabanan using image processing and YoloV8 ultralytics deep learning. The program trains a sign language gesture detection model on Google Colab's GPU. The SIBI sign language images were used to train a YoloV8 object detection model. The camera captures movements, which the YoloV8 algorithm trained on SIBI gesture data processes. It can recognize gestures in real time and generate text to non-sign language users. The dataset has 107 class vocabulary and 7 class affix prefixes for complete gesture recognition. Shirt color, room brightness, and webcam quality affect detection rates. Optimal detection accuracy is 87.74% and subpar 58.02%. Despite these limitations, the strategy helps deaf students communicate more effectively with non-sign language speakers. This program improves inclusivity and communication in schools, making learning easier for hearing-impaired pupils. This work provides a reliable and quick sign language identification system to help deaf educators and caregivers with daily interactions and education.

Leveraging BERT to Improve Spoken Language Identification of Code-Switching Speech

Language identification (LID) involves automatically determining the language being spoken in a given segment. In code-switching speech, there are rapid switches between two or more languages within a single conversation. Despite LID attaining high accuracy on medium or long utterances, the performance on short utterances of code-switching speech is currently unsatisfactory. We propose a bidirectional encoder representation from transformers (BERT)-based LID system (BERT-LID) to enhance LID performance, especially for short-duration code-switching speech segments. The original BERT model is expanded by incorporating phonetic posterior grams (PPGs) extracted from a front-end phone recognizer as input. Then it is followed by the deployment of an optimal deep classifier for LID. Our BERT-LID model demonstrates a significant improvement of approximately 6.5% in accuracy for long-segment identification and 19.9% for short-segment identification, thereby demonstrating its effectiveness in code-switching speech LID tasks.

A comparison of cepstral and spectral features using recurrent neural network for spoken language identification

Spoken language identification is the process of confirming labels regarding the language of an audio slice regardless of various features such as length, ambiance, duration, topic or message, age, gender, region, emotions, etc. Language identification systems are of great significance in the domain of natural language processing, more specifically multi-lingual machine translation, language recognition, and automatic routing of voice calls to particular nodes speaking or knowing a particular language. In his paper, we are comparing results based on various cepstral and spectral feature techniques such as Mel-frequency Cepstral Coefficients (MFCC), Relative spectral-perceptual linear prediction coefficients (RASTA-PLP), and spectral features (roll-off, flatness, centroid, bandwidth, and contrast) in the process of spoken language identification using Recurrent Neural Network-Long Short Term Memory (RNN-LSTM) as a procedure of sequence learning. The system or model has been implemented in six different languages, which contain Ladakhi and the five official languages of Jammu and Kashmir (Union Territory). The dataset used in experimentation consists of TV audio recordings for Kashmiri, Urdu, Dogri, and Ladakhi languages. It also consists of standard corpora IIIT-H and VoxForge containing English and Hindi audio data. Pre-processing of the dataset is done by slicing different types of noise with the use of the Spectral Noise Gate (SNG) and then slicing into audio bursts of 5 seconds duration. The performance is evaluated using standard metrics like F1 score, recall, precision, and accuracy. The experimental results showed that using spectral features, MFCC and RASTA-PLP achieved an average accuracy of 76%, 83%, and 78%, respectively. Therefore, MFCC proved to be the most convenient feature to be exploited in language identification using a recurrent neural network long short-term memory classifier.

Towards audio-based identification of Ethio-Semitic languages using recurrent neural network

In recent times, there is an increasing interest in employing technology to process natural language with the aim of providing information that can benefit society. Language identification refers to the process of detecting which speech a speaker appears to be using. This paper presents an audio-based Ethio-semitic language identification system using Recurrent Neural Network. Identifying the features that can accurately differentiate between various languages is a difficult task because of the very high similarity between characters of each language. Recurrent Neural Network (RNN) was used in this paper in relation to the Mel-frequency cepstral coefficients (MFCCs) features to bring out the key features which helps provide good results. The primary goal of this research is to find the best model for the identification of Ethio-semitic languages such as Amharic, Geez, Guragigna, and Tigrigna. The models were tested using an 8-h collection of audio recording. Experiments were carried out using our unique dataset with an extended version of RNN, Long Short Term Memory (LSTM) and Bidirectional Long Short Term Memory (BLSTM), for 5 and 10 s, respectively. According to the results, Bidirectional Long Short Term Memory (BLSTM) with a 5 s delay outperformed Long Short Term Memory (LSTM). The BLSTM model achieved average results of 98.1, 92.9, and 89.9% for training, validation, and testing accuracy, respectively. As a result, we can infer that the best performing method for the selected Ethio-Semitic language dataset was the BLSTM algorithm with MFCCs feature running for 5 s.

A Hybrid Convolutional Bi-Directional Gated Recurrent Unit System for Spoken Languages of JK and Ladakhi

Spoken language identification is the process of recognising language in an audio segment and is the precursor for several technologies such as automatic call routing, language recognition, multilingual conversation, language parsing, and sentimental analysis. Language identification has become a challenging task for low-resource languages like Kashmiri and Ladakhi spoken in the UT’s of Jammu and Kashmir (JK) and Ladakh, India. This is mainly due to speaker variations like duration, moderator, and ambiance particularly when training and testing are done on different datasets whilst analysing the accuracy of language identification system in actual implementation, thus producing low accuracy results. In order to tackle this problem, we propose a hybrid convolutional bi-directional gated recurrent unit (Bi-GRU) utilising the effects of both static and dynamic behaviour of the audio signal in order to achieve better results as compared to state-of-the-art models. The audio signals are first converted into two-dimensional structures called Mel-spectrograms to represent the frequency distribution over time. To investigate the spectral behaviour of audio signals, we employ a convolutional neural network (CNN) that perceives Mel-spectrograms in multiple dimensions. The CNN-learned feature vector serves as input to the Bi-GRU that maintains the dynamic behaviour of the audio signal. Experiments are done on six spoken languages, i.e. Ladakhi, Kashmiri, Hindi, Urdu, English, and Dogri. The data corpora used for experimentation are the International Institute of Information Technology Hyderabad-Indian Language Speech Corpus (IIITH-ILSC) and the self-created data corpus for the Ladakhi language. The model is tested on two datasets, i.e. speaker-dependent and speaker-independent. Results show that when validating the efficiency of our proposed model on both speaker-dependent and speaker-independent datasets, we achieve optimal accuracies of 99% and 91%, respectively, thus achieving promising results in comparison to the state-of-the-art models available.

Identifying languages in a novel dataset: ASMR-whispered speech.

The Autonomous Sensory Meridian Response (ASMR) is a combination of sensory phenomena involving electrostatic-like tingling sensations, which emerge in response to certain stimuli. Despite the overwhelming popularity of ASMR in the social media, no open source databases on ASMR related stimuli are yet available, which makes this phenomenon mostly inaccessible to the research community; thus, almost completely unexplored. In this regard, we present the ASMR Whispered-Speech (ASMR-WS) database. ASWR-WS is a novel database on whispered speech, specifically tailored to promote the development of ASMR-like unvoiced Language Identification (unvoiced-LID) systems. The ASMR-WS database encompasses 38 videos-for a total duration of 10 h and 36 min-and includes seven target languages (Chinese, English, French, Italian, Japanese, Korean, and Spanish). Along with the database, we present baseline results for unvoiced-LID on the ASMR-WS database. Our best results on the seven-class problem, based on segments of 2s length, and on a CNN classifier and MFCC acoustic features, achieved 85.74% of unweighted average recall and 90.83% of accuracy. For future work, we would like to focus more deeply on the duration of speech samples, as we see varied results with the combinations applied herein. To enable further research in this area, the ASMR-WS database, as well as the partitioning considered in the presented baseline, is made accessible to the research community.

Sentiment analysis based offensive language identification system for code-mixed data

Hate speech identification is the process of categorising textual information into hate and non- hate speech with the goal of identifying hate speech's targeted features. The objective of this research work is to take the Dataset from FIRE 2021 shared task code mixed data that includes YouTube comments and Twitter conversations and to detect whether the messages represents the offensive or non-offensive category. To detect the offensive language sentences, various deep learning models like Long Short-Term Memory, Bidirectional Long Short-Term Memory, Convolutional Neural Network, Gated Recurrent Unit, and hybrid model like Convolutional Neural Network with Bidirectional Long Short-Term Memory methods were utilised in this research work. The performance of all the mentioned models is evaluated using precision, recall, F1-score, and accuracy. Out of all the models, both LSTM and GRU models perform better with the accuracy of 0.81, precision of 0.85 and recall of 0.95.

Review of Automatic Language Identification System in Indian languages from the Non-Uniform Region

Abstract—The process of language identification involves au- tomatically detecting which natural language(s) are given in a speech sample. Using automatic language recognition, an audio clip can be recognised as being spoken in a specific language (LID). Identifying the language correctly from a given speech sample is the language identification system’s fundamental purpose. There are two types of systems they are explicit language identification and implicit language identification. The phoneme sequence used in the explicit language recognition system is produced from a speech sample. Language is determined using the phoneme sequence that is retrieved. There is no need to identify the phoneme in the implicit language recognition method, which determines using certain speech properties. The language identification process involves training and a classifier model. the first step is to extract the speech sample using phonemes then it is trained using classifiers. Data are randomly collected from male and female peoples in society. Index Terms—Language identification (LID), Mel-frequency cepstral coefficients (MFCC), broad phoneme classifier (BPC),Neural network, support vector machine (SVM), LDA, CNN,i-vector,GMM Super Vector(GSV),Factor Analysis(FA),Native Language Identification (NLI),Stylistically Related Text Samples (SSTs),Deep Dumb Multi Layer Perceptron (DDMLP), Deep Convolutional Neural Network (DCNN), and Semi-supervised Generative Adversarial Network(SSGAN),Gated Recurrent Units (GRUs),Hidden Markov models (HMMs),phone error rate (PER),Convolutional and Long Short Term Memory Recurrent (CLSTM), Neural Networks ,Singing Language Identification (SLID),Automatic Speech Recognition (ASR).

Improved Support Vector-Recurrent Neural Network with Optimal Feature Selection-based Spoken Language Identification System

Objective: Spoken language identification being the fore-front of language recognition tasks and most significant medium of communication has to be enhanced in order to improve the accuracy of recently developed spoken language recognition systems. The purpose of this paper is to enhance the Spoken Language Identification (SLID) model using hybrid machine learning with deep learning model for regionally spoken languages of Jammu & Kashmir (JK) and Ladakh. Method: Initially, the speech signals of different languages of JK and Ladakh are manually collected from diverse sources, and it is preprocessed using Spectral Noise Gate (SNG) filtering technique. Once the speech signals are pre-processed, the feature extraction is performed by the cepstral features like Mel-frequency Cepstral Coefficients (MFCCs), Relative Spectral Transform-Perceptual Linear Prediction (RASTA-PLP), and spectral features like spectral roll off, spectral flatness. Findings: From this feature extraction, the length of the feature vector seems to be long, and it is required to reduce the feature length. Hence, optimal feature selection is accomplished using the new meta-heuristic algorithm termed Adaptive Distance-based Tunicate Swarm Algorithm (AD-TSA) by considering the minimum correlation as objective. Finally, the language identification is handled by the hybrid classifier termed Improved Support Vector Machine-Recurrent Neural Network (ISVM-RNN). Novelty: The identification learning algorithm is enhanced by the AD-TSA by considering the minimum correlation as objective among features in order to get minimum number of features that are sufficient for language identification process. The efficiency of the proposed hybrid approach is validated by simulating the experiment on a user-defined language database of JK and Ladakh speech signals in the working platform of Python. Keywords: Language Identification; Kashmir Languages; Optimal Feature Selection; Improved Support Vector MachineRecurrent Neural Network; Adaptive DistanceBased Tunicate Swarm Algorithm

Cross-corpora spoken language identification with domain diversification and generalization

This work addresses the cross-corpora generalization issue for the low-resourced spoken language identification (LID) problem. We have conducted the experiments in the context of Indian LID and identified strikingly poor cross-corpora generalization due to corpora-dependent non-lingual biases. Our contribution to this work is twofold. First, we propose domain diversification, which diversifies the limited training data using different audio data augmentation methods. We then propose the concept of maximally diversity-aware cascaded augmentations and optimize the augmentation fold-factor for effective diversification of the training data. Second, we introduce the idea of domain generalization considering the augmentation methods as pseudo-domains. Towards this, we investigate both domain-invariant and domain-aware approaches. Our LID system is based on the state-of-the-art emphasized channel attention, propagation, and aggregation based time delay neural network (ECAPA-TDNN) architecture. We have conducted extensive experiments with three widely used corpora for Indian LID research. In addition, we conduct a final blind evaluation of our proposed methods on the Indian subset of VoxLingua107 corpus collected in the wild. Our experiments demonstrate that the proposed domain diversification is more promising over commonly used simple augmentation methods. The study also reveals that domain generalization is a more effective solution than domain diversification. We also notice that domain-aware learning performs better for same-corpora LID, whereas domain-invariant learning is more suitable for cross-corpora generalization. Compared to basic ECAPA-TDNN, its proposed domain-invariant extensions improve the cross-corpora EER up to 5.23%. In contrast, the proposed domain-aware extensions also improve performance for same-corpora test scenarios.

Spoken language identification using a genetic-based fusion approach to combine acoustic and universal phonetic results

Identification of the spoken languages in an audio file is performed automatically using the spoken language identification (LID) process. In this paper, we proposed a genetic-based fusion method to combine the score probabilities of an x-vector-based acoustic LID (ALID) and a phonetic LID (PLID) system. The ALID system is based on an LDA classifier able to identify different languages using x-vectors, while the PLID system is based on an SVM classifier which takes into account perplexities as its feature vector, which are derived from phone language models utilizing a universal phone recognizer named Allosaurus. With the help of genetic-based fusion, 54 weights will be extracted. Having 27 languages in our database and two different LID systems results in 54 weights for our fusion. The individual results of our acoustic and phonetic LID systems are eventually combined by applying these weights. Based on the experimental results on 27 languages from the NIST-LRE09 database, the fusion of the acoustic system and the phonetic system results in 93.30% accuracy, which has approximately a 21% reduction in identification error to our best baseline system with 91.50% accuracy.

An Overview of Indian Spoken Language Recognition from Machine Learning Perspective

Automatic spoken language identification (LID) is a very important research field in the era of multilingual voice-command-based human-computer interaction. A front-end LID module helps to improve the performance of many speech-based applications in the multilingual scenario. India is a populous country with diverse cultures and languages. The majority of the Indian population needs to use their respective native languages for verbal interaction with machines. Therefore, the development of efficient Indian spoken language recognition systems is useful for adapting smart technologies in every section of Indian society. The field of Indian LID has started gaining momentum since the early 2000s, mainly due to the development of several standard multilingual speech corpora for the Indian languages. Even though significant research progress has already been made in this field, to the best of our knowledge, there are not many attempts to analytically review them collectively. In this work, we have conducted one of the very first attempts to present a comprehensive review of the Indian spoken language recognition research field. In-depth analysis has been presented to emphasize the unique challenges of low-resource and mutual influences for developing LID systems in the Indian contexts. Several essential aspects of the Indian LID research, such as the detailed description of the available speech corpora, the major research contributions, including the earlier attempts based on statistical modeling to the recent approaches based on different neural network architectures, and the future research trends are discussed. This review work will help assess the state of the present Indian LID research by any active researcher or any research enthusiasts from related fields.

Language Identification-Based Evaluation of Single Channel Speech Separation of Overlapped Speeches

In multi-lingual, multi-speaker environments (e.g., international conference scenarios), speech, language, and background sounds can overlap. In real-world scenarios, source separation techniques are needed to separate target sounds. Downstream tasks, such as ASR, speaker recognition, speech recognition, VAD, etc., can be combined with speech separation tasks to gain a better understanding. Since most of the evaluation methods for monophonic separation are either single or subjective, this paper used the downstream recognition task as an overall evaluation criterion. Thus, the performance could be directly evaluated by the metrics of the downstream task. In this paper, we investigated a two-stage training scheme that combined speech separation and language identification tasks. To analyze and optimize the separation performance of single-channel overlapping speech, the separated speech was fed to a language identification engine to evaluate its accuracy. The speech separation model was a single-channel speech separation network trained with WSJ0-2mix. For the language identification system, we used an Oriental Language Dataset and a dataset synthesized by directly mixing different proportions of speech groups. The combined effect of these two models was evaluated for various overlapping speech scenarios. When the language identification network model was based on single-person single-speech frequency spectrum features, Chinese, Japanese, Korean, Indonesian, and Vietnamese had significantly improved recognition results over the mixed audio spectrum.

A comprehensive approach for performance evaluation of Indian language identification systems

Good feature representation is the chief requirement for improving Language Identification (LID) system recognition performance. In this work LID system for Indian languages is proposed based on unsupervised feature learning utilizing Deep Belief Network (DBN). The proposed methodology is implemented in two parts. The first phase of this work is based on extracting MFCC features combined with SDC hybrid features. The resultant hybrid features are further stacked to Deep Belief Network (DBN). The second phase of the proposed work is investigating the performance of various Feed forward back propagation neural network models for classification using different training algorithms. Effect of combining different activation functions and varying the hidden neurons is also investigated The performance of the resultant models is evaluated on the basis of some performance metrics such as the epochs, training time, Mean Square Error, Regression and Mean Absolute Percentage Error. Results indicate that optimal performance is achieved in model trained with Levenberg Marquardt (LM) training algorithm. The activation functions used in the hidden and output layer are “tansig” and “purelin”. Similarly, the effect of varying the number of neurons in the hidden layer is not significant in improving the performance of the derived models. FFBPNN models trained with PL and TS activation functions gave best performance indices. A user defined language database in four different languages Hindi, English, Tamil and Malayalam is used for this work.

Spoken Language Identification System Using Convolutional Recurrent Neural Network

Following recent advancements in deep learning and artificial intelligence, spoken language identification applications are playing an increasingly significant role in our day-to-day lives, especially in the domain of multi-lingual speech recognition. In this article, we propose a spoken language identification system that depends on the sequence of feature vectors. The proposed system uses a hybrid Convolutional Recurrent Neural Network (CRNN), which combines a Convolutional Neural Network (CNN) with a Recurrent Neural Network (RNN) network, for spoken language identification on seven languages, including Arabic, chosen from subsets of the Mozilla Common Voice (MCV) corpus. The proposed system exploits the advantages of both CNN and RNN architectures to construct the CRNN architecture. At the feature extraction stage, it compares the Gammatone Cepstral Coefficient (GTCC) feature and Mel Frequency Cepstral Coefficient (MFCC) feature, as well as a combination of both. Finally, the speech signals were represented as frames and used as the input for the CRNN architecture. After conducting experiments, the results of the proposed system indicate higher performance with combined GTCC and MFCC features compared to GTCC or MFCC features used individually. The average accuracy of the proposed system was 92.81% in the best experiment for spoken language identification. Furthermore, the system can learn language-specific patterns in various filter size representations of speech files.

Spoken language identification in unseen channel conditions using modified within-sample similarity loss

• A novel mWSSL for improving the channel-invariance of the LID network. • Re-implementation of the original WSSL to achieve better performance. • Extensive experimentation on the proposed approach. State-of-the-art spoken language identification (LID) systems use sophisticated training strategies to improve the robustness to unseen channel conditions in the real-world test samples. However, all these approaches require training samples from multiple channels with corresponding channel-labels, which is not available in many cases. Recent research in this regard has shown the possibility of learning a channel-invariant representation of the speech using an auxiliary loss function called within-sample similarity loss (WSSL), which does not require samples from multiple channels. Specifically, the WSSL encourages the LID network to ignore channel-specific contents in the speech by minimizing the similarities between two utterance-level embeddings of same sample. However, as WSSL approach operates at sample-level, it ignores the channel variations that may be present across different training samples within same dataset. In this work, we propose a modification to the WSSL approach to address this limitation. Specifically, along with the WSSL, the proposed modified WSSL (mWSSL) approach additionally considers the similarities with two global-level embeddings which represent the average channel-specific contents in a given mini-batch of training samples. The proposed modification allows the network to have a better view of the channel-specific contents in the training dataset , leading to improved performance in unseen channel conditions.

A Systematic Review on Language Identification of Code-Mixed Text: Techniques, Data Availability, Challenges, and Framework Development

The mix of native language with other languages (code-mixing) in social media has posed a severe challenge for language identification (LID) systems. It has encouraged research on code-mixed LID solutions. This study investigated the techniques, challenges, and dataset availability with corresponding quality criteria and developed a comprehensive framework for code-mixed LID. This study addressed four research issues to identify gaps and future work opportunities in tackling code-mixed LID challenges. Based on our analysis of reviewed studies, we outlined key points for future research in code-mixed LID. We demonstrated a taxonomy of applied techniques for code-mixed LID and highlighted the different technique variants. In code-mixed LID tasks, we discovered four significant challenges: ambiguity, lexical borrowing, non-standard words, and intra-word code-mixing. This systematic literature review recognised 32 code-mixed datasets available for LID. We proposed five features to describe the quality criteria dataset. The features are the number of instances or sentences, percentage of code-mixed types in the data, number of tokens, number of unique tokens, and average sentence length. Finally, we synthesised the methodologies and proposed a conceptual framework for subsequent studies through our literature analysis.