ASR System Research Articles

An amalgamation of integrated features with DeepSpeech2 architecture and improved spell corrector for improving Gujarati language ASR system

An amalgamation of integrated features with DeepSpeech2 architecture and improved spell corrector for improving Gujarati language ASR system

Performance Enhancement of ASR System using Noise Reduction Technique as Pre-processing step

Performance Enhancement of ASR System using Noise Reduction Technique as Pre-processing step

Development of efficient techniques for ASR System for Speech Detection and Recognization system using Gaussian Mixture Model- Universal Background Model

Some practical uses of ASR have been implemented, including the transcription of meetings and the usage of smart speakers. It is the process by which speech waves are transformed into text that allows computers to interpret and act upon human speech. Scalable strategies for developing ASR systems in languages where no voice transcriptions or pronunciation dictionaries exist are the primary focus of this work. We first show that the necessity for voice transcription into the target language can be greatly reduced through cross-lingual acoustic model transfer when phonemic pronunciation lexicons exist in the new language. Afterwards, we investigate three approaches to dealing with languages that lack a pronunciation lexicon. Secondly, we have a look at the efficiency of graphemic acoustic model transfer, which makes it easy to build pronunciation dictionaries. Thesis problems can be solved, in part, by investigating optimization strategies for training on huge corpora (such as GA+HMM and DE+HMM). In the training phase of acoustic modelling, the suggested method is applied to traditional methods. Read speech and HMI voice experiments indicated that while each data augmentation strategy alone did not always increase recognition performance, using all three techniques together did. Power normalised cepstral coefficient (PNCC) features are tweaked somewhat in this work to enhance verification accuracy. To increase speaker verification accuracy, we suggest employing multiple “Gaussian Mixture Model-Universal Background Model (GMM-UBM) and SVM classifiers”. Importantly, pitch shift data augmentation and multi-task training reduced bias by more than 18% absolute compared to the baseline system for read speech, and applying all three data augmentation techniques during fine tuning reduced bias by more than 7% for HMI speech, while increasing recognition accuracy of both native and non-native Dutch speech.

Amalgamation of noise elimination and TDNN acoustic modelling techniques for the advancements in continuous Kannada ASR system

Amalgamation of noise elimination and TDNN acoustic modelling techniques for the advancements in continuous Kannada ASR system

DEVELOPMENT OF A REAL-TIME SPEECH RECOGNITION SYSTEM FOR THE AZERBAIJANI LANGUAGE

This paper investigates the development of a real-time automatic speech recognition system dedicated to the Azerbaijani language, focusing on addressing the prevalent gap in speech recognition system for underrepresented languages. Our research integrates a hybrid acoustic modeling approach that combines Hidden Markov Model and Deep Neural Network to interpret the complexities of Azerbaijani acoustic patterns effectively. Recognizing the agglutinative nature of Azerbaijani, the ASR system employs a syllable-based n-gram model for language modeling, ensuring the system accurately captures the syntax and semantics of Azerbaijani speech. To enable real-time capabilities, we incorporate WebSocket technology, which facilitates efficient bidirectional communication between the client and server, necessary for processing streaming speech data instantly. The Kaldi and SRILM toolkits are used for the training of acoustic and language models, respectively, contributing to the system's robust performance and adaptability. We have conducted comprehensive experiments to test the effectiveness of our system, the results of which strongly corroborate the utility of the syllable-based subword modeling approach for Azerbaijani language recognition. Our proposed ASR system shows superior performance in terms of recognition accuracy and rapid response times, outperforming other systems tested on the same language data. The system's success not only proves beneficial for Azerbaijani language recognition but also provides a valuable framework for potential future applications in other agglutinative languages, thereby contributing to the promotion of linguistic diversity in automatic speech recognition technology.

An end-to-end continuous Kannada ASR system under uncontrolled environment

An end-to-end continuous Kannada ASR system under uncontrolled environment

Improvements in ASR system to access the real-time agricultural commodity prices and weather information in Kannada language/dialects

Improvements in ASR system to access the real-time agricultural commodity prices and weather information in Kannada language/dialects

The concentration of chromium and cobalt ions and parameters of oxidative stress in serum and their impact on clinical outcomes after metaphyseal hip arthroplasty with modular metal heads

PurposeCurrent epidemiological data forecast an almost 40% increase in the number of hip arthroplasty performed in the population of patients with osteoarthritis in 2060, compared to year 2018. On the basis of 10 years of observation, the failure rate after a metal-on-metal hip replacement is between 56.7 and 88.9%, depending on the used implant.MethodsSeventy-six men operated using metaphyseal hip prostheses, with modular metal heads: the J&J DePuy ASR and Biomet Recap-Magnum systems, after a period of about 5–7 years after the procedure, were assessed twice (an interval of 6 months) in terms of the parameters of oxidative stress and the concentration of chromium, cobalt and ions nickel, as well as their impact on the current clinical status and quality of life.ResultsThe mean values of the Co and Cr ion concentrations increased in a statistically significant manner at the individual stages of the study (13.20 Co and 18.16 Cr) for J&J DePuy ASR. Using the WOMAC-hip, HHS and SF-12 rating scales, the functional status of operated patients in both study groups did not change in a statistically significant manner during subsequent visits. There was a statistically significant increase in perceived pain in patients operated bilaterally with the J&J DePuy ASR system. The severity of pain could be related to the increase in the concentration of Co and Cr ions; however, it concerned a small group of bilaterally operated patients (n = 3 + n = 4).ConclusionsMetal-on-metal configuration in hip arthroplasty significantly influences with the increase in the concentration of chromium and cobalt ions in a double assessment. A statistically significant increase in the concentration of the tested Co and Cr ions in the blood correlates with an increase in the intensity of pain, especially in patients undergoing bilateral surgery. The limitation of this study is the relatively small number of bilaterally operated patients. Elevated levels of Co and Cr ions in the blood of patients operated on with the J&J DePuy ASR system increased steadily during both follow-up visits.

Enhancing Performance of End-to-End Gujarati Language ASR using combination of Integrated Feature Extraction and Improved Spell Corrector Algorithm

A number of intricate deep learning architectures for effective End-to-End (E2E) speech recognition systems have emerged due to recent advancements in algorithms and technical resources. The proposed work develops an ASR system for the publicly accessible dataset on Gujarati language. The approach provided in this research combines features like Mel frequency Cepstral Coefficients (MFCC) and Constant Q Cepstral Coefficients (CQCC) at front-end feature extraction methodologies. Enhanced spell corrector with BERT-based algorithm and Gated Recurrent Units (GRU) based DeepSpeech2 architecture are used to implement the back end portion of the proposed ASR system. The proposed study shown that combining the MFCC features and CQCC features extracted from speech with the GRU-based DeepSpeech2 model and the upgraded or enhanced spell corrector improves the Word Error Rate (WER) by 17.46% when compared to the model without post processing.

Automatic Speaker Recognition System Based on Gaussian Mixture Models, Cepstral Analysis, and Genetic Selection of Distinctive Features

This article presents the Automatic Speaker Recognition System (ASR System), which successfully resolves problems such as identification within an open set of speakers and the verification of speakers in difficult recording conditions similar to telephone transmission conditions. The article provides complete information on the architecture of the various internal processing modules of the ASR System. The speaker recognition system proposed in the article, has been compared very closely to other competing systems, achieving improved speaker identification and verification results, on known certified voice dataset. The ASR System owes this to the dual use of genetic algorithms both in the feature selection process and in the optimization of the system's internal parameters. This was also influenced by the proprietary feature generation and corresponding classification process using Gaussian mixture models. This allowed the development of a system that makes an important contribution to the current state of the art in speaker recognition systems for telephone transmission applications with known speech coding standards.

Regional Language Speech Recognition from Bone-Conducted Speech Signals through Different Deep Learning Architectures.

Bone-conducted microphone (BCM) senses vibrations from bones in the skull during speech to electrical audio signal. When transmitting speech signals, bone-conduction microphones (BCMs) capture speech signals based on the vibrations of the speaker's skull and have better noise-resistance capabilities than standard air-conduction microphones (ACMs). BCMs have a different frequency response than ACMs because they only capture the low-frequency portion of speech signals. When we replace an ACM with a BCM, we may get satisfactory noise suppression results, but the speech quality and intelligibility may suffer due to the nature of the solid vibration. Mismatched BCM and ACM characteristics can also have an impact on ASR performance, and it is impossible to recreate a new ASR system using voice data from BCMs. The speech intelligibility of a BCM-conducted speech signal is determined by the location of the bone used to acquire the signal and accurately model phonemes of words. Deep learning techniques such as neural network have traditionally been used for speech recognition. However, neural networks have a high computational cost and are unable to model phonemes in signals. In this paper, the intelligibility of BCM signal speech was evaluated for different bone locations, namely the right ramus, larynx, and right mastoid. Listener and deep learning architectures such as CapsuleNet, UNet, and S-Net were used to acquire the BCM signal for Tamil words and evaluate speech intelligibility. As validated by the listener and deep learning architectures, the Larynx bone location improves speech intelligibility.

Building a neural speech recognizer for quranic recitations

This work is an effort towards building Neural Speech Recognizers system for Quranic recitations that can be effectively used by anyone regardless of their gender and age. Despite having a lot of recitations available online, most of them are recorded by professional male adult reciters, which means that an ASR system trained on such datasets would not work for female/child reciters. We address this gap by adopting a benchmark dataset of audio records of Quranic recitations that consists of recitations by both genders from different ages. Using this dataset, we build several speaker-independent NSR systems based on the DeepSpeech model and use word error rate (WER) for evaluating them. The goal is to show how an NSR system trained and tuned on a dataset of a certain gender would perform on a test set from the other gender. Unfortunately, the number of female recitations in our dataset is rather small while the number of male recitations is much larger. In the first set of experiments, we avoid the imbalance issue between the two genders and down-sample the male part to match the female part. For this small subset of our dataset, the results are interesting with 0.968 WER when the system is trained on male recitations and tested on female recitations. The same system gives 0.406 WER when tested on male recitations. On the other hand, training the system on female recitations and testing it on male recitation gives 0.966 WER while testing it on female recitations gives 0.608 WER.

ASR - VLSP 2021: An Efficient Transformer-based Approach for Vietnamese ASR Task

Various techniques have been applied to enhance automatic speech recognition during the last few years. Reaching auspicious performance in natural language processing makes Transformer architecture becoming the de facto standard in numerous domains. This paper first presents our effort to collect a 3000-hour Vietnamese speech corpus. After that, we introduce the system used for VLSP 2021 ASR task 2, which is based on the Transformer. Our simple method achieves a favorable syllable error rate of 6.72% and gets second place on the private test. Experimental results indicate that the proposed approach dominates traditional methods with lower syllable error rates on general-domain evaluation sets. Finally, we show that applying Vietnamese word segmentation on the label does not improve the efficiency of the ASR system.

Code-switched end-to-end Marathi speech recognition for especially abled people

There is no need for eyes or arms in speech communication. Instead, both humans and machines communicate by speaking and listening. Therefore, speech communication not only helps physically disabled but also help people with hearing loss and/or language disorders. It is also used by people with vision or motor disabilities. There are various types of devices that are used for people with hearing loss. They can be classified as audio-visual stimulation devices, tactile devices, and speech processing devices. These devices consider the characteristics of speech and/or deploy automatic speech recognition inside it. This article focuses on such ASR system based on E2E model which can be useful in such devices. This research is carried out on the language used by the families of India’s over 80 million speakers i.e., Marathi. These assessments provide valuable insights into the development of SR models for low-resourced Indian languages. We present E2E ASR model trained with code-switched text of Marathi language and decoded similarly to a normal E2E ASR system. This model achieves a better word error rate of 35.09%, according to experimental results on the code-switched Marathi corpus. We compared our model to Facebook’s wav2vec2 model, which produces a WER of 23.72 percent on code-switched Marathi datasets without any language model and 12.10 percent on open ASR Marathi datasets.

Uneven success: automatic speech recognition and ethnicity-related dialects

Addressing racial bias in automatic speech recognition is an area of concern in fields associated with human-computer interaction. Research to date suggests that sociolinguistic variation, namely systematic sources of sociophonetic variation, has yet to be extensively exploited in acoustic model architectures. This paper reports a study that evaluates the performance of one ASR system for a multi-ethnic sample of speakers from the American Pacific Northwest (including Native American, African American, European American and ChicanX speakers). Using a sociophonetic approach to characterizing vocalic and consonantal variation, we ask which dialect features appear to be most challenging for our ASR system. We also ask which error types are particular to the four ethnic dialects sampled. Recordings of both conversational and read speech were coded for a common set of 18 sociophonetic variables with distinct phonetic profiles. Automatic transcription was achieved using CLOx, a custom-built ASR system created for sociolinguistic analysis. Normalized error frequency rates were compared across ethnic samples to evaluate CLOx performance. Nf error rates demonstrate clear differential performance in the ASR system, pointing to racial bias in system output. Specific predictions are made regarding approaches that might be taken to leverage sociophonetic knowledge to improve social dialect-recognition accuracy in ASR systems.

Enhancements in Continuous Kannada ASR System by Background Noise Elimination

Enhancements in Continuous Kannada ASR System by Background Noise Elimination

Spectral warping and data augmentation for low resource language ASR system under mismatched conditions

The performance of an Automatic Speech Recognition System (ASR) system deteriorates while using it on children speech, due to large variations and mismatch of acoustic and linguistic variables between spoken utterances of adults and children. Another important reason for the low efficiency of ASR models is the data scarcity of children speech data for low resource-language like Punjabi. The proposed work in this paper tries to address the both challenges i.e. acoustic and linguistic variations challenge, and data scarcity problem, thereby improves performance of a children speech ASR system for Punjabi language. To handle the first issue of acoustic and linguistic variations, the proposed work uses formant modification as a spectral warping technique to reduce the variation between children speech and adult speech. Further, to address the second issue of data scarcity, this paper discusses training of ASR models on augmented children speech data. Also, the work combines well established Mel-Frequency Cepstral Coefficients (MFCC) features extraction technique with Frequency Domain Linear Prediction (FDLP) to propose MFCC-FDLP hybrid approach for front end feature extraction. For implementing the data augmentation, Tacotron 2, an end-to-end Text to Speech (TTS) generative model has been used. The proposed work uses MFCC, FDLP and hybrid MFCC + FDLP techniques for front end feature extraction, Time Delay Neural Network (TDNN) for backend acoustic modeling, and trigram language model to implement continuous Punjabi language ASR systems. To increase robustness of the proposed ASR system, we have included a batch of lexically diverse words in our pronunciation model which achieved a relative improvement of 29.44%.

Deep learning based assistive technology on audio visual speech recognition for hearing impaired

Assistive technology would be an immense benefit for hearing impaired people by using Audio Visual Speech Recognition (AVSR). Around 466 million people worldwide suffer from hearing loss. Hearing impaired student rely on lip reading for understanding the speech. Lack of trained sign language facilitators and high cost of assistive devices are some of the major challenges faced by hearing impaired students. In this work, we have identified a visual speech recognition technique using cutting edge deep learning models. Moreover, the existing VSR techniques are erroneous. Hence to address the gaps identified, we propose a novel technique by fusion the results from audio and visual speech. This study proposes a new deep learning based audio visual speech recognition model for efficient lip reading. In this paper, an effort has been made to improve the performance of the system significantly by achieving a lowered word error rate of about 6.59% for ASR system and accuracy of about 95% using lip reading model.

Hallucination of Speech Recognition Errors With Sequence to Sequence Learning

Prior work in this domain has focused on modeling errors at the phonetic level, while using a lexicon to convert the phones to words, usually accompanied by an FST Language model. We present novel end-to-end models to directly predict hallucinated ASR word sequence outputs, conditioning on an input word sequence as well as a corresponding phoneme sequence. This improves prior published results for recall of errors from an in-domain ASR system’s transcription of unseen data, as well as an out-of-domain ASR system’s transcriptions of audio from an unrelated task, while additionally exploring an in-between scenario when limited characterization data from the test ASR system is obtainable. To verify the extrinsic validity of the method, we also use our hallucinated ASR errors to augment training for a spoken question classifier, finding that they enable robustness to real ASR errors in a downstream task, when scarce or even zero task-specific audio was available at train-time.

A formant modification method for improved ASR of children’s speech

Differences in acoustic characteristics between children’s and adults’ speech degrade performance of automatic speech recognition systems when systems trained using adults’ speech are used to recognize children’s speech. This performance degradation is due to the acoustic mismatch between training and testing. One of the main sources of the acoustic mismatch is the difference in vocal tract resonances (formant frequencies) between adult and child speakers. The present study aims to reduce the mismatch in formant frequencies by modifying formants of children’s speech to better correspond to formants of adults’ speech. This is carried out by warping the linear prediction (LP) spectrum computed from children’s speech. The warped LP spectra computed in a frame-based manner from children’s speech are used with the corresponding LP residuals to synthesize speech whose formant structure is closer to that of adults’ speech. When used in testing of an ASR system trained using adults’ speech, the warping reduces the spectral mismatch in speech between training and testing and improves the system performance in recognition of children’s speech. Experiments were conducted using narrowband (8 kHz) and wideband (16 kHz) speech of adult and child speakers from the WSJCAM0 and PF_STAR databases, respectively, and by recognizing children’s speech using acoustic models trained with adults’ speech. The proposed method gave relative improvements of 24% and 11% for the DNN and TDNN acoustic models, respectively, for narrowband speech. For wideband speech, the technique gave relative improvements of 27% and 13% for the DNN and TDNN acoustic models, respectively. The performance of the proposed method was also compared to two speaker adaptation methods: vocal tract length normalization (VTLN) and speaking rate adaptation (SRA). This comparison showed the best recognition performance for the proposed method. We also combined the proposed method with VTLN and SRA, and found that the combined method gave a further reduction in WER. Moreover, our experiments carried out for noisy speech using various types of additive noise and signal-to-noise ratios showed that the proposed method performs well also for degraded speech.