Performance Of Speech Recognition Systems Research Articles

Comparing human and machine speech recognition in noise with QuickSIN.

A test is proposed to characterize the performance of speech recognition systems. The QuickSIN test is used by audiologists to measure the ability of humans to recognize continuous speech in noise. This test yields the signal-to-noise ratio at which individuals can correctly recognize 50% of the keywords in low-context sentences. It is argued that a metric for automatic speech recognizers will ground the performance of automatic speech-in-noise recognizers to human abilities. Here, it is demonstrated that the performance of modern recognizers, built using millions of hours of unsupervised training data, is anywhere from normal to mildly impaired in noise compared to human participants.

Quantification of Automatic Speech Recognition System Performance on d/Deaf and Hard of Hearing Speech.

To evaluate the performance of commercial automatic speech recognition (ASR) systems on d/Deaf and hard-of-hearing (d/Dhh) speech. A corpus containing 850 audio files of d/Dhh and normal hearing (NH) speech from the University of Memphis Speech Perception Assessment Laboratory was tested on four speech-to-text application program interfaces (APIs): Amazon Web Services, Microsoft Azure, Google Chirp, and OpenAI Whisper. We quantified the Word Error Rate (WER) of API transcriptions for 24 d/Dhh and nine NH participants and performed subgroup analysis by speech intelligibility classification (SIC), hearing loss (HL) onset, and primary communication mode. Mean WER averaged across APIs was 10 times higher for the d/Dhh group (52.6%) than the NH group (5.0%). APIs performed significantly worse for "low" and "medium" SIC (85.9% and 46.6% WER, respectively) as compared to "high" SIC group (9.5% WER, comparable to NH group). APIs performed significantly worse for speakers with prelingual HL relative to postlingual HL (80.5% and 37.1% WER, respectively). APIs performed significantly worse for speakers primarily communicating with sign language (70.2% WER) relative to speakers with both oral and sign language communication (51.5%) or oral communication only (19.7%). Commercial ASR systems underperform for d/Dhh individuals, especially those with "low" and "medium" SIC, prelingual onset of HL, and sign language as primary communication mode. This contrasts with Big Tech companies' promises of accessibility, indicating the need for ASR systems ethically trained on heterogeneous d/Dhh speech data. 3 Laryngoscope, 2024.

Evaluating OpenAI's Whisper ASR: Performance analysis across diverse accents and speaker traits.

This study investigates Whisper's automatic speech recognition (ASR) system performance across diverse native and non-native English accents. Results reveal superior recognition in American compared to British and Australian English accents with similar performance in Canadian English. Overall, native English accents demonstrate higher accuracy than non-native accents. Exploring connections between speaker traits [sex, native language (L1) typology, and second language (L2) proficiency] and word error rate uncovers notable associations. Furthermore, Whisper exhibits enhanced performance in read speech over conversational speech with modifications based on speaker gender. The implications of these findings are discussed.

Speech enhancement augmentation for robust speech recognition in noisy environments

Abstract. The use of augmentations as a data enrichment method has become an important element in improving the performance of speech recognition systems. To work effectively in noisy conditions, augmentation is usually used to simulate the presence of background noise. However, the quality of speech recognition on samples pre-processed by noise reduction models does not increase. This paper proposes a new approach to speech data augmentation when training ASR systems, intended for their joint use with models for speech enhancement. It was based on the creation of several additional data samples containing speech samples processed by the enhancement model. The proposed approach was tested on the E-Branchformer neural network model using data from the Librispeech set. The quality of speech samples was assessed using the DNSMOS metric. By means of a 100-hour sample of clean speech samples it was shown that the proposed augmentation allows for an improvement in the WER metric of more than 4% in absolute value compared to the generally accepted approach based on adding noisy speech samples. Experiments on 960-hour data demonstrated the robustness of this approach as the training set size increased.

Training speech recognition models at the National Library of Sweden

The abundance of openly available audio data in English enables pretraining of automatic speech recognition (ASR) systems on hundreds of thousands to millions of hours of recorded speech. As a result speech recognition systems are approaching human level robustness in English. Other languages’ performance in multilingual speech recognition systems tend to stand in proportion to the amount of data included from the language – or the language family – in question. For low to mid resource languages with fewer speakers, the amount of openly available data may be limited, and as a consequence these languages tend to be underrepresented in large scale efforts to train multilingual speech recognition systems. The National Library of Sweden hosts large collections of audio recordings. These resources allow us to potentially bridge some of the existing speech recognition performance gaps between Swedish and higher resource languages. We believe Swedish speech recognition can be further improved upon by scaling up the amount of training data. We have constructed an inclusive and transparent speech corpus with emphasis on all variations of spoken Swedish that we will use to train speech-to-text models for Swedish.

Oral Voice Recognition System Based on Deep Neural Network Posteriori Probability Algorithm

The unstoppable trend of globalization has made the integration of voice recognition with science and technology increasingly close, and has called for the emergence of more powerful oral voice recognition systems. At present, most speech recognition research uses Hidden Markov Model-Gaussian Mixture Modell (HMM-GMM) and onnectionist Temporal Classification (CTC) end-to-end method, which has obvious limitations. Therefore, based on neural network, this paper introduces deep neural network posterior probability algorithm into speech recognition system, and constructs the spoken speech recognition system based on deep neural network posterior probability algorithm. The design principle of the constructed speech system is discussed and the performance analysis and effectiveness evaluation are carried out. The experimental results show that the performance of the neural network-based speech recognition system constructed in this paper has made a breakthrough compared with the traditional technology, and the Character Error Rate (CER) is only 4.6%, and the End-to-End Latency and Equal Error Rate (EER) have reached a low level. This study points out the future development direction of speech recognition and provides the impetus to promote its further development

Crossing language identification: Multilingual ASR framework based on semantic dataset creation & Wav2Vec 2.0

This study proposes an innovative methodology to enhance the performance of multilingual Automatic Speech Recognition (ASR) systems by capitalizing on the high semantic similarity between sentences across different languages and eliminating the requirement for Language Identification (LID). To achieve this, special bilingual datasets were created from the Mozzila Common Voices datasets in Spanish, Russian, and Portuguese. The process involves computing sentence embeddings using Language-agnostic BERT and selecting sentence pairs based on high and low cosine similarity. Subsequently, we train the Wav2vec 2.0 XLSR53 model on these datasets and assess its performance utilizing Character Error Rate (CER) and Word Error Rate (WER) metrics. The experimental results indicate that models trained on high-similarity samples consistently surpass their low-similarity counterparts, emphasizing the significance of high semantic similarity data selection for precise and dependable ASR performance. Furthermore, the elimination of LID contributes to a simplified system with reduced computational costs and the capacity for real-time text output. The findings of this research offer valuable insights for the development of more efficient and accurate multilingual ASR systems, particularly in real-time and on-device applications.

Interactive System for Gender Classification

Abstract: A study on the detection of gender, based on voice using artificial neural networks. In today’s fast-moving world Gender classification through voice plays an important role to enhance performance in Speech recognition systems, Forensic investigation, and Marketing. The dataset has 3,168 recorded voice samples of male and female voices. The samples are produced by using acoustic analysis. The primary goal of the proposed model is to automate the system to identify gender based on the audio signals and to test the voice of a human on the spot. Multilayer Perceptron (MLP) with ReLU activation function as a model has been trained to predict gender. Nadam optimizer is used for the optimization of neural networks, K-Nearest Neighbor and Support Vector Machines are trained on the dataset of 3,168 records. The obtained best accuracy is 97% on the given dataset by MLP algorithm. An Interactive web page has been built to test the voice without interruption and to predict its gender. 125 real time samples are tested out of which model could classify every record into male/female. 19 male records are incorrectly classified.

Quantifying and Improving the Performance of Speech Recognition Systems on Dysphonic Speech.

This study seeks to quantify how current speech recognition systems perform on dysphonic input and if they can be improved. Experimental machine learning methods based on a retrospective database. Single academic voice center. A database of dysphonic speech recordings was created and tested against 3 speech recognition platforms. Platform performance on dysphonic voice input was compared to platform performance on normal voice input. A custom speech recognition model was trained on voice from patients with spasmodic dysphonia or vocal cord paralysis. Custom model performance was compared to base model performance. All platforms performed well on normal voice, and 2 platforms performed significantly worse on dysphonic speech. Accuracy metrics on dysphonic speech returned values of 84.55%, 88.57%, and 93.56% for International Business Machines (IBM) Watson, Amazon Transcribe, and Microsoft Azure, respectively. The secondary analysis demonstrated that the lower performance of IBM Watson and Amazon Transcribe was driven by performance on spasmodic dysphonia and vocal fold paralysis. Thus, a custom model was built to increase the accuracy of these pathologies on the Microsoft platform. Overall, the performance of the custom model on dysphonic voices was 96.43% and on normal voices was 97.62%. Current speech recognition systems generally perform worse on dysphonic speech than on normal speech. We theorize that poor performance is a consequence of a lack of dysphonic voices in each platform's original training dataset. We address this limitation with transfer learning used to increase the performance of these systems on all dysphonic speech.

Dysarthria severity assessment using squeeze-and-excitation networks

Automated dysarthria severity identification can aid clinicians in monitoring the patient’s progress, and can improve the performance of dysarthric speech recognition systems. In this paper, we explore the potency of squeeze-and-excitation (SE) networks for dysarthria severity level classification using mel spectrograms. Deep convolutional neural network (CNN) models built using SE components and residual blocks are analysed in this regard. A comparison of the system performance is done against a shallow CNN and a convolutional recurrent neural network built using a bidirectional long short-term memory network. The models are evaluated on the Universal Access dysarthric speech corpus under speaker-dependent (SD) and speaker-independent (SID) scenarios. The proposed model obtained appreciable results in the SD scenario, and its margin of about 10% over the baselines in the SID scenario promises its applicability in automated dysarthria severity level classification.

PITCH ESTIMATION FRAMEWORK FOR SPEECH SEGREGATION USING COCHLEAGRAM MORPHING

Computational auditory scene analysis (CASA) has significant role in speech segregation from monaural audio mixtures and generally a measure for performance of speech recognition systems. Pitch estimation has a substantial role in performance of CASA systems. This study presents a novel pitch estimation framework for speech segregation from monaural audio mixtures using cochleagram morphing. The proposed framework takes the rough estimation of target pitch from given audio mixtures containing speech and background interferences. Discrete set consisting morphed versions of cochleagram is obtained using k-Means clustering. The estimated pitch values are improved by validating and smoothing them to morphed cochleagram. Measure of refined estimated pitch contours along with harmonicity and temporal continuity are used to segregate target speech. The proposed framework produced 83.13% accuracy for MIR-1k dataset which is considerably higher than the existing methods.

Accurate Recognition of Natural language Using Machine Learning and Feature Fusion Processing

To enhance the performance of Chinese language pronunciation evaluation and speech recognition systems, researchers are focusing on developing intelligent techniques for multilevel fusion processing of data, features, and decisions using deep learning-based computer-aided systems. With a combination of score level, rank level, and hybrid level fusion, as well as fusion optimization and fusion score improvement, these systems can effectively combine multiple models and sensors to improve the accuracy of information fusion. Additionally, intelligent systems for information fusion, including those used in robotics and decision-making, can benefit from techniques such as multimedia data fusion and machine learning for data fusion. Furthermore, optimization algorithms and fuzzy approaches can be applied to data fusion applications in cloud environments and e-systems, while spatial data fusion can be used to enhance the quality of image and feature data In this paper, a new approach has been presented to identify the tonal language in continuous speech. This study proposes the Machine learning-assisted automatic speech recognition framework (ML-ASRF) for Chinese character and language prediction. Our focus is on extracting highly robust features and combining various speech signal sequences of deep models. The experimental results demonstrated that the machine learning neural network recognition rate is considerably higher than that of the conventional speech recognition algorithm, which performs more accurate human-computer interaction and increases the efficiency of determining Chinese language pronunciation accuracy.

Silent Speech Recognition Based on Surface Electromyography Using a Few Electrode Sites Under the Guidance From High-Density Electrode Arrays

Although surface electromyogram recorded from high-density electrode array is believed to carry sufficient spatial information that can benefit the decoding of motor intentions, the complexity of using the array hindered its widespread applications especially in wearable devices. This study is aimed to develop a non-acoustic modality of silent speech recognition that transfers knowledge learned from high-density array to a system using a few channels, with both high portability and performance. A convolutional neural network was established for recognizing a vocabulary of 33 Chinese words during subvocal speech production. The network was trained by the data recorded from face and neck muscles using two arrays with 64 channels in the source domain. Then it was calibrated through a transfer learning approach to grant its adaption to a new target domain with the data recorded by 8 separated electrodes, while its good capability of characterizing subvocal speech word patterns is expected to be maintained. The proposed method significantly outperformed three common classification approaches and the baseline approach without transfer learning (a network trained with data just from the target domain). Under conditions of electrode shift and cross-user variability, it still obtained performance improvements. The method is demonstrated to be viable for transfer learning across domains of electrode settings and it facilitates to improve the performance of silent speech recognition systems using separate electrode sites under the guidance from high-density of arrays.

Automatic Speech Recognition for Uyghur, Kazakh, and Kyrgyz: An Overview

With the emergence of deep learning, the performance of automatic speech recognition (ASR) systems has remarkably improved. Especially for resource-rich languages such as English and Chinese, commercial usage has been made feasible in a wide range of applications. However, most languages are low-resource languages, presenting three main difficulties for the development of ASR systems: (1) the scarcity of the data; (2) the uncertainty in the writing and pronunciation; (3) the individuality of each language. Uyghur, Kazakh, and Kyrgyz as examples are all low-resource languages, involving clear geographical variation in their pronunciation, and each language possesses its own unique acoustic properties and phonological rules. On the other hand, they all belong to the Altaic language family of the Altaic branch, so they share many commonalities. This paper presents an overview of speech recognition techniques developed for Uyghur, Kazakh, and Kyrgyz, with the purposes of (1) highlighting the techniques that are specifically effective for each language and generally effective for all of them and (2) discovering the important factors in promoting the speech recognition research of low-resource languages, by a comparative study of the development path of these three neighboring languages.

Ask2Mask: Guided Data Selection for Masked Speech Modeling

Masked speech modeling (MSM) methods such as wav2vec2 or w2v-BERT learn representations over speech frames which are randomly masked within an utterance. While these methods improve performance of Automatic Speech Recognition (ASR) systems, they have one major limitation. They treat all unsupervised speech samples with equal weight, which hinders learning as not all samples have relevant information to learn meaningful representations. In this work, we address this limitation. We propose ask2mask (ATM), a novel approach to focus on specific samples during MSM pre-training. ATM employs an external ASR model or <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">scorer</i> to weight unsupervised input samples in two different ways: 1) A fine-grained data selection is performed by masking over the highly confident input frames as chosen by the scorer. This allows the model to learn meaningful representations. 2) ATM is further extended to focus at utterance-level by weighting the final MSM loss with the utterance-level confidence score. We conduct fine-tuning experiments on two well-benchmarked corpora: LibriSpeech (matching the pre-training data) and Commonvoice, TED-LIUM, AMI and CHiME-6 (not matching the pre-training data). The results substantiate the efficacy of ATM on significantly improving the recognition performance under mismatched conditions (up to 11.6% relative over published results and upto 4.46% relative over our internal baseline) while still yielding modest improvements under matched conditions.

Improving Hybrid CTC/Attention Architecture for Agglutinative Language Speech Recognition.

Unlike the traditional model, the end-to-end (E2E) ASR model does not require speech information such as a pronunciation dictionary, and its system is built through a single neural network and obtains performance comparable to that of traditional methods. However, the model requires massive amounts of training data. Recently, hybrid CTC/attention ASR systems have become more popular and have achieved good performance even under low-resource conditions, but they are rarely used in Central Asian languages such as Turkish and Uzbek. We extend the dataset by adding noise to the original audio and using speed perturbation. To develop the performance of an E2E agglutinative language speech recognition system, we propose a new feature extractor, MSPC, which uses different sizes of convolution kernels to extract and fuse features of different scales. The experimental results show that this structure is superior to VGGnet. In addition to this, the attention module is improved. By using the CTC objective function in training and the BERT model to initialize the language model in the decoding stage, the proposed method accelerates the convergence of the model and improves the accuracy of speech recognition. Compared with the baseline model, the character error rate (CER) and word error rate (WER) on the LibriSpeech test-other dataset increases by 2.42% and 2.96%, respectively. We apply the model structure to the Common Voice—Turkish (35 h) and Uzbek (78 h) datasets, and the WER is reduced by 7.07% and 7.08%, respectively. The results show that our method is close to the advanced E2E systems.

Towards improving speech recognition model with post-processing spell correction using BERT

Speech recognition has now become ubiquitous and plays an inevitable role in almost all sectors. Numerous works have been proposed on speech recognition; however, more accurate transcriptions are not possible. Exploration of various studies related to spell correction implies that several kinds of research have been carried out in this field but still it is a very challenging problem. This led to the need for a new spell corrector framework capable of leveraging the performance of the automatic speech recognition (ASR) system. The proposed work unveils state-of-the-art Bidirectional Encoder Representations from Transformers (BERT) based spell correction module developed on top of the deep recurrent neural network (RNN) based ASR system. The impact of BERT-based spell correction on the ASR system is evaluated on three different accent datasets in the perspective of word error rate (WER), character error rate (CER), and Bilingual evaluation understudy (BLEU) score. The experimental results inferred that the enhanced spell correction module is efficacious in detecting and correcting spell errors, by achieving the WER of 5.025% on librispeech corpus, 6.35% on voxforge, and 7.05% on NPTEL corpus.

Modeling and Simulation of English Speech Optimization Teaching Recognition Based on Intelligent Edge Detection Algorithm

English speech modeling is one of the key problems in the field of speech recognition. Its accuracy directly affects the performance of the English speech recognition system, and how to establish a more accurate acoustic model has always been the focus of researchers. This paper is based on the intelligent edge detection algorithm of the English speech optimization teaching recognition modeling simulation analysis to improve the accuracy of acoustic models such as parameters and the performance of continuous speech recognition system as the main purpose. In this paper, the accuracy of the neural network is improved on the premise of improving the training speed and decoding speed of the model. It proposes a new model and how to use the intelligent edge detection algorithm to build a complete English speech optimization teaching recognition system. The whole system includes the mobile terminal and the server, which realizes the most basic business logic of the speech recognition system. The experimental results of this paper show that from the point of view of the average recognition rate, the recognition effect of the optimized feature set has been further improved compared with the fusion feature. From the point of view of the recognition rate under different SNR environments, the recognition rate of the optimized feature set PCA-Features2 decreased by 0.47% compared with the FFPRLS_D + TEOCC feature set under 10dB10 words. Compared with the FFPLMS_D + TEOCC feature set, the recognition rate under 5dB10 words also drops by 0.47%.

A study of transformer-based end-to-end speech recognition system for Kazakh language

Today, the Transformer model, which allows parallelization and also has its own internal attention, has been widely used in the field of speech recognition. The great advantage of this architecture is the fast learning speed, and the lack of sequential operation, as with recurrent neural networks. In this work, Transformer models and an end-to-end model based on connectionist temporal classification were considered to build a system for automatic recognition of Kazakh speech. It is known that Kazakh is part of a number of agglutinative languages and has limited data for implementing speech recognition systems. Some studies have shown that the Transformer model improves system performance for low-resource languages. Based on our experiments, it was revealed that the joint use of Transformer and connectionist temporal classification models contributed to improving the performance of the Kazakh speech recognition system and with an integrated language model it showed the best character error rate 3.7% on a clean dataset.

Spectral warping and data augmentation for low resource language ASR system under mismatched conditions

The performance of an Automatic Speech Recognition System (ASR) system deteriorates while using it on children speech, due to large variations and mismatch of acoustic and linguistic variables between spoken utterances of adults and children. Another important reason for the low efficiency of ASR models is the data scarcity of children speech data for low resource-language like Punjabi. The proposed work in this paper tries to address the both challenges i.e. acoustic and linguistic variations challenge, and data scarcity problem, thereby improves performance of a children speech ASR system for Punjabi language. To handle the first issue of acoustic and linguistic variations, the proposed work uses formant modification as a spectral warping technique to reduce the variation between children speech and adult speech. Further, to address the second issue of data scarcity, this paper discusses training of ASR models on augmented children speech data. Also, the work combines well established Mel-Frequency Cepstral Coefficients (MFCC) features extraction technique with Frequency Domain Linear Prediction (FDLP) to propose MFCC-FDLP hybrid approach for front end feature extraction. For implementing the data augmentation, Tacotron 2, an end-to-end Text to Speech (TTS) generative model has been used. The proposed work uses MFCC, FDLP and hybrid MFCC + FDLP techniques for front end feature extraction, Time Delay Neural Network (TDNN) for backend acoustic modeling, and trigram language model to implement continuous Punjabi language ASR systems. To increase robustness of the proposed ASR system, we have included a batch of lexically diverse words in our pronunciation model which achieved a relative improvement of 29.44%.