Speech Recognition Accuracy Research Articles

Gauging the accuracy of automatic speech data harvesting in five under-resourced languages

Recent research on deep-learning architectures has resulted in substantial improvements in automatic speech recognition accuracy. The leaps of progress made in well-resourced languages can be attributed to the fact that these architectures are able to effectively represent spoken language in all its diversity and complexity. However, developing advanced models of a language without appropriate corpora of speech and text data remains a challenge. For many under-resourced languages, including those spoken in South Africa, such resources simply do not exist. The aim of the work reported on in this paper is to address this situation by investigating the possibility to create diverse speech resources from unannotated broadcast data. The paper describes how existing speech and text resources were used to develop a semi-automatic data harvesting procedure for two genres of broadcast data, namely news bulletins and radio dramas. It was found that adapting acoustic models with less than 10 hours of manually annotated data from the same domain significantly reduced transcription error rates for speaking styles and acoustic conditions that are not represented in any of the existing speech corpora. Results also indicated that much more automatically transcribed adaptation data is required to achieve similar results.

Simultaneous Execution of Dereverberation, Denoising, and Speaker Separation Using a Neural Beamformer for Adapting Robots to Real Environments

It remains challenging for robots to accurately perform sound source localization and speech recognition in a real environment with reverberation, noise, and the voices of multiple speakers. Accordingly, we propose “U-TasNet-Beam,” a speech extraction method for extracting only the target speaker’s voice from all ambient sounds in a real environment. U-TasNet-Beam is a neural beamformer comprising three elements: a neural network for removing reverberation and noise, a second neural network for separating the voices of multiple speakers, and a minimum variance distortionless response (MVDR) beamformer. Experiments with simulated data and recorded data show that the proposed U-TasNet-Beam can improve the accuracy of sound source localization and speech recognition in robots compared to the conventional methods in a noisy, reverberant, and multi-speaker environment. In addition, we propose the spatial correlation matrix loss (SCM loss) as a loss function for the neural network learning the spatial information of the sound. By using the SCM loss, we can improve the speech extraction performance of the neural beamformer.

Temporary improvement of cognitive and behavioral scales for Dementia elderly by Shiritori word game with a dialogue robot: A pilot study.

Communication therapies based on conversations with caregivers, such as reminiscence therapy and music therapy, have been proposed to delay the progression of dementia. Although these therapies have been reported to improve the cognitive and behavioral functions of elderly people suffering from dementia, caregivers do not have enough time to spend on administering such communication therapies, especially in Japan where the workforce of caregivers is inadequate. Consequently, the progression of dementia in the elderly and the accompanying increased burden on caregivers has become a social problem. While the automation of communication therapy using robots and virtual agents has been proposed, the accuracy of both speech recognition and dialogue control is still insufficient to improve the cognitive and behavioral functions of the dementia elderly. In this study, we examine the effect of a Japanese word-chain game (Shiritori game) with an interactive robot and that of music listening on the maintenance and improvement of cognitive and behavioral scales [Mini-Mental State Examination (MMSE) and Dementia Behavior Disturbance scale (DBD)] of the dementia elderly. These activities can provide linguistic and phonetic stimuli, and they are simpler to implement than conventional daily conversation. The results of our Wizard-of-Oz-based experiments show that the cognitive and behavioral function scores of the elderly who periodically played the Shiritori game with an interactive robot were significantly improved over the elderly in a control group. On the other hand, no such effect was observed with the music listening stimuli. Our further experiments showed that, in the Shiritori intervention group, there was a ceiling on the increase in MMSE. The lower the MMSE before participating in the experiment, the greater the increase. Furthermore, greater improvement in DBD was observed when the participants actively played the Shiritori game. Since the Shiritori game is relatively easy to automate, our findings show the potential benefits of automating dementia therapies to maintain cognitive and behavioral functions.

Improving the performance of Uyghur speech recognition based on Factorized Time-Delay Neural Network

Speech recognition systems have low accuracy in recognizing the Uyghur language, a low-resource language, due to its strong language specificity and few public training datasets. Given this problem, considering the characteristics of Uyghur, we use morpheme units to build a language model and use mixture data augmentation methods to expand the training data. A 9-layer TDNN-F is applied, which can effectively utilize contextual information. An optimal 9.88% WER (Word Error Rate) is achieved in experiments on the open-source dataset THUYVG-20. Compared to the baseline system of this dataset, the WER is reduced by 6.7%, which significantly improves the accuracy of the Uyghur speech recognition, and provides a reference in other low-resource languages for speech recognization.

Silent Speech Recognition with Strain Sensors and Deep Learning Analysis of Directional Facial Muscle Movement.

Silent communication based on biosignals from facial muscle requires accurate detection of its directional movement and thus optimally positioning minimum numbers of sensors for higher accuracy of speech recognition with a minimal person-to-person variation. So far, previous approaches based on electromyogram or pressure sensors are ineffective in detecting the directional movement of facial muscles. Therefore, in this study, high-performance strain sensors are used for separately detecting x- and y-axis strain. Directional strain distribution data of facial muscle is obtained by applying three-dimensional digital image correlation. Deep learning analysis is utilized for identifying optimal positions of directional strain sensors. The recognition system with four directional strain sensors conformably attached to the face shows silent vowel recognition with 85.24% accuracy and even 76.95% for completely nonobserved subjects. These results show that detection of the directional strain distribution at the optimal facial points will be the key enabling technology for highly accurate silent speech recognition.

An online intelligent electronic medical record system via speech recognition

Traditional electronic medical record systems in hospitals rely on healthcare workers to manually enter patient information, resulting in healthcare workers having to spend a significant amount of time each day filling out electronic medical records. This inefficient interaction seriously affects the communication between doctors and patients and reduces the speed at which doctors can diagnose patients’ conditions. The rapid development of deep learning–based speech recognition technology promises to improve this situation. In this work, we build an online electronic medical record system based on speech interaction. The system integrates a medical linguistic knowledge base, a specialized language model, a personalized acoustic model, and a fault-tolerance mechanism. Hence, we propose and develop an advanced electronic medical record system approach with multi-accent adaptive technology for avoiding the mistakes caused by accents, and it improves the accuracy of speech recognition obviously. For testing the proposed speech recognition electronic medical record system, we construct medical speech recognition data sets using audio and electronic medical records from real medical environments. On the data sets from real clinical scenarios, our proposed algorithm significantly outperforms other machine learning algorithms. Furthermore, compared to traditional electronic medical record systems that rely on keyboard inputs, our system is much more efficient, and its accuracy rate increases with the increasing online time of the proposed system. Our results show that the proposed electronic medical record system is expected to revolutionize the traditional working approach of clinical departments, and it serves more efficient in clinics with low time consumption compared with traditional electronic medical record systems depending on keyboard inputs, which has less recording mistakes and lows down the time consumption in modification of medical recordings; due to the proposed speech recognition electronic medical record system is built on knowledge database of medical terms, so it has a good generalized application and adaption in the clinical scenarios for hospitals.

Bioinspired dual-channel speech recognition using graphene-based electromyographic and mechanical sensors

Automatic speech recognition (ASR) is helpful to improve quality of life. However, the performance of ASR degrades in the case of noisy environment, limited privacy, and speech disorders. Herein, we analyze the generation mechanism of speech and utilize dual-biological-channel for speech recognition. We also propose a dual-channel graphene-based electromyographic (EMG) and mechanical sensor (DGEMS) that can simultaneously collect two bio-signals at the same point. The EMG electrodes have better performance compared with commercial electrodes, and the mechanical sensors exhibit excellent repeatability during 10 million fatigue testing. Based on the excellent performance of EMG electrodes and mechanical sensors, 100% accuracy is achieved on digits dataset and 96.85% accuracy on a dataset containing 71 words. We also demonstrate that the DGEMS can be very resistant to noise, and the accuracy is always higher than 95%. Dual biological channels can greatly improve the performance of speech recognition in more scenarios. • A dual-channel graphene-based EMG and mechanical sensor for speech recognition • The graphene-based mechanical sensors exhibit excellent repeatability • An accuracy of 96.85% is achieved on a 71-word dataset • High speech recognition accuracy can be maintained in noisy environments In this work, Tian et al. propose a dual-channel graphene-based electromyographic and mechanical sensor for speech recognition with 100% accuracy based on digits dataset. This work highlights a promising solution to improve the performance of speech recognition in complex environments, such as noisy environment, limited privacy, and speech disorders.

Improving Hybrid CTC/Attention Architecture for Agglutinative Language Speech Recognition.

Unlike the traditional model, the end-to-end (E2E) ASR model does not require speech information such as a pronunciation dictionary, and its system is built through a single neural network and obtains performance comparable to that of traditional methods. However, the model requires massive amounts of training data. Recently, hybrid CTC/attention ASR systems have become more popular and have achieved good performance even under low-resource conditions, but they are rarely used in Central Asian languages such as Turkish and Uzbek. We extend the dataset by adding noise to the original audio and using speed perturbation. To develop the performance of an E2E agglutinative language speech recognition system, we propose a new feature extractor, MSPC, which uses different sizes of convolution kernels to extract and fuse features of different scales. The experimental results show that this structure is superior to VGGnet. In addition to this, the attention module is improved. By using the CTC objective function in training and the BERT model to initialize the language model in the decoding stage, the proposed method accelerates the convergence of the model and improves the accuracy of speech recognition. Compared with the baseline model, the character error rate (CER) and word error rate (WER) on the LibriSpeech test-other dataset increases by 2.42% and 2.96%, respectively. We apply the model structure to the Common Voice—Turkish (35 h) and Uzbek (78 h) datasets, and the WER is reduced by 7.07% and 7.08%, respectively. The results show that our method is close to the advanced E2E systems.

Small-world spiking neural network with anti-interference ability based on speech recognition under interference

The external interference can hamper the normal function of neuromorphic hardware under complex noise environment. Therefore, the study of brain-like models with anti-interference ability is a crucial issue. A bio-brain has the advantage of self-adaptability. The existing brain-like models are lack of bio-interpretability. Drawing from the advantage of bio-brain, the purpose of this paper is to investigate the anti-interference ability of brain-like models with bio-interpretability. In this study, we construct a spiking neural network with a small-world topology (SWSNN) as a brain-like model, in which Izhikevich neuron models and synaptic plasticity models with time-delay are used to represent the nodes and edges of the network, respectively. Then, the anti-interference ability of our SWSNN against different external noise is investigated, and its mechanism is explored. Further, by taking the speech recognition task as the case study, we verify the anti-interference ability of the SWSNN. Our simulation results indicate that: (i) The anti-interference ability of SWSNN is better than that of SNNs with other topologies. (ii) The discussion of neural information processing of the SWSNN under interference implies that the dynamic regulation of synaptic plasticity is an intrinsic factor of the anti-interference, and the topology is a factor that affects the anti-interference at the level of performance. (iii) Taking the speech recognition task as a case study, the SWSNN under different external noises still have almost the same high speech recognition accuracy, compared with the SWSNN without interference, and SWSNN yields better anti-interference ability than SNNs with other topologies in terms of the speech recognition accuracy. In addition, the anti-interference ability of our SWSNN framework is superior to that of existing liquid state machine (LSM) framework. Our simulation results lay a preliminary foundation for the neuromorphic hardware with robustness under complex noise environment.

A Russian Continuous Speech Recognition System Based on the DTW Algorithm under Artificial Intelligence

In order to improve the effect of continuous speech recognition, this paper combines the DTW algorithm to construct a continuous Russian speech recognition system and proposes a continuous Russian speech detection method based on VGDTW-MPCA with an unequal interval process. Moreover, considering the influence of the correlation between variables on the synchronization of the DTW algorithm, this paper constructs a DTW algorithm on a local data set to synchronize in different variable groups. Then, this paper integrates the obtained data into complete 3D data for modeling. It can be seen from the simulation research that the Russian continuous speech recognition system based on DTW proposed in this paper has a high continuous Russian speech recognition accuracy.

Improving N-Best Rescoring in Under-Resourced Code-Switched Speech Recognition Using Pretraining and Data Augmentation

In this study, we present improvements in N-best rescoring of code-switched speech achieved by n-gram augmentation as well as optimised pretraining of long short-term memory (LSTM) language models with larger corpora of out-of-domain monolingual text. Our investigation specifically considers the impact of the way in which multiple monolingual datasets are interleaved prior to being presented as input to a language model. In addition, we consider the application of large pretrained transformer-based architectures, and present the first investigation employing these models in English-Bantu code-switched speech recognition. Our experimental evaluation is performed on an under-resourced corpus of code-switched speech comprising four bilingual code-switched sub-corpora, each containing a Bantu language (isiZulu, isiXhosa, Sesotho, or Setswana) and English. We find in our experiments that, by combining n-gram augmentation with the optimised pretraining strategy, speech recognition errors are reduced for each individual bilingual pair by 3.51% absolute on average over the four corpora. Importantly, we find that even speech recognition at language boundaries improves by 1.14% even though the additional data is monolingual. Utilising the augmented n-grams for lattice generation, we then contrast these improvements with those achieved after fine-tuning pretrained transformer-based models such as distilled GPT-2 and M-BERT. We find that, even though these language models have not been trained on any of our target languages, they can improve speech recognition performance even in zero-shot settings. After fine-tuning on in-domain data, these large architectures offer further improvements, achieving a 4.45% absolute decrease in overall speech recognition errors and a 3.52% improvement over language boundaries. Finally, a combination of the optimised LSTM and fine-tuned BERT models achieves a further gain of 0.47% absolute on average for three of the four language pairs compared to M-BERT. We conclude that the careful optimisation of the pretraining strategy used for neural network language models can offer worthwhile improvements in speech recognition accuracy even at language switches, and that much larger state-of-the-art architectures such as GPT-2 and M-BERT promise even further gains.

A Flexible Graphene-Based Fabric Ultrasound Source for Machine Learning Enhanced Information Encryption

Information encryption has become an essential trend in the era of big data. Ultrasound can be used as an excellent encrypted transmission medium carrying information due to its strong concealment and small interference. And, better wearable needs cannot be ignored. Here, we propose a flexible graphene-based fabric ultrasound source (GUS) for machine learning enhanced information encryption. GUS is prepared by one-step laser method to generate laser-induced graphene on NOMEX fabric substrate. The easily captured low-frequency sound signals are modulated to high frequency to become ultrasound signals and emit through GUS. The key to success of this work is excellent sound pressure level (SPL) output of GUS in high frequency. Besides, the Mel Frequency Cestrum Coefficient (MFCC) feature of ultrasound is extracted by convolutional neural network, and the accuracy of speech recognition is 98.2 %.

When A Nonnative Accent Lets You Spot All the Errors: Examining the Syntactic Interlanguage Benefit.

In our continuously globalizing world, cross-cultural and cross-linguistic communications are far from exceptional. A wealth of research has indicated that the processing of nonnative-accented speech can be challenging for native listeners, both at the level of phonology. However, few online studies have examined the underpinnings of accented speech recognition from the perspective of the nonnative listener, even though behavioral studies indicate that accented input may be easier to process for such individuals (i.e., the interlanguage speech intelligibility benefit. The current EEG study first examined the phonological and syntactic analysis of nonnative-accented speech among nonnative listeners. As such, 30 English learners of Spanish listened to syntactically correct and incorrect Spanish sentences produced in native and nonnative-accented Spanish. The violation in the incorrect sentences was caused by errors that are typical (i.e., gender errors; *la color) or atypical (i.e., number errors; *los color) for English learners of Spanish. Results indicated that nonnative listeners elicit a phonological mismatch negativity (PMN) when attending to speech produced by a native Spanish speaker. Furthermore, the nonnative listeners showed a P600 for all grammatical violations, indicating that they repair all errors regardless of their typicality or the accent in which they are produced. Follow-up analyses compared our novel data to the data of native listeners from the methodologically identical precursor study. These analyses showed that native and nonnative listeners exhibit directionally opposite PMN effects; whereas natives exhibited a larger PMN for English-accented Spanish, nonnatives displayed a larger PMN in response to native Spanish utterances (a classic interlanguage speech intelligibility benefit). An additional difference was observed at the syntactic level: Whereas natives repaired only atypical number errors when they were English-accented, nonnative participants exhibited a P600 in response to all English-accented syntactic errors, regardless of their typicality (a syntactic interlanguage speech intelligibility benefit). Altogether, these results suggest that accented speech is not inherently difficult to process; in fact, nonnatives may benefit from the presence of a nonnative accent. Thus, our data provide some of the first electrophysiological evidence supporting the existence of the classic interlanguage speech intelligibility benefit and its novel syntactic counterpart.

Voice Quality Evaluation of Singing Art Based on 1DCNN Model

Traditional speech recognition still has the problems of poor robustness and low signal-to-noise ratio, which makes the accuracy of speech recognition not ideal. Combining the idea of one-dimensional convolutional neural network with objective evaluation, an improved CNN speech recognition method is proposed in this paper. The simulation experiment is carried out with MATLAB. The effectiveness and feasibility of this method are verified by simulation. This new method is based on one-dimensional convolutional neural network. The traditional 1DNN algorithm is optimized by using the fractional processing node theory, and the corresponding parameters are set. Establish an objective evaluation system based on improved 1DCNN. Through the comparison with other neural networks, the results show that the evaluation method based on the improved 1DCNN has high stability, and the error between subjective score and evaluation method is the smallest.

The effect of lexical tone experience on English intonation perception in Mandarin-speaking cochlear-implanted children.

To examine the effect of lexical tone experience on English intonation perception in Mandarin-speaking cochlear-implanted children during second language acquisition in Taiwan. A retrospective cohort study. A tertiary referred center. Fourteen children with cochlear implant (CI) in the experimental group, and 9 normal hearing children in the control group were enrolled in this study. Cochlear implantation and hearing rehabilitation. Two speech recognition accuracies were examined: (1) Lexical tone recognition (4-alternative forced choice, AFC), (2) English Sentence Intonation (2AFC). The overall accuracies for tone perception are 61.13% (standard deviation, SD = 10.84%) for CI group and 93.82% (SD = 1.80%) for normal hearing group. Tone 4 and Tone 1 were more easily to be recognized than tone 2 and tone 3 in the pediatric CI recipients (cCI) group. In English intonation perception, the overall accuracies are 61.82% (SD = 16.85%) for CI group, and 97.59% (SD = 4.73%) for normal hearing group. Significant high correlation (R = .919, P ≦ .000) between lexical tone perception and English intonation perception is noted. There is no significant difference for English intonation perception accuracies between Mandarin-speaking cCI (61.82%) and English-speaking cCI (70.13%, P = .11). Mandarin-speaking cochlear-implanted children showed significant deficits in perception of lexical tone and English intonation relative to normal hearing children. There was no tonal language benefit in Mandarin-speaking cochlear-implanted children's English intonation perception, compared to the English-speaking cochlear-implanted peers. For cochlear-implanted children, better lexical tone perception comes with better English intonation perception. Enhancing Mandarin prosodic perception for cochlear-implanted children may benefit their command of intonation in English.

A new speech corpus of super-elderly Japanese for acoustic modeling

The development of accessible speech recognition technology will allow the elderly to more easily access electronically stored information. However, the necessary level of recognition accuracy for elderly speech has not yet been achieved using conventional speech recognition systems, due to the unique features of the speech of elderly people. To address this problem, we have created a new speech corpus named EARS (Elderly Adults Read Speech), consisting of the recorded read speech of 123 super-elderly Japanese people (average age: 83.1), as a resource for training automated speech recognition models for the elderly. In this study, we investigated the acoustic features of super-elderly Japanese speech using our new speech corpus. In comparison to the speech of less elderly Japanese speakers, we observed a slower speech rate and extended vowel duration for both genders, a slight increase in fundamental frequency for males, and a slight decrease in fundamental frequency for females. To demonstrate the efficacy of our corpus, we also conducted speech recognition experiments using two different acoustic models (DNN-HMM and transformer-based), trained with a combination of data from our corpus and speech data from three conventional Japanese speech corpora. When using the DNN-HMM trained with EARS and speech data from existing corpora, the character error rate (CER) was reduced by 7.8% (to just over 9%), compared to a CER of 16.9% when using only the baseline training corpora. We also investigated the effect of training the models with various amounts of EARS data, using a simple data expansion method. The acoustic models were also trained for various numbers of epochs without any modifications. When using the Transformer-based end-to-end speech recognizer, the character error rate was reduced by 3.0% (to 11.4%) by using a doubled EARS corpus with the baseline data for training, compared to a CER of 13.4% when only data from the baseline training corpora were used.

A hidden Markov optimization model for processing and recognition of English speech feature signals

Abstract Speech recognition plays an important role in human–computer interaction. The higher the accuracy and efficiency of speech recognition are, the larger the improvement of human–computer interaction performance. This article briefly introduced the hidden Markov model (HMM)-based English speech recognition algorithm and combined it with a back-propagation neural network (BPNN) to further improve the recognition accuracy and reduce the recognition time of English speech. Then, the BPNN-combined HMM algorithm was simulated and compared with the HMM algorithm and the BPNN algorithm. The results showed that increasing the number of test samples increased the word error rate and recognition time of the three speech recognition algorithms, among which the word error rate and recognition time of the BPNN-combined HMM algorithm were the lowest. In conclusion, the BPNN-combined HMM can effectively recognize English speeches, which provides a valid reference for intelligent recognition of English speeches by computers.

Isolated word recognition based on PNCC with different classifiers in a noisy environment

Speech Recognition is one of the important methods to control robotics, mobile applications, automobile, etc. Nevertheless, still, the Speech recognition rate needs to be improved especially in noisy environments. The main goal of this paper is to investigate the best features for isolated word recognition between the most common features extraction technique Mel Frequency Cepstral Coefficient MFCC and noise immunity features extraction technique Power Normalized Cepstral Coefficient PNCC, with the aid of four classifiers, Weighted KNN, Medium KNN, Linear SVM, Quadratic SVM in different levels of noise. The database that is used for this study is Speech Commands Data Set v0.01 which is containing 64,727 audio files to get a good estimation of the recognition rate. The algorithm based on MFCC is recommended for Speaker dependent speech recognition a little bit better (1.1%) than the algorithm based on PNCC in a noise-free environment. Nevertheless, the proposed algorithm based on PNCC is more efficient than the algorithm based on MFCC for Speaker independent speech recognition, and overall Recognition accuracy. Moreover, it has more immunity to noise and it is more accurate than MFCC by 8.7% on average with already trained speakers and 18.528% on average with new trained speakers’ tests. Finally, the proposed algorithm based on the PNCC feature extraction with aid of Weighted KNN is more accurate than other algorithms for Speaker independent speech recognition and overall Recognition accuracy. Moreover, it has more immunity to noise.

Hybrid Algorithm for English Translation Speech Recognition Based on Deep Learning Model and Clustering

Speech recognition is the most important research direction in human-computer interaction. It is the key to the connection between human beings and machines and the expression of intelligence and automation in the information society. Taking English as the research object, using the related knowledge of speech recognition, it is based on the hidden Markov model technology of deep learning and clustering analysis algorithm and evaluated according to the cross-language English phonemic recognition system of sparse autoencoder (SA) method. By studying the speech recognition algorithm of the English translation, the influence of the speech recognition environment on the accuracy of speech recognition is confirmed. This provides a direction for humans to study speech recognition at a deeper level. Based on the language model of Transformer and the language model based on Seq2Seq, it sets different vocabularies, and the data are collected in the laboratory and outdoors, respectively, and the posttest template library is formed after collection. In the task of restoring phonetic symbols to English characters when phonemes are modeling units, the error rate is the lowest. The error rate on the test set reached 9.54%, which was 6.97 percentage points higher than that of the syllable modeling unit.

Intelligent Robot English Speech Recognition Method Based on Online Database

In order to solve the problem of low accuracy of traditional English speech recognition, an intelligent robot English speech recognition method based on online database is proposed. A speech recognition device is installed on the intelligent robot as the hardware support for running the speech recognition method. The online English speech standard database is constructed to provide reference data for speech recognition. The real-time speech information is collected, and the speech signal is preprocessed by pre-emphasis, framing, windowing and other steps. According to the principle of speech signal generation, the features of speech signal are extracted, and the results of English speech recognition are obtained by similarity calculation and matching. Compared with the traditional recognition method, the experimental results show that the recognition rate of the optimised speech recognition method is improved by 1.3%, i.e. the recognition accuracy is improved.