Speech Recognition Accuracy Research Articles

MPSA-Conformer-CTC/Attention: A High-Accuracy, Low-Complexity End-to-End Approach for Tibetan Speech Recognition.

This study addresses the challenges of low accuracy and high computational demands in Tibetan speech recognition by investigating the application of end-to-end networks. We propose a decoding strategy that integrates Connectionist Temporal Classification (CTC) and Attention mechanisms, capitalizing on the benefits of automatic alignment and attention weight extraction. The Conformer architecture is utilized as the encoder, leading to the development of the Conformer-CTC/Attention model. This model first extracts global features from the speech signal using the Conformer, followed by joint decoding of these features through CTC and Attention mechanisms. To mitigate convergence issues during training, particularly with longer input feature sequences, we introduce a Probabilistic Sparse Attention mechanism within the joint CTC/Attention framework. Additionally, we implement a maximum entropy optimization algorithm for CTC, effectively addressing challenges such as increased path counts, spike distributions, and local optima during training. We designate the proposed method as the MaxEnt-Optimized Probabilistic Sparse Attention Conformer-CTC/Attention Model (MPSA-Conformer-CTC/Attention). Experimental results indicate that our improved model achieves a word error rate reduction of 10.68% and 9.57% on self-constructed and open-source Tibetan datasets, respectively, compared to the baseline model. Furthermore, the enhanced model not only reduces memory consumption and training time but also improves generalization capability and accuracy.

Open source platform for Estonian speech transcription

AbstractThis paper presents our progress in developing and maintaining a public speech and speaker recognition platform for the Estonian language. The platform consists of a speech processing pipeline and a web-based user interface for end-users, offering transcript post-editing functionality. It is offered for free as a public service and is in active use. The service provides significantly higher speech recognition accuracy than commercial alternatives. We discuss the switch to a workflow management system and how it has improved the core speech processing pipeline. The core systems behind the platform have been made available as open-source code and deployed internally by multiple public and private institutions.

Effective Acoustic Model-Based Beamforming Training for Static and Dynamic Hri Applications.

Human-robot collaboration will play an important role in the fourth industrial revolution in applications related to hostile environments, mining, industry, forestry, education, natural disaster and defense. Effective collaboration requires robots to understand human intentions and tasks, which involves advanced user profiling. Voice-based communication, rich in complex information, is key to this. Beamforming, a technology that enhances speech signals, can help robots extract semantic, emotional, or health-related information from speech. This paper describes the implementation of a system that provides substantially improved signal-to-noise ratio (SNR) and speech recognition accuracy to a moving robotic platform for use in human-robot interaction (HRI) applications in static and dynamic contexts. This study focuses on training deep learning-based beamformers using acoustic model-based multi-style training with measured room impulse responses (RIRs). The results show that this approach outperforms training with simulated RIRs or matched measured RIRs, especially in dynamic conditions involving robot motion. The findings suggest that training with a broad range of measured RIRs is sufficient for effective HRI in various environments, making additional data recording or augmentation unnecessary. This research demonstrates that deep learning-based beamforming can significantly improve HRI performance, particularly in challenging acoustic environments, surpassing traditional beamforming methods.

Interference of mid-level sound statistics underlie human speech recognition sensitivity in natural noise.

Recognizing speech in noise, such as in a busy restaurant, is an essential cognitive skill where the task difficulty varies across environments and noise levels. Although there is growing evidence that the auditory system relies on statistical representations for perceiving 1-5 and coding4,6-9 natural sounds, it's less clear how statistical cues and neural representations contribute to segregating speech in natural auditory scenes. We demonstrate that human listeners rely on mid-level statistics to segregate and recognize speech in environmental noise. Using natural backgrounds and variants with perturbed spectro-temporal statistics, we show that speech recognition accuracy at a fixed noise level varies extensively across natural backgrounds (0% to 100%). Furthermore, for each background the unique interference created by summary statistics can mask or unmask speech, thus hindering or improving speech recognition. To identify the neural coding strategy and statistical cues that influence accuracy, we developed generalized perceptual regression, a framework that links summary statistics from a neural model to word recognition accuracy. Whereas a peripheral cochlear model accounts for only 60% of perceptual variance, summary statistics from a mid-level auditory midbrain model accurately predicts single trial sensory judgments, accounting for more than 90% of the perceptual variance. Furthermore, perceptual weights from the regression framework identify which statistics and tuned neural filters are influential and how they impact recognition. Thus, perception of speech in natural backgrounds relies on a mid-level auditory representation involving interference of multiple summary statistics that impact recognition beneficially or detrimentally across natural background sounds.

Voice recognition by deep transfer learning and vision transformers to secure voice authentication

Speech recognition is crucial for ensuring the security of personal devices and financial transactions. Attaining high accuracy and robustness in voice authentication is challenging due to the presence of voice and environmental variability. Recent advancements in the field of deep learning, particularly in transfer learning and visual transformers, have the potential to enhance voice recognition systems. This study employs advanced deep transfer learning techniques, including Vision Transformers (ViT), VGG16, and a customized Convolutional Neural Network (CNN), to enhance the accuracy and security of speech authentication. The objective is to evaluate and contrast various solutions' voice recognition and authentication accuracy. The experiment included 3000 voice samples, with an equal distribution of 1500 samples from male participants and 1500 from female participants. The dataset was used to train Vision Transformers, VGG16 with transfer learning, and a custom CNN. The models were assessed based on their accuracy in identifying and authenticating voice samples. The VGG16 model achieved the highest level of accuracy in speech recognition, with a precision rate of 95%. The Vision Transformer and custom CNN exhibited satisfactory performance. However, VGG16 demonstrated higher accuracy. The most accurate voice authentication model studied is the VGG16 model based on transfer learning. This study suggests that the security and reliability of voice recognition systems can be enhanced through the use of deep learning techniques.

Comparative study on the accuracy of speech recognition using a contact microphone attached to the surface of the head and neck

The accuracy of speech recognition through an air-conducted microphone can be less accurate under a highly noisy environment or when the volume of the user’s voice is relatively low. One solution to this problem is the use of contact microphones. However, neither the microphone locations that provide optimal speech recognition accuracy for each user nor the mechanisms underlying these contact forces have been clarified. In this study, we experimentally investigated the effects of placement, contact force, user gender, and speech recognition platform on the accuracy of speech recognition with contact microphones placed on the surface of the head and neck. The experimental results indicated that the mechanism underlying the influence of each factor on speech recognition accuracy differs for speech acquired at the neck and head locations. In particular, the effect of the user’s gender was significant for the neck-acquired sound, but not the head-acquired sound. The results also revealed that the microphone contact force did not affect the recognition accuracy or user discomfort for the head-acquired sound. Moreover, the results of speech recognition experiments in a simulated noisy environment showed that bone-conducted sounds acquired on the head and neck surfaces were more robust than air-conducted sounds.

Designing a task-based conversational agent for EFL in German schools: Student needs, actions, and perceptions

As part of an iterative evaluation process incorporating student stakeholders within a task-based language teaching (TBLT) framework, we examine the design and early learner data of a conversational agent for English as a foreign language (EFL) in the German school context. Students (around age 12 and at an A2 proficiency level) from three 7th grade classes interacted with five tasks and completed a questionnaire assessing their needs and perceptions. Results of the needs analysis revealed that most students speak some English outside of class and that they wish for tasks targeting both authentic and curricular needs. The interaction logs showed a strong preference for typing over speaking and moderate automatic speech recognition (ASR) accuracy for the spoken interactions. Differing rates of task success and interest between the five tasks confirmed the importance of task characteristics. Despite some noted limitations, student perceptions were largely positive, with the tasks appearing more fun and more challenging for students who speak less English outside of class. Overall, our findings highlight the promise and challenges of designing a task-based conversational agent for the school EFL classroom, offering insights into balancing pedagogical and technological design, the demands of official curricula, and students’ authentic needs.

Analysis of virtual standardized patients for assessing clinical fundamental skills of medical students: a prospective study

BackgroundHistory-taking is an essential clinical competency for qualified doctors. The limitations of the standardized patient (SP) in taking history can be addressed by the virtual standardized patient (VSP). This paper investigates the accuracy of virtual standardized patient simulators and evaluates the applicability of the improved system’s accuracy for diagnostic teaching support and performance assessment.MethodsData from the application of VSP to medical residents and students were gathered for this prospective study. In a human–machine collaboration mode, students completed exams involving taking SP histories while VSP provided real-time scoring. Every participant had VSP and SP scores. Lastly, using the voice and text records as a guide, the technicians will adjust the system’s intention recognition accuracy and speech recognition accuracy.ResultsThe research revealed significant differences in scoring across several iterations of VSP and SP (p < 0.001). Across various clinical cases, there were differences in application accuracy for different versions of VSP (p < 0.001). Among training groups, the diarrhea case showed significant differences in speech recognition accuracy (Z = -2.719, p = 0.007) and intent recognition accuracy (Z = -2.406, p = 0.016). Scoring and intent recognition accuracy improved significantly after system upgrades.ConclusionVSP has a comprehensive and detailed scoring system and demonstrates good scoring accuracy, which can be a valuable tool for history-taking training.

Speech recognition in videos using features of different enhancement filters

Video Speech Recognition (VSR) is the ability of software to extract spoken text from input sources, such as offline videos and YouTube. It achieves a good recognition accuracy rate for videos with clean environments. However, with unwanted noisy elements in speech, this presents a major challenge. Depending on where the noise originates from, there are considerable differences in the impact of various noise kinds on video speech recognition, including background music, multiple speakers, microphone quality, and dialects. It is not possible to create a filter for every kind of noise because the sources of noise might differ greatly and the speech environment’s circumstances can change over time. Hence, instead of using a single filter for each type of noise, this study proposed to combine features from a small number of noise removal filters applied to the same video speech signal, resulting in a better VSR output. According to experimental results, using the proposed framework increases the accuracy of video speech recognition when evaluated on YouTube videos with different noise levels. In test experiments, the proposed framework was evaluated against state-of-the-art approaches. The accuracy mean of the relevant current approaches was improved by 16.67% and against the best accuracy of them by 7.78%. This research contributes to the video speech recognition topic as the proposed framework can facilitate the extraction of spoken phrases in millions of hours of video available online, indexing them, and then searching for them through their index.

Refining maritime Automatic Speech Recognition by leveraging synthetic speech

Maritime transport serves as a critical component of global trade and logistics, enabling the movement of goods and resources across oceans and waterways. Especially in busy waterways and ports, effective and accurate communication is essential, as it ensures the seamless exchange of information and the coordinated execution of port activities. However, comprehensibility is often hindered by factors such as poor audio quality, background noise, and diverse languages and accents. Automatic Speech Recognition (ASR) systems can mitigate these issues by providing real-time transcription and enabling the implementation of automated, value-adding services to enhance situational awareness. While pre-trained ASR models excel on general speech, maritime ASR faces unique challenges due to a lack of annotated data, diverse accents, and specialized terminology.To this end, we focus on improving the transcription quality of pre-trained ASR models for maritime communication with a particular focus on accurately recognizing maritime-specific terminology such as vessel and location names. Due to the scarcity of transcribed maritime communication, we create a synthetic training dataset tailored to regional maritime terminology. The synthetic audio is augmented with general human speech and used to fine-tune an end-to-end ASR model under various settings. The evaluation of the models employs a proprietary dataset of regional maritime radio communication from the port of Hamburg.The experimental results demonstrate a notable enhancement in ASR performance. Specifically, our approach yields an absolute improvement over the pre-trained baseline of 13.46% Word-Error-Rate and an increase of 41.57% recall for vessel names and 38.65% recall for locations. Our findings underscore the efficacy of integrating synthetic training data to address the challenges encountered in maritime ASR, paving the way for more robust and accurate speech recognition systems tailored to maritime applications.

Integrating international Chinese visualization teaching and vocational skills training: leveraging attention-connectionist temporal classification models.

The teaching of Chinese as a second language has become increasingly crucial for promoting cross-cultural exchange and mutual learning worldwide. However, traditional approaches to international Chinese language teaching have limitations that hinder their effectiveness, such as outdated teaching materials, lack of qualified instructors, and limited access to learning facilities. To overcome these challenges, it is imperative to develop intelligent and visually engaging methods for teaching international Chinese language learners. In this article, we propose leveraging speech recognition technology within artificial intelligence to create an oral assistance platform that provides visualized pinyin-formatted feedback to learners. Additionally, this system can identify accent errors and provide vocational skills training to improve learners' communication abilities. To achieve this, we propose the Attention-Connectionist Temporal Classification (CTC) model, which utilizes a specific temporal convolutional neural network to capture the location information necessary for accurate speech recognition. Our experimental results demonstrate that this model outperforms similar approaches, with significant reductions in error rates for both validation and test sets, compared with the original Attention model, Claim, Evidence, Reasoning (CER) is reduced by 0.67%. Overall, our proposed approach has significant potential for enhancing the efficiency and effectiveness of vocational skills training for international Chinese language learners.

Deep learning approach for automatic speech recognition in the presence of noise

Accurate recognition of speech in noisy environment is still an obstacle for wider application of speech recognition technology. The robustness of a speech recognition system is heavily influenced by the ability to handle the presence of background noise. In this research work, we propose a model based on Deep Fourier Neural Network (DFNN) for Automatic Speech Recognition (ASR) using LibriSpeech dataset. Most of the existing speech recognition techniques lack the robustness of handling background noise, as a result these techniques are not applicable in real-time. In order to mitigate the challenges of background noise, this research work proposes an efficient recognition technique which analyses in detail the raw audio waveforms using the Deep Fourier Neural Network (DFNN). This novel deep learning approach has a concise architecture and is an efficient approach for automatic speech recognition. The proposed deep learning approach embeds the Fourier transform, which is one of the most popular feature representations transform for audio signal processing. The Fourier transform extracts the core information from waveforms in the form of short term spectra of the speech signal as a function of time. The extracted short term spectra are analyzed deeply in the proposed DFNN model for accurate speech recognition in the presence of noise.

Implementation of Sound Direction Detection and Mixed Source Separation in Embedded Systems.

In recent years, embedded system technologies and products for sensor networks and wearable devices used for monitoring people's activities and health have become the focus of the global IT industry. In order to enhance the speech recognition capabilities of wearable devices, this article discusses the implementation of audio positioning and enhancement in embedded systems using embedded algorithms for direction detection and mixed source separation. The two algorithms are implemented using different embedded systems: direction detection developed using TI TMS320C6713 DSK and mixed source separation developed using Raspberry Pi 2. For mixed source separation, in the first experiment, the average signal-to-interference ratio (SIR) at 1 m and 2 m distances was 16.72 and 15.76, respectively. In the second experiment, when evaluated using speech recognition, the algorithm improved speech recognition accuracy to 95%.

Automatic speech recognition for Indonesian medical dictation in cloud environment

&lt;span lang="EN-US"&gt;This paper introduces SPWPM, an automatic speech recognition (ASR) system designed specifically for Indonesian medical dictation. The main objective of SPWPM is to assist medical professionals in producing medical reports and diagnosing patients. Deployed within a cloud computing service architecture, SPWPM strives to achieve a minimum speech recognition accuracy of 95%. The ASR model of SPWPM is developed using Kaldi and PyChain technologies—creating a comprehensive training dataset involving collaboration with PT Dua-Empat-Tujuh and Harapan Kita Hospital. Several optimization techniques were applied, including language modeling with smoothing, lexicon generation using the Grapheme-to-Phoneme Converter, and data augmentation. The readiness of this technology to assist hospital users was assessed through two evaluations: the SPWPM architecture test and the SPWPM speech recognition test. The results demonstrate the system's preparedness in accurately transcribing medical dictation, showcasing its potential to enhance medical reporting for healthcare professionals in hospital environments.&lt;/span&gt;

An Approach Towards to Real Time AI Desktop Voice Assistant

The advent of artificial intelligence (AI) has revolutionized human-computer interaction, making it more natural and intuitive. This research paper presents the development and implementation of an AI Desktop Voice Assistant designed to enhance productivity and accessibility for users. The voice assistant leverages advanced speech recognition, natural language processing (NLP), and machine learning techniques to understand and execute user commands. Key functionalities include voice-activated application control, web searches, personalized reminders, and real-time information retrieval. Our system integrates with widely-used APIs and services, providing a seamless user experience across various tasks. The paper discusses the architecture of the voice assistant, detailing the integration of components such as the speech recognition engine, NLP models, and the dialogue management system. We explore the challenges encountered during development, including accurate speech recognition in noisy environments and handling ambiguous user commands. Solutions implemented to address these challenges are also presented. Furthermore, we conduct a usability study to evaluate the effectiveness and user satisfaction of the voice assistant. The results indicate a high level of user engagement and satisfaction, demonstrating the practical benefits of incorporating AI-driven voice interfaces into desktop environments. Our findings contribute to the ongoing research in human-computer interaction, suggesting pathways for future enhancements in AI voice assistants..

Strategy for developing a speech recognition model specialized for patients with depression or Parkinson's disease with small size speech database.

Most of speech recognition models currently in use have been dealt with speech of normal people. The speech recognition rate for patients with depression or Parkinson's disease (PD) who show differences in speech characteristics compared to normal subjects is lower than that of normal subjects. This study explores the model to enhance accuracy of speech recognition for individuals who have depression or PD, aiming to provide them more accurate service. In this study, considering the speech features of patients with depression or PD, we designed a model with the assumption that understanding the overall meaning and context of speech through the utilization of global information, rather than local information, is more effective in enhancing recognition accuracy. We propose the m-Globalformer, a model based on the Globalformer architecture that combines the squeeze-and-excitation (SE) module with the Transformer. The m-Globalformer enhances the utilization of global information by modifying the base SE module. The model employs pre-training and fine-tuning strategies, considering the limited speech data of the patients. In the initial training phase, a large-scale normal speech dataset was used, followed by fine-tuning the model with a small-scale dataset of depression or PD patients. The m-Globalformer demonstrated superior performance in our experiments, achieved character error rates (CER) of 11.28% for depression and 19.67% for PD.

A Study on Speech Recognition by a Neural Network Based on English Speech Feature Parameters

In this study, from the perspective of English speech feature parameters, two feature parameters, the mel-frequency cepstral coefficient (MFCC) and filter bank (Fbank), were selected to identify English speech. The algorithms used for recognition employed the classical back-propagation neural network (BPNN), recurrent neural network (RNN), and long short-term memory (LSTM) that were obtained by improving RNN. The three recognition algorithms were compared in the experiments, and the effects of the two feature parameters on the performance of the recognition algorithms were also compared. The LSTM model had the best identification performance among the three neural networks under different experimental environments; the neural network model using the MFCC feature parameter outperformed the neural network using the Fbank feature parameter; the LSTM model had the highest correct rate and the highest speed, while the RNN model ranked second, and the BPNN model ranked worst. The results confirm that the application of the LSTM model in combination with MFCC feature parameter extraction to English speech recognition can achieve higher speech recognition accuracy compared to other neural networks.

Sla-former: conformer using shifted linear attention for audio-visual speech recognition

Conformer-based models have proven highly effective in Audio-visual Speech Recognition, integrating auditory and visual inputs to significantly enhance speech recognition accuracy. However, the widely utilized softmax attention mechanism within conformer models encounters scalability issues, with its spatial and temporal complexity escalating quadratically with sequence length. To address these challenges, this paper introduces the Shifted Linear Attention Conformer, an evolved iteration of the conformer architecture. Shifted Linear Attention Conformer adopts shifted linear attention as a scalable alternative to softmax attention. We conducted a thorough analysis of the factors constraining the efficiency of linear attention. To mitigate these issues, we propose the utilization of a straightforward yet potent mapping function and an efficient rank restoration module, enhancing the effectiveness of self-attention while maintaining low computational complexity. Furthermore, we integrate an advanced attention-shifting technique facilitating token manipulation within attentional mechanisms, thereby enhancing information flow across various groups. This three-part approach enhances cognitive computations, particularly beneficial for processing longer sequences. Our model achieves exceptional Word Error Rates of 1.9% and 1.5% on the Lip Reading Sentences 2 and Lip Reading Sentences 3 datasets, respectively, showcasing its state-of-the-art performance in audio-visual speech recognition tasks.

Wavoice: An mmWave-Assisted Noise-Resistant Speech Recognition System

As automatic speech recognition evolves, deployment of the voice user interface (VUI) has boomingly expanded. Especially since the COVID-19 pandemic, the VUI has gained more attention in online communication owing to its non-contact property. However, the VUI struggles to be applied in public scenes due to the degradation of received audio signals caused by various ambient noises. In this article, we propose Wavoice , the first noise-resistant multi-modal speech recognition system that fuses two distinct voices sensing modalities (i.e., millimeter-wave signals and audio signals from a microphone) together. One key contribution is to model the inherent correlation between millimeter-wave and audio signals. Based on it, Wavoice facilitates the real-time noise-resistant voice activity detection and user targeting from multiple speakers. Additionally, we elaborate on two novel modules for multi-modal fusion embedded into the neural network, leading to accurate speech recognition. Extensive experiments prove the effectiveness of Wavoice under adverse conditions—that is, the character recognition error rate below 1% in a range of 7 m. In terms of robustness and accuracy, Wavoice considerably outperforms existing audio-only speech recognition methods with lower character error and word error rates.

Audio–visual speech recognition based on regulated transformer and spatio–temporal fusion strategy for driver assistive systems

This article presents a research methodology for audio–visual speech recognition (AVSR) in driver assistive systems. These systems necessitate ongoing interaction with drivers while driving through voice control for safety reasons. The article introduces a novel audio–visual speech command recognition transformer (AVCRFormer) specifically designed for robust AVSR. We propose (i) a multimodal fusion strategy based on spatio–temporal fusion of audio and video feature matrices, (ii) a regulated transformer based on iterative model refinement module with multiple encoders, (iii) a classifier ensemble strategy based on multiple decoders. The spatio–temporal fusion strategy preserves contextual information of both modalities and achieves their synchronization. An iterative model refinement module can bridge the gap between acoustic and visual data by leveraging their impact on speech recognition accuracy. The proposed multi-prediction strategy demonstrates superior performance compared to traditional single-prediction strategy, showcasing the model’s adaptability across diverse audio–visual contexts. The transformer proposed has achieved the highest values of speech command recognition accuracy, reaching 98.87% and 98.81% on the RUSAVIC and LRW corpora, respectively. This research has significant implications for advancing human–computer interaction. The capabilities of AVCRFormer extend beyond AVSR, making it a valuable contribution to the intersection of audio–visual processing and artificial intelligence.