Speech Recognition In Noisy Environments Research Articles

An Approach Towards to Real Time AI Desktop Voice Assistant

The advent of artificial intelligence (AI) has revolutionized human-computer interaction, making it more natural and intuitive. This research paper presents the development and implementation of an AI Desktop Voice Assistant designed to enhance productivity and accessibility for users. The voice assistant leverages advanced speech recognition, natural language processing (NLP), and machine learning techniques to understand and execute user commands. Key functionalities include voice-activated application control, web searches, personalized reminders, and real-time information retrieval. Our system integrates with widely-used APIs and services, providing a seamless user experience across various tasks. The paper discusses the architecture of the voice assistant, detailing the integration of components such as the speech recognition engine, NLP models, and the dialogue management system. We explore the challenges encountered during development, including accurate speech recognition in noisy environments and handling ambiguous user commands. Solutions implemented to address these challenges are also presented. Furthermore, we conduct a usability study to evaluate the effectiveness and user satisfaction of the voice assistant. The results indicate a high level of user engagement and satisfaction, demonstrating the practical benefits of incorporating AI-driven voice interfaces into desktop environments. Our findings contribute to the ongoing research in human-computer interaction, suggesting pathways for future enhancements in AI voice assistants..

Enhancing Cochlear Implant Outcomes across Age Groups: The Interplay of Forward Focus and Advanced Combination Encoder Coding Strategies in Noisy Conditions.

Background: Hearing in noise is challenging for cochlear implant users and requires significant listening effort. This study investigated the influence of ForwardFocus and number of maxima of the Advanced Combination Encoder (ACE) strategy, as well as age, on speech recognition threshold and listening effort in noise. Methods: A total of 33 cochlear implant recipients were included (age ≤ 40 years: n = 15, >40 years: n = 18). The Oldenburg Sentence Test was used to measure 50% speech recognition thresholds (SRT50) in fluctuating and stationary noise. Speech was presented frontally, while three frontal or rear noise sources were used, and the number of ACE maxima varied between 8 and 12. Results: ForwardFocus significantly improved the SRT50 when noise was presented from the back, independent of subject age. The use of 12 maxima further improved the SRT50 when ForwardFocus was activated and when noise and speech were presented frontally. Listening effort was significantly worse in the older age group compared to the younger age group and was reduced by ForwardFocus but not by increasing the number of ACE maxima. Conclusion: Forward Focus can improve speech recognition in noisy environments and reduce listening effort, especially in older cochlear implant users.

Speech enhancement augmentation for robust speech recognition in noisy environments

Abstract. The use of augmentations as a data enrichment method has become an important element in improving the performance of speech recognition systems. To work effectively in noisy conditions, augmentation is usually used to simulate the presence of background noise. However, the quality of speech recognition on samples pre-processed by noise reduction models does not increase. This paper proposes a new approach to speech data augmentation when training ASR systems, intended for their joint use with models for speech enhancement. It was based on the creation of several additional data samples containing speech samples processed by the enhancement model. The proposed approach was tested on the E-Branchformer neural network model using data from the Librispeech set. The quality of speech samples was assessed using the DNSMOS metric. By means of a 100-hour sample of clean speech samples it was shown that the proposed augmentation allows for an improvement in the WER metric of more than 4% in absolute value compared to the generally accepted approach based on adding noisy speech samples. Experiments on 960-hour data demonstrated the robustness of this approach as the training set size increased.

Enhancing Computer Digital Signal Processing through the Utilization of RNN Sequence Algorithms

With the increase in computing power and the availability of large amounts of data, deep learning techniques, especially convolutional neural networks (CNNS) and recurrent neural networks (RNNS), have become important tools for processing complex signals. These methods show excellent performance in speech recognition, image processing, natural language processing and so on. In this paper, we explore the application of recurrent neural network (RNN) sequence algorithms in the field of computer digital signal processing, highlighting current artificial intelligence techniques and their capabilities in solving complex signal processing problems. First, the paper reviews the basic principles and development of deep learning and RNN sequence algorithms, highlighting the advances these advanced technologies have made in simulating the way the human brain processes information. The practical application and effect of RNN sequence algorithm in computer digital signal processing are demonstrated through experimental data. By comparing with traditional algorithms, we demonstrate the efficiency and accuracy of RNN in processing complex signals, such as speech recognition in noisy environments and real-time video data processing. The experimental data not only demonstrate the effectiveness of RNNS in this field, but also highlight the unique advantages of deep learning methods when dealing with large and high-dimensional data. Through these empirical studies, this paper aims to provide researchers and engineers with an in-depth understanding of the potential of RNNS in digital signal processing applications, and looks forward to the future development direction of artificial intelligence technology in this field.

Research on the application of machine learning models in speech recognition in noisy environments

In a world increasingly reliant on verbal communication with machines, this paper undertakes a rigorous exploration of the application of machine learning models in speech recognition within noisy environments. An important focus of this research lies in demonstrating the viability and robustness of these models when challenged by real-world noise interference. In particular, the utilization of deep learning models such as Convolutional Neural Networks (CNNs) and Long Short-Term Memory Networks (LSTMs) is thoroughly examined. Using concrete data and case studies, this paper uncovers a promising trajectory in the development and implementation of noise-robust speech recognition systems. In so doing, it lays a foundation for future research to enhance the practicality and effectiveness of machine learning models in this area. The ultimate goal is to ensure more reliable and accurate speech recognition that can withstand various noise conditions, thus expanding the potential for speech recognition applications in diverse and adverse environments.

An automatic lip reading for short sentences using deep learning nets

One study whose importance has significantly grown in recent years is lip-reading, particularly with the widespread of using deep learning techniques. Lip reading is essential for speech recognition in noisy environments or for those with hearing impairments. It refers to recognizing spoken sentences using visual information acquired from lip movements. Also, the lip area, especially for males, suffers from several problems, such as the mouth area containing the mustache and beard, which may cover the lip area. This paper proposes an automatic lip-reading system to recognize and classify short English sentences spoken by speakers using deep learning networks. The input video extracts frames and each frame is passed to the Viola-Jones to detect the face area. Then 68 landmarks of the facial area are determined, and the landmarks from 48 to 68 represent the lip area extracted based on building a binary mask. Then, the contrast is enhanced to improve the quality of the lip image by applying contrast adjustment. Finally, sentences are classified using two deep learning models, the first is AlexNet, and the second is VGG-16 Net. The database consists of 39 participants (32 males and 7 females). Each participant repeats the short sentences five times. The outcomes demonstrate the accuracy rate of AlexNet is 90.00%, whereas the accuracy rate for VGG-16 Net is 82.34%. We concluded that AlexNet performs better for classifying short sentences than VGG-16 Net.

Improving Speech Recognition Performance in Noisy Environments by Enhancing Lip Reading Accuracy.

The current accuracy of speech recognition can reach over 97% on different datasets, but in noisy environments, it is greatly reduced. Improving speech recognition performance in noisy environments is a challenging task. Due to the fact that visual information is not affected by noise, researchers often use lip information to help to improve speech recognition performance. This is where the performance of lip recognition and the effect of cross-modal fusion are particularly important. In this paper, we try to improve the accuracy of speech recognition in noisy environments by improving the lip reading performance and the cross-modal fusion effect. First, due to the same lip possibly containing multiple meanings, we constructed a one-to-many mapping relationship model between lips and speech allowing for the lip reading model to consider which articulations are represented from the input lip movements. Audio representations are also preserved by modeling the inter-relationships between paired audiovisual representations. At the inference stage, the preserved audio representations could be extracted from memory by the learned inter-relationships using only video input. Second, a joint cross-fusion model using the attention mechanism could effectively exploit complementary intermodal relationships, and the model calculates cross-attention weights on the basis of the correlations between joint feature representations and individual modalities. Lastly, our proposed model achieved a 4.0% reduction in WER in a -15 dB SNR environment compared to the baseline method, and a 10.1% reduction in WER compared to speech recognition. The experimental results show that our method could achieve a significant improvement over speech recognition models in different noise environments.

Performance on stochastic figure-ground perception varies with individual differences in speech-in-noise recognition and working memory capacity.

Speech recognition in noisy environments can be challenging and requires listeners to accurately segregate a target speaker from irrelevant background noise. Stochastic figure-ground (SFG) tasks in which temporally coherent inharmonic pure-tones must be identified from a background have been used to probe the non-linguistic auditory stream segregation processes important for speech-in-noise processing. However, little is known about the relationship between performance on SFG tasks and speech-in-noise tasks nor the individual differences that may modulate such relationships. In this study, 37 younger normal-hearing adults performed an SFG task with target figure chords consisting of four, six, eight, or ten temporally coherent tones amongst a background of randomly varying tones. Stimuli were designed to be spectrally and temporally flat. An increased number of temporally coherent tones resulted in higher accuracy and faster reaction times (RTs). For ten target tones, faster RTs were associated with better scores on the Quick Speech-in-Noise task. Individual differences in working memory capacity and self-reported musicianship further modulated these relationships. Overall, results demonstrate that the SFG task could serve as an assessment of auditory stream segregation accuracy and RT that is sensitive to individual differences in cognitive and auditory abilities, even among younger normal-hearing adults.

Investigation of correlation between dichotic listening performance and speech in noise perception with hearing aid outcomes in the elderly

Background and purpose: Presbycusis, which occurs both in peripheral and central forms, is one of the most common disorders in the elderly. Presbycusis is generally characterized by decreased auditory sensitivity, poor speech comprehension and decreased speech recognition in noisy environments. This disease is also associated with a decrease of more than 20% of a person’s maximum speech perception in the presence of a competitive signal compared to silence. The inability of the central auditory systems of elderly hearing aid users to decode sounds, especially in the presence of peripheral hearing loss, can affect hearing aid outcomes. This study investigates the correlation between assessments related to auditory processing and hearing aid outcomes. Method: This cross-sectional study was performed on 39 elderly hearing aid users in two groups: behind-the-ear (BTE; n = 23) and in-the-ear (n = 16) hearing aid users. After pure tone audiometry, patients who met the inclusion criteria underwent assessments related to the central auditory system, including double dichotic digit (DDT) and quick noise speech (Q-SIN) tests. Then, the correlation between the tests and hearing aid outcomes was evaluated based on the international outcome inventory (IOI-HA), a comprehensive assessment of the outcomes in different aspects. Results: In the BTE and in-the-canal (ITC) hearing aid users, there was a significant correlation between DDT scores in the right ear (r = 0.60, 0.52) left ear (r = 0.43, 0.54), speech-to-noise ratio (SNR) loss (r = −0.50, −0.58) and IOI-HA questionnaire scores respectively. The stepwise regression analysis results indicated that unlike other variables such as age or peripheral hearing loss, the Q-SIN was the only variable that significantly (p < 0.001) participated in IOI-HA prediction with a correlation coefficient of 0.504, which simulates the conditions most closely related to hearing aid inefficiency and patient dissatisfaction. Conclusions: The Q-SIN test could be used partially to predict the outcomes of hearing aid use.

Multisensory benefits for speech recognition in noisy environments.

A series of our previous studies explored the use of an abstract visual representation of the amplitude envelope cues from target sentences to benefit speech perception in complex listening environments. The purpose of this study was to expand this auditory-visual speech perception to the tactile domain. Twenty adults participated in speech recognition measurements in four different sensory modalities (AO, auditory-only; AV, auditory-visual; AT, auditory-tactile; AVT, auditory-visual-tactile). The target sentences were fixed at 65 dB sound pressure level and embedded within a simultaneous speech-shaped noise masker of varying degrees of signal-to-noise ratios (-7, -5, -3, -1, and 1 dB SNR). The amplitudes of both abstract visual and vibrotactile stimuli were temporally synchronized with the target speech envelope for comparison. Average results showed that adding temporally-synchronized multimodal cues to the auditory signal did provide significant improvements in word recognition performance across all three multimodal stimulus conditions (AV, AT, and AVT), especially at the lower SNR levels of -7, -5, and -3 dB for both male (8-20% improvement) and female (5-25% improvement) talkers. The greatest improvement in word recognition performance (15-19% improvement for males and 14-25% improvement for females) was observed when both visual and tactile cues were integrated (AVT). Another interesting finding in this study is that temporally synchronized abstract visual and vibrotactile stimuli additively stack in their influence on speech recognition performance. Our findings suggest that a multisensory integration process in speech perception requires salient temporal cues to enhance speech recognition ability in noisy environments.

Converging Telco-Grade Solutions 5G and beyond to Support Production in Industry 4.0

The Industry 4.0 initiative has been showing the way for industrial production to optimize operations based on collecting, processing, and sharing data. There are new requirements on the production floor: flexible but ultra-reliable, low latency wireless communications through interoperable systems can share data. Further challenges of data sharing and storage arise when diverse systems come into play at the Manufacturing Operations Management and Business Planning &amp; Logistics levels. The emerging complex cyber-physical systems of systems need to be engineered with care. Regarding industrial requirements, the telecommunication industry has many similarities to production—including ultra-reliability, high complexity, and having humans “in-the-loop”. The current paper aims to provide an overview of converging telco-grade solutions that can be successfully applied in the wide sense of industrial production. These toolsets range from model-driven engineering through system interoperability frameworks, 5G- and 6G-supported manufacturing, and the telco-cloud to speech recognition in noisy environments.

Read my lips: Artificial intelligence word-level arabic lipreading system

Lipreading is the ability to recognize words or sentences from the mouth movements of a speaking person. This process is also known as Visual Speech Recognition (VSR). Lipreading has two main advantages: facilitate communication for people with hearing or speaking problems and aid speech recognition in noisy environments. In this paper, we propose a lipreading computing system capable of recognizing ten common Arabic words by performing word extraction from the mouth movements. The system receives a video of a person uttering an Arabic word as an input and outputs the text of the predicted word. During the implementation stage of the proposed system, three deep learning and neural network architectures are alternatively used to train, validate, and test the system using a locally collected and preprocessed dataset. The dataset contains 1051 videos and will be made available upon request. Moreover, a voting model that combines the three architectures is proposed. The highest testing accuracy (i.e. 82.84%) is achieved by leveraging the voting model.

Brain Frequency-Specific Changes in the Spontaneous Neural Activity Are Associated With Cognitive Impairment in Patients With Presbycusis.

Presbycusis (PC) is characterized by preferential hearing loss at high frequencies and difficulty in speech recognition in noisy environments. Previous studies have linked PC to cognitive impairment, accelerated cognitive decline and incident Alzheimer’s disease. However, the neural mechanisms of cognitive impairment in patients with PC remain unclear. Although resting-state functional magnetic resonance imaging (rs-fMRI) studies have explored low-frequency oscillation (LFO) connectivity or amplitude of PC-related neural activity, it remains unclear whether the abnormalities occur within all frequency bands or within specific frequency bands. Fifty-one PC patients and fifty-one well-matched normal hearing controls participated in this study. The LFO amplitudes were investigated using the amplitude of low-frequency fluctuation (ALFF) at different frequency bands (slow-4 and slow-5). PC patients showed abnormal LFO amplitudes in the Heschl’s gyrus, dorsolateral prefrontal cortex (dlPFC), frontal eye field and key nodes of the speech network exclusively in slow-4, which suggested that abnormal spontaneous neural activity in PC was frequency dependent. Our findings also revealed that stronger functional connectivity between the dlPFC and the posterodorsal stream of auditory processing, as well as lower functional coupling between the PCC and key nodes of the DMN, which were associated with cognitive impairments in PC patients. Our study might underlie the cross-modal plasticity and higher-order cognitive participation of the auditory cortex after partial hearing deprivation. Our findings indicate that frequency-specific analysis of ALFF could provide valuable insights into functional alterations in the auditory cortex and non-auditory regions involved in cognitive impairment associated with PC.

Modulation of the Primary Auditory Thalamus When Recognizing Speech with Background Noise.

Recognizing speech in background noise is a strenuous daily activity, yet most humans can master it. An explanation of how the human brain deals with such sensory uncertainty during speech recognition is to-date missing. Previous work has shown that recognition of speech without background noise involves modulation of the auditory thalamus (medial geniculate body; MGB): there are higher responses in left MGB for speech recognition tasks that require tracking of fast-varying stimulus properties in contrast to relatively constant stimulus properties (e.g., speaker identity tasks) despite the same stimulus input. Here, we tested the hypotheses that (1) this task-dependent modulation for speech recognition increases in parallel with the sensory uncertainty in the speech signal, i.e., the amount of background noise; and that (2) this increase is present in the ventral MGB, which corresponds to the primary sensory part of the auditory thalamus. In accordance with our hypothesis, we show, by using ultra-high-resolution functional magnetic resonance imaging (fMRI) in male and female human participants, that the task-dependent modulation of the left ventral MGB (vMGB) for speech is particularly strong when recognizing speech in noisy listening conditions in contrast to situations where the speech signal is clear. The results imply that speech in noise recognition is supported by modifications at the level of the subcortical sensory pathway providing driving input to the auditory cortex.SIGNIFICANCE STATEMENT Speech recognition in noisy environments is a challenging everyday task. One reason why humans can master this task is the recruitment of additional cognitive resources as reflected in recruitment of non-language cerebral cortex areas. Here, we show that also modulation in the primary sensory pathway is specifically involved in speech in noise recognition. We found that the left primary sensory thalamus (ventral medial geniculate body; vMGB) is more involved when recognizing speech signals as opposed to a control task (speaker identity recognition) when heard in background noise versus when the noise was absent. This finding implies that the brain optimizes sensory processing in subcortical sensory pathway structures in a task-specific manner to deal with speech recognition in noisy environments.

Comparison of Speech Recognition in Cochlear Implant Users with Different Speech Processors.

Speech recognition in noisy environments is a challenge for both cochlear implant (CI) users and device manufacturers. CI manufacturers have been investing in technological innovations for processors and researching strategies to improve signal processing and signal design for better aesthetic acceptance and everyday use. This study aimed to compare speech recognition in CI users using off-the-ear (OTE) and behind-the-ear (BTE) processors. A cross-sectional study was conducted with 51 CI recipients, all users of the BTE Nucleus 5 (CP810) sound processor. Speech perception performances were compared in quiet and noisy conditions using the BTE sound processor Nucleus 5 (N5) and OTE sound processor Kanso. Each participant was tested with the Brazilian-Portuguese version of the hearing in noise test using each sound processor in a randomized order. Three test conditions were analyzed with both sound processors: (i) speech level fixed at 65 decibel sound pressure level in a quiet, (ii) speech and noise at fixed levels, and (iii) adaptive speech levels with a fixed noise level. To determine the relative performance of OTE with respect to BTE, paired comparison analyses were performed. The paired t-tests showed no significant difference between the N5 and Kanso in quiet conditions. In all noise conditions, the performance of the OTE (Kanso) sound processor was superior to that of the BTE (N5), regardless of the order in which they were used. With the speech and noise at fixed levels, a significant mean 8.1 percentage point difference was seen between Kanso (78.10%) and N5 (70.7%) in the sentence scores. CI users had a lower signal-to-noise ratio and a higher percentage of sentence recognition with the OTE processor than with the BTE processor.

A comparative study for Arabic speech recognition system in noisy environments

Speech recognition in noisy environments is one of the long-standing research themes but remains a very important challenge nowadays. Therefore, there is much research into all techniques and approaches to improve the performance of speech recognition systems, even in poor conditions. This paper presents a comparative study under various conditions based on two architectures (GMM-HMM and DNN-HMM), the Hybrid GMM-HMM models using the CMU Sphinx tools and the Hybrid DNN-HMM using the KALDI toolkit in noise environment. In this study, we compare the Hybrid GMM-HMM models and the Hybrid DNN-HMM models to evaluate the performance of the proposed system. The novelty of this paper is to test if the presented tools could be, with good accuracy, recognize the Arabic speech principally in noisy environment. In addition, we adopted the noisy training theory in this paper based on GMM-HMM and DNN-HMM model. We use the public Arabic Speech Corpus for Isolated Words (20 words), three noise levels, and three noise types. The implementation of our system consists of two phases: Features extraction using Mel-frequency Cepstral Coefficient (MFCC) and the classification phase will use separately the previous two models. In order to test the performance of these methods a simulation will presented for different SNR and for different district type of noise.

What is the Right Treatment for Adults With Unilateral Deafness?

What is the Right Treatment for Adults With Unilateral Deafness?

Strategies of hearing aids fitting for improving speech recognition in noisy environments

To assess speech recognition in users of mono- and binaural hearing aids (HA) with dynamic environment control system (DECS). 25 HA users with symmetrical bilateral moderate to severe sensorineural hearing loss and 10 individuals with normal hearing underwent speech audiometry in a free sound field by listening to an adaptive Russian matrix sentence test with a background noise. The testing was conducted for hearing impaired patients with one and two HA. The Speech Recognition Thresholds in noise (SRTN) using HA with DESC and using HA without DESC were compared. The dichotic digits test and the rapidly alternating speech perception test were performed to evaluate the central auditory system function. The SRTN in normal hearing listeners was -16.4±1.9 dB SNR. In the case of monaural using HA without DESC the SRTN was -2.5±4.4 dB SNR, using HA with DESC it was -5.8±4.0 dB SNR; in the case of using binaural HAs it was -6.3±3.8 dB SNR and -9.9±3.1 dB SNR accordingly. HAs with Dynamic Environment Control System are highly effective for speech perception in noisy environments. Binaural HA use is more effective than monaural HA use regardless of HA model. Speech recognition in noisy environments in cases of binaural HA use correlates significantly with results of tests assessing the central auditory system function.

DNN-Based Mask Estimation for Distributed Speech Enhancement in Spatially Unconstrained Microphone Arrays

Deep neural network (DNN)-based speech enhancement algorithms in microphone arrays have now proven to be efficient solutions to speech understanding and speech recognition in noisy environments. However, in the context of ad-hoc microphone arrays, many challenges remain and raise the need for distributed processing. In this paper, we propose to extend a previously introduced distributed DNN-based time-frequency mask estimation scheme that can efficiently use spatial information in form of so-called compressed signals which are pre-filtered target estimations. We study the performance of this algorithm named Tango under realistic acoustic conditions and investigate practical aspects of its optimal application. We show that the nodes in the microphone array cooperate by taking profit of their spatial coverage in the room. We also propose to use the compressed signals not only to convey the target estimation but also the noise estimation in order to exploit the acoustic diversity recorded throughout the microphone array.

Hindi speech recognition in noisy environment using hybrid technique

Automatic speech recognition is generally analyzed for two types of word utterances; isolated and continuous-words speech. Continuous-words speech is almost natural way of speaking but is difficult to be recognized through machines (speech recognizers). It is also highly sensitive to environmental variations. There are various parameters which are directly affecting the performance of automatic speech recognition like size of datasets/corpus, type of data sets (isolated, spontaneous or continuous) and environment variations (noisy/clean). The performance of speech recognizers is generally good in clean environments for isolated words, but it becomes typical in noisy environments especially for continuous words/sentences and is still a challenge. In this paper, a hybrid feature extraction technique is proposed by joining core blocks of PLP (perceptual linear predictive) and Mel frequency cepstral coefficients (MFCC) that can be utilized to improve the performance of speech recognizers under such circumstances. Voice activity and detection (VAD)-based frame dropping formula has been used solely within the training part of ASR (automatic speech recognition) procedure obviating its need in actual implementations. The motivation to use this formula is for removal of pauses and distorted elements of speech improving the phonemes modeling further. The proposed method shows average improvement in performance by 12.88% for standard datasets.