Speech Enhancement Algorithm Research Articles

Enhanced Speech Processing for Air Traffic Control Using an Optimized Generative Adversarial Network

In order to improve the quality of voice calls during air traffic control, an improved SEGAN air traffic control speech enhancement algorithm is proposed. Aiming at the problem that the traditional speech enhancement algorithm based on generative adversarial network (SEGAN) is drowned under low signal-to-noise ratio conditions, a multi-stage, multi-mapping, multi-dimensional output generator and a multi-scale, multi-discriminator network model are proposed based on the SEGAN network model. First, the speech semantic features are extracted based on the deep neural network structure to complete the semantic segmentation of air traffic control speech. Secondly, multiple generators are set to further optimize the speech signal. Then, a downsampling module is added to the convolution layer to improve the model's utilization of speech information and reduce the loss of speech information. Finally, multi-scale, multiple discriminators are used to learn the distribution law and information of speech samples in multiple directions. The results show that under low signal-to-noise ratio conditions, the improved SEGAN model improves the short-term objective intelligibility (STOI) and the perceptual evaluation of speech quality (PESQ) by and respectively , which can quickly and effectively perform air traffic control speech enhancement and provide preparation for subsequent air traffic control speech recognition.

In-Car Speech Enhancement Based on Source Separation Technique

Purpose: The purpose of this study was to investigate and analyze to increase the quality and intelligibility of speech in cars. The passenger dialogue inside the car, the sound of other equipment, and a wide range of interference effects are major challenges in the task of speech separation in-car environment.Methods: Speech enhancement based on the source separation algorithm has been proposed to enhance the preferred speech signals using a microphone array inside a car. The proposed approach determines the signal direction in the time domain by utilizing the time difference of arrival (TDOA). TDOA signals are processed, and an adaptive least mean square method is used to determine the enhanced preferred signal.Results: Experimental results show that the proposed approach yields an signal-to-noise ratio (SNR) of 7.4, perceptual evaluation of speech quality (PESQ) of 2.33 respectively. The proposed strategy outperforms existing methods in terms of PESQ, and SNR. The PESQ of the proposed method is 2.45%, and 5.22% better than the existing Independent Component Analyses, and Run length Source techniques.Conclusion: Finally, compared to the existing methods, the suggested speech enhancement algorithm is more reliable and flexible, and it can properly identify the exact location of the sound.

Research on Speech Enhancement Algorithm by Fusing Improved EMD and GCRN Networks

Research on Speech Enhancement Algorithm by Fusing Improved EMD and GCRN Networks

Speech Enhancement Using Machine Learning

The incorporation of machine learning, specifically deep learning, into speech enhancement algorithms represents an advanced methodology aimed at restoring original speech signals from distorted counterparts. This innovative approach incorporates the use of Charlier polynomials-based discrete transform, particularly the discrete Charlier transform (DCHT), to extract spectra from noisy signals employing a fully connected neural network. Leveraging the capabilities of deep learning, particularly in handling nonlinear mapping challenges, the system acquires contextual information from speech signals, resulting in enhanced speech characterized by improved quality and intelligibility. The proposed algorithm undergoes rigorous empirical testing through self-comparison, fine-tuning the DCHT parameter to optimize the performance of speech enhancement models. The experimentation entails the variation of DCHT parameter values, with evaluation conducted using the TIMIT database. Diverse speech measures are employed for comprehensive assessment, revealing the effectiveness of the DCHT-based trained model in enhancing speech signals within specific conditions.

Objective intelligibility measurement of reverberant vocoded speech for normal-hearing listeners: Towards facilitating the development of speech enhancement algorithms for cochlear implants.

Cochlear implant (CI) recipients often struggle to understand speech in reverberant environments. Speech enhancement algorithms could restore speech perception for CI listeners by removing reverberant artifacts from the CI stimulation pattern. Listening studies, either with cochlear-implant recipients or normal-hearing (NH) listeners using a CI acoustic model, provide a benchmark for speech intelligibility improvements conferred by the enhancement algorithm but are costly and time consuming. To reduce the associated costs during algorithm development, speech intelligibility could be estimated offline using objective intelligibility measures. Previous evaluations of objective measures that considered CIs primarily assessed the combined impact of noise and reverberation and employed highly accurate enhancement algorithms. To facilitate the development of enhancement algorithms, we evaluate twelve objective measures in reverberant-only conditions characterized by a gradual reduction of reverberant artifacts, simulating the performance of an enhancement algorithm during development. Measures are validated against the performance of NH listeners using a CI acoustic model. To enhance compatibility with reverberant CI-processed signals, measure performance was assessed after modifying the reference signal and spectral filterbank. Measures leveraging the speech-to-reverberant ratio, cepstral distance and, after modifying the reference or filterbank, envelope correlation are strong predictors of intelligibility for reverberant CI-processed speech.

CNN-based noise reduction for multi-channel speech enhancement system with discrete wavelet transform (DWT) preprocessing.

Speech enhancement algorithms are applied in multiple levels of enhancement to improve the quality of speech signals under noisy environments known as multi-channel speech enhancement (MCSE) systems. Numerous existing algorithms are used to filter noise in speech enhancement systems, which are typically employed as a pre-processor to reduce noise and improve speech quality. They may, however, be limited in performing well under low signal-to-noise ratio (SNR) situations. The speech devices are exposed to all kinds of environmental noises which may go up to a high-level frequency of noises. The objective of this research is to conduct a noise reduction experiment for a multi-channel speech enhancement (MCSE) system in stationary and non-stationary environmental noisy situations with varying speech signal SNR levels. The experiments examined the performance of the existing and the proposed MCSE systems for environmental noises in filtering low to high SNRs environmental noises (-10 dB to 20 dB). The experiments were conducted using the AURORA and LibriSpeech datasets, which consist of different types of environmental noises. The existing MCSE (BAV-MCSE) makes use of beamforming, adaptive noise reduction and voice activity detection algorithms (BAV) to filter the noises from speech signals. The proposed MCSE (DWT-CNN-MCSE) system was developed based on discrete wavelet transform (DWT) preprocessing and convolution neural network (CNN) for denoising the input noisy speech signals to improve the performance accuracy. The performance of the existing BAV-MCSE and the proposed DWT-CNN-MCSE were measured using spectrogram analysis and word recognition rate (WRR). It was identified that the existing BAV-MCSE reported the highest WRR at 93.77% for a high SNR (at 20 dB) and 5.64% on average for a low SNR (at -10 dB) for different noises. The proposed DWT-CNN-MCSE system has proven to perform well at a low SNR with WRR of 70.55% and the highest improvement (64.91% WRR) at -10 dB SNR.

Frequency-domain distributed multichannel wiener filtering speech enhancement algorithm

A frequency-domain distributed microphone multi-channel Wiener filter speech enhancement algorithm is proposed in this paper. In this paper, the distributed microphone speech model is considered. First, the speech signal in the time domain is converted into the speech signal in the frequency domain by the discrete Fourier transform method. Then, the unconstrained minimization problem of the noise reduction and speech distortion of the complex linear filter in the frequency domain is established. Simulation results show that the proposed algorithm is superior to some existing multi-channel speech enhancement algorithms.

Single-channel speech enhancement algorithm based on ME-MGCRN in low signal-to-noise scenario

Single-channel speech enhancement algorithm based on ME-MGCRN in low signal-to-noise scenario

SASEGAN-TCN: Speech enhancement algorithm based on self-attention generative adversarial network and temporal convolutional network.

Traditional unsupervised speech enhancement models often have problems such as non-aggregation of input feature information, which will introduce additional noise during training, thereby reducing the quality of the speech signal. In order to solve the above problems, this paper analyzed the impact of problems such as non-aggregation of input speech feature information on its performance. Moreover, this article introduced a temporal convolutional neural network and proposed a SASEGAN-TCN speech enhancement model, which captured local features information and aggregated global feature information to improve model effect and training stability. The simulation experiment results showed that the model can achieve 2.1636 and 92.78% in perceptual evaluation of speech quality (PESQ) score and short-time objective intelligibility (STOI) on the Valentini dataset, and can accordingly reach 1.8077 and 83.54% on the THCHS30 dataset. In addition, this article used the enhanced speech data for the acoustic model to verify the recognition accuracy. The speech recognition error rate was reduced by 17.4%, which was a significant improvement compared to the baseline model experimental results.

Mitigating Information Interruptions by COVID-19 Face Masks: A Three-Stage Speech Enhancement Scheme

The coronavirus disease 2019 (COVID-19) preventive measures have resulted in significant lifestyle changes. One of the COVID-19 new normal is the usage of face masks for protection against airborne aerosol which creates distractions and interruptions in voice communication. It has a different influence on speech than the standard concept of noise affecting speech communication. Furthermore, it has varied effects on speech in different frequency bands. To provide a solution to this problem, a three-stage adaptive speech enhancement (SE) scheme is developed in this article. In the first stage, the tunable <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$</tex-math> </inline-formula> -factor wavelet transform (TQWT) features are extracted by properly setting the quality factor values and the number of levels from the input speech signal. In the second stage, the adjustable parameters of the preemphasis filter and modified multiband spectral subtraction (MBSS) are determined using bio-inspired techniques for different masking and signal-to-noise ratio (SNR) conditions. In the third stage, the weights, center values, standard deviation of the Gaussian radial basis functions, and input patterns of the radial basis function neural networks (RBFNNs) are updated to predict the optimized parameters from the input TQWT-based cepstral features (TQCFs). In the end, the performance of the proposed algorithm is compared with the standard SE algorithms using two speech datasets.

New modified Bat algorithm for blind speech enhancement in time domain

We address the speech enhancement problem for dual convolutif mixed channel by viewing it in a blind separation sources setting. One widely used technique to separate mixed signals is to apply adaptive filtering, the challenge is to identify an unknown finite impulse response. traditionally we apply a gradient-based algorithm to adapt filter coefficients. However, such algorithm often suffers from premature convergence ,when using large filters and non-stationary inputs , leading to the so-called local minimum problem , which affects the quality of enhanced signals significatively .one alternative to overcome this problem is to apply a population-based metaheuristic algorithms in which filter coefficients are adapted iteratively by minimizing a cost function .But even with this metaheuristic based solution, local minimum problem at large filters still exists. In order to avoid local minima and improve the chance to reach the global solution, we propose in this paper, a novel algorithm called a modified bat algorithm to render the search process efficiently by enhancing its capability of exploration and exploitation. Several experiments under different noise types are carried out using our proposed modified bat algorithm in comparison with some of the popular state-of-the-art algorithms. The enhanced signals obtained by each algorithm at the separation process outputs, show good behavior and superiority of our proposed algorithm. In terms of system misalignment, as well as a segmental signal-to-noise ratio.

Performance of single-channel speech enhancement algorithms on Mandarin listeners with different immersion conditions in New Zealand English

Speech enhancement (SE) is a widely used technology to improve the quality and intelligibility of noisy speech. So far, SE algorithms were designed and evaluated on native listeners only, but not on non-native listeners who are known to be more disadvantaged when listening in noisy environments. This paper investigates the performance of five widely used single-channel SE algorithms on early-immersed New Zealand English (NZE) listeners and native Mandarin listeners with different immersion conditions in NZE under negative input signal-to-noise ratio (SNR) by conducting a subjective listening test in NZE sentences. The performance of the SE algorithms in terms of speech intelligibility in the three participant groups was investigated. The result showed that the early-immersed group always achieved the highest intelligibility. The late-immersed group outperformed the non-immersed group for higher input SNR conditions, possibly due to the increasing familiarity with the NZE accent, whereas this advantage disappeared at the lowest tested input SNR conditions. The SE algorithms tested in this study failed to improve and rather degraded the speech intelligibility, indicating that these SE algorithms may not be able to reduce the perception gap between early-, late- and non-immersed listeners, nor able to improve the speech intelligibility under negative input SNR in general. These findings have implications for the future development of SE algorithms tailored to Mandarin listeners, and for understanding the impact of language immersion on speech perception in noise.

Optical Microphone-Based Speech Reconstruction System With Deep Learning for Individuals With Hearing Loss.

Although many speech enhancement (SE) algorithms have been proposed to promote speech perception in hearing-impaired patients, the conventional SE approaches that perform well under quiet and/or stationary noises fail under nonstationary noises and/or when the speaker is at a considerable distance. Therefore, the objective of this study is to overcome the limitations of the conventional speech enhancement approaches. This study proposes a speaker-closed deep learning-based SE method together with an optical microphone to acquire and enhance the speech of a target speaker. The objective evaluation scores achieved by the proposed method outperformed the baseline methods by a margin of 0.21-0.27 and 0.34-0.64 in speech quality (HASQI) and speech comprehension/intelligibility (HASPI), respectively, for seven typical hearing loss types. The results suggest that the proposed method can enhance speech perception by cutting off noise from speech signals and mitigating interference caused by distance. The results of this study show a potential way that can help improve the listening experience in enhancing speech quality and speech comprehension/intelligibility for hearing-impaired people.

Performance of speech enhancement algorithms on the speech intelligibility of native Mandarin listeners immersed in English-speaking environment

Speech enhancement algorithms have been developed to improve speech intelligibility for listeners under noisy conditions. However, all existing algorithms were evaluated only by native listeners, the performance of such algorithms on non-native listeners was rarely investigated. This study conducts a subjective listening test on native New Zealand English listeners and native Mandarin listeners who have been immersed in New Zealand English-speaking environment for more than one year. The participants were asked to transcribe noisy English sentences processed by five widely used single-channel speech enhancement algorithms. The speech intelligibility of the two groups was quantified and compared to investigate the effectiveness of the speech enhancement algorithms on non-native listeners who are familiar with the target language.

Advancements in end-to-end isolated Kannada ASR system by combining robust noise elimination technique and TDNN

The demonstration of the latest enhancements in the end-to-end (E2E) isolated Kannada automatic speech recognition (ASR) system, achieved by combining a robust background noise elimination technique and a time delay neural network (TDNN), is presented in this work. An E2E Kannada ASR system consists of an interactive voice response system (IVRS), ASR models, and databases containing weather and agricultural commodity prices information. In the earlier spoken query system (SQS), the presence of babble, street noise, and other background noises led to a decrease in both offline and online speech recognition accuracies. To properly train the models, we increase the size of the database by collecting the Kannada speech data from an additional 500 farmers under real-time conditions. Moreover, the proposed noise elimination technique is employed to enhance the degraded speech data. Additionally, the efficacy of the TDNN is explored to improve the recognition accuracy of ASR models and the SQS system.The outcomes of the proposed speech enhancement algorithm demonstrate the absence of audible musical noise and other types of background noises in enhanced NOIZEUS and isolated Kannada speech databases. Leveraging Kannada language resources and the amalgamation of the proposed noise reduction technique and TDNN, a significant 1.1% enhancement in speech recognition accuracy is achieved in comparison with the previously developed deep neural network-hidden Markov model (DNN-HMM) based SQS. The enhanced isolated E2E ASR SQS system undergoes testing by 500 farmers, enabling them to access real-time agricultural commodity prices and weather forecasting information in their native Kannada language/dialects. This practical validation highlights the applicability and effectiveness of our advancements in real-world scenarios. The source code of proposed noise elimination technique, experimental results of ASR models and demo conversation of SQS are made publicly available at: https://sites.google.com/view/thimmarajayadavag/downloads

Deep neural network-based speech enhancement for cochlear implants

Noisy conditions make understanding speech with a cochlear implant (CI) difficult. Speech enhancement (SE) algorithms based on signal statistics can be beneficial in stationary noise, but rarely provide benefit in modulated multi-talker babble. Current approaches using deep neural networks (DNNs) rely on a data driven approach for training and promise improvements in a wide variety of noisy conditions. In this study a DNN-based SE algorithm was evaluated in CI listeners. The network was trained on a large database of publicly available recordings. A double-blinded acute evaluation was conducted with 10 adult CI users by assessing intelligibility and quality of speech embedded in a range of different noise types. The DNN-based SE algorithm provided significant benefits in speech intelligibility and sound quality in all noise types that were evaluated. Speech reception thresholds, the SNR required to understand 50% of the speech material, improved by 1.8 to 3.5 dB depending on noise type. Benefits varied with the SNR of the input signal and the mixing ratio parameter that was used to combine the original and de-noised signals. The results demonstrate that DNN-based SE can provide benefits in natural, modulated noise conditions, which is critical to CI users in their day-to-day environment.

Deep-learning driven reverberation-aware microphone array speech enhancement

Traditional microphone array speech enhancement algorithms suffer from significant performance degradation in complex and diverse speech application scenarios. The adoption of data-driven microphone array optimization models based on machine learning (deep learning) is capable of achieve significant performance improvements. However, such methods exhibit data dependency. A deep learning based reverberation-aware microphone array speech enhancement algorithm is proposed. The proposed algorithm first obtains the reverberation pattern through reverberation-aware environment probing, and then extracts the main component features to train reverberation-aware network (RAN). This operation enables the RAN to have the ability to decouple and generate adaptive de-reverberation beamforming coefficients based on reverberation features. Simulation and experiments are provided to verify the effectiveness and robustness of the proposed method in different reverbrant environments.

Speech enhancement for helicopter headsets: Simulation and implementation on an FPGA platform

Clear and effective communication is crucial for the safe operation of aircraft. In helicopters, high levels of noise are generated by the engine, gears, and aerodynamics, which negatively impact speech intelligibility. To address this issue, modern aircraft headsets utilize active noise control (ANC) to reduce noise levels for both the crew and passengers. However, the speech signals captured by these headsets often contain high levels of background noise, thereby hindering internal and external flight communication. This paper introduces a dual microphone dual stage speech enhancement algorithm that combines basic spectral subtraction with a Wiener Filter, enhanced by the a priori and a posteriori signal-to-noise ratio. Audio data from within a helicopter cabin were recorded during a test flight. In a series of simulations, the Wiener Filter implementation is compared to other algorithms based only on spectral subtraction methods. The results are evaluated using established performance measures for speech quality. The Wiener Filter implementation results in the highest speech quality and is, therefore, implemented on an FPGA-platform for validation in a laboratory experiment. The simulations and measurements demonstrate significant improvements in speech quality and, consequently, enhance speech intelligibility using the proposed method.

Dual Stages of Speech Enhancement Algorithm Based on Super Gaussian Speech Models

Various speech enhancement Algorithms (SEA) have been developed in the last few decades. Each algorithm has its advantages and disadvantages because the speech signal is affected by environmental situations. Distortion of speech results in the loss of important features that make this signal challenging to understand. SEA aims to improve the intelligibility and quality of speech that different types of noise have degraded. In most applications, quality improvement is highly desirable as it can reduce listener fatigue, especially when the listener is exposed to high noise levels for extended periods (e.g., manufacturing). SEA reduces or suppresses the background noise to some degree, sometimes called noise suppression algorithms. In this research, the design of SEA based on different speech models (Laplacian model or Gaussian model) has been implemented using two types of discrete transforms, which are Discrete Tchebichef Transform and Discrete Tchebichef-Krawtchouk Transforms. The proposed estimator consists of dual stages of a wiener filter that can effectively estimate the clean speech signal. The evaluation measures' results show the proposed SEA's ability to enhance the noisy speech signal based on a comparison with other types of speech models and a self-comparison based on different types and levels of noise. The presented algorithm's improvements ratio regarding the average SNRseq are 1.96, 2.12, and 2.03 for Buccaneer, White, and Pink noise, respectively.

Speech Enhancement Algorithm Combining Cochlear Features and Deep Neural Network with Skip Connections

Speech Enhancement Algorithm Combining Cochlear Features and Deep Neural Network with Skip Connections