Speech Enhancement Algorithm Research Articles

An improved MMSE estimator based modified group delay spectrum for Forensic Automatic Speaker Recognition

This paper presents an improved speech enhancement algorithm called Minimum Mean Square Error (MMSE), based on the MODified Group Delay spectrum (MODGD), for Forensic Automatic Speaker Recognition (FASR) under noisy environments. This algorithm uses the MODGD instead of the amplitude spectrum, to compute the power spectrum of the noise-corrupt signal. In the proposed estimator, the MODGD retains most of the formants information. Therefore, it enhances the noisy speech signal with high quality even at extremely low Signal-to-Noise Ratio (SNR) levels. The evaluation of the improved algorithm in simulated FASR scenarios was performed by adding different noise levels, extracted from the NOISEX-92 database to the clean NIST2000-traces. The results obtained show that the proposed MMSE–MODGD estimator provides greater suppression of noise components in regions of low SNR than the MMSE estimator. In addition, there is a drastic reduction in Equal Proportion Probability (EPP) (the improvements are 1.84% for babble noise and 1.25% for factory and white noises), combining FASR techniques with the proposed MMSE–MODGD estimator than with the conventional estimator.

Speech Enhancement Algorithm Based on a Hybrid Estimator

Speech is the essential way to interact between humans or between human and machine. However, it is always contaminated with different types of environment noise. Therefore, speech enhancement algorithms (SEA) have appeared as a significant approach in speech processing filed to suppress background noise and return back the original speech signal. In this paper, a new efficient two-stage SEA with low distortion is proposed based on minimum mean square error sense. The estimation of clean signal is performed by taking the advantages of Laplacian speech and noise modeling based on orthogonal transform (Discrete Krawtchouk-Tchebichef transform) coefficients distribution. The Discrete Krawtchouk-Tchebichef transform (DKTT) has a high energy compaction and provides a high matching between Laplacian density and its coefficients distribution that affects positively on reducing residual noise without sacrificing speech components. Moreover, a cascade combination of hybrid speech estimator is proposed by using two stages filters (non-linear and linear) based on DKTT domain to lessen the residual noise effectively without distorting the speech signal. The linear estimator is considered as a post processing filter that reinforces the suppression of noise by regenerate speech components. To this end, the output results have been compared with existing work in terms of different quality and intelligibility measures. The comparative evaluation confirms the superior achievements of the proposed SEA in various noisy environments. The improvement ratio of the presented algorithm in terms of PESQ measure are 5.8% and 1.8% for white and babble noise environments, respectively. In addition, the improvement ratio of the presented algorithm in terms of OVL measure are 15.7% and 9.8% for white and babble noise environments, respectively.

Hybrid PSO-NLMS (HPSO-NLMS) algorithm for blind speech quality enhancement in time domain

Speech intelligibility enhancement by adaptive two-channel filtering algorithms in the time domain is a fundamental problem in the areas of signal processing application. In this paper, we address this problem and suggest a new adaptive hybrid algorithm combined with Forward blind source separation (FBSS) structure to retrieve a speech signal from noisy observations. Usually, the Normalized Least Mean Square algorithm (NLMS) is widely used in adaptive filtering application, however conventional NLMS presents difficulties to solve the trade-off between the speed of convergence and the low steady state error. Recently, numerous variable step-size NLMS (VNLMS) algorithms were proposed to solve this trade-off, our proposed algorithm falls within the same type of algorithm. The proposed Hybrid PSO-NLMS (HPSO-NLMS) algorithm is a combination of the NLMS and the Particle Swarm Optimization algorithms (PSO), where we use PSO to obtain the appropriate NMLS step size at each iteration. We integrate the new HPSO-NLMS in the FBSS structure, and compare their performances to conventional NLMS and PSO algorithms. We show that the proposed algorithm can produce better noise reduction performance, in terms of several objective metric such as the system misalignment (SM), the Segmental signal to noise Ratio (SegSNR) and the segmented mean square error (SegMSE).

Exploiting the directional coherence function for multichannel source extraction

The desired speech detector plays an important role for controlling the speech distortion in spatial filtering based speech enhancement algorithms. However, the conventional complex coherence(CC) based algorithms can only distinguish the coherent speech and diffuse noise. To improve the performance on the scenarios that both the coherent interference and diffuse noise are present, we propose a directional coherence function(DCF) based detector in this paper. Based on a pair of complementary filters which can suppress the diffuse noise and the coherent interference respectively, the DCF is computed as the normalized correlation between the filters’ outputs. Meanwhile, the filters are solved by convex programming method and satisfy the constraints on speech distortionless and white noise gain(WNG). Consequently, the value of DCF will be close to 1 only for the desired speech dominated time-frequency(T-F) bins and much smaller than 1 for the noise or interference dominated T-F bins. To extract the desired speech, the DCF based Desired Speech Presence Probability(DSPP) is used to control the adaptation in general sidelobe canceler(GSC), and subsequently used as the post-filtering weight. Systematical experiments on several scenarios show that the proposed algorithm achieves significantly and consistently better noise suppression performance than the narrowband direction-of-arrival(DOA) estimates based algorithms.

A Multi-Resolution Approach to GAN-Based Speech Enhancement

Recently, generative adversarial networks (GANs) have been successfully applied to speech enhancement. However, there still remain two issues that need to be addressed: (1) GAN-based training is typically unstable due to its non-convex property, and (2) most of the conventional methods do not fully take advantage of the speech characteristics, which could result in a sub-optimal solution. In order to deal with these problems, we propose a progressive generator that can handle the speech in a multi-resolution fashion. Additionally, we propose a multi-scale discriminator that discriminates the real and generated speech at various sampling rates to stabilize GAN training. The proposed structure was compared with the conventional GAN-based speech enhancement algorithms using the VoiceBank-DEMAND dataset. Experimental results showed that the proposed approach can make the training faster and more stable, which improves the performance on various metrics for speech enhancement.

Impulsive Noise Detection for Speech Enhancement in HHT Domain

This paper introduces a novel single channel speech enhancement method in the time domain to mitigate the effects of acoustic impulsive noises. The ensemble empirical mode decomposition is applied to analyze the noisy speech signal. The estimation and selection of noise components is based on the impulsiveness index of decomposition modes. An adaptive threshold is proposed to define the criterion to select the noise components. The proposed method is evaluated in speech enhancement experiments considering four acoustic noises with different impulsiveness indices and non-stationarity degrees under various signal-to-noise ratios. Four speech enhancement algorithms are adopted as baseline in the evaluation analysis considering spectral and time domains. Seven objective measures are adopted to compare the proposed and baseline approaches in terms of speech quality and intelligibility. Results show that the proposed solution outperforms the competing algorithms for most of the noisy scenarios. The novel method shows particularly interesting performance when speech signals are corrupted by highly impulsive acoustic noises.

Towards More Efficient DNN-Based Speech Enhancement Using Quantized Correlation Mask

Many studies on deep learning-based speech enhancement (SE) utilizing the computational auditory scene analysis method typically employs the ideal binary mask or the ideal ratio mask to reconstruct the enhanced speech signal. However, many SE applications in real scenarios demand a desirable balance between denoising capability and computational cost. In this study, first, an improvement over the ideal ratio mask to attain more superior SE performance is proposed through introducing an efficient adaptive correlation-based factor for adjusting the ratio mask. The proposed method exploits the correlation coefficients among the noisy speech, noise and clean speech to effectively re-distribute the power ratio of the speech and noise during the ratio mask construction phase. Second, to make the supervised SE system more computationally-efficient, quantization techniques are considered to reduce the number of bits needed to represent floating numbers, leading to a more compact SE model. The proposed quantized correlation mask is utilized in conjunction with a 4-layer deep neural network (DNN-QCM) comprising dropout regulation, pre-training and noise-aware training to derive a robust and high-order mapping in enhancement, and to improve generalization capability in unseen conditions. Results show that the quantized correlation mask outperforms the conventional ratio mask representation and the other SE algorithms used for comparison. When compared to a DNN with ideal ratio mask as its learning targets, the DNN-QCM provided an improvement of approximately 6.5% in the short-time objective intelligibility score and 11.0% in the perceptual evaluation of speech quality score. The introduction of the quantization method can reduce the neural network weights to a 5-bit representation from a 32-bit, while effectively suppressing stationary and non-stationary noise. Timing analyses also show that with the techniques incorporated in the proposed DNN-QCM system to increase its compactness, the training and inference time can be reduced by 15.7% and 10.5%, respectively.

Improving the Intelligibility of Speech for Simulated Electric and Acoustic Stimulation Using Fully Convolutional Neural Networks.

Combined electric and acoustic stimulation (EAS) has demonstrated better speech recognition than conventional cochlear implant (CI) and yielded satisfactory performance under quiet conditions. However, when noise signals are involved, both the electric signal and the acoustic signal may be distorted, thereby resulting in poor recognition performance. To suppress noise effects, speech enhancement (SE) is a necessary unit in EAS devices. Recently, a time-domain speech enhancement algorithm based on the fully convolutional neural networks (FCN) with a short-time objective intelligibility (STOI)-based objective function (termed FCN(S) in short) has received increasing attention due to its simple structure and effectiveness of restoring clean speech signals from noisy counterparts. With evidence showing the benefits of FCN(S) for normal speech, this study sets out to assess its ability to improve the intelligibility of EAS simulated speech. Objective evaluations and listening tests were conducted to examine the performance of FCN(S) in improving the speech intelligibility of normal and vocoded speech in noisy environments. The experimental results show that, compared with the traditional minimum-mean square-error SE method and the deep denoising autoencoder SE method, FCN(S) can obtain better gain in the speech intelligibility for normal as well as vocoded speech. This study, being the first to evaluate deep learning SE approaches for EAS, confirms that FCN(S) is an effective SE approach that may potentially be integrated into an EAS processor to benefit users in noisy environments.

Speech enhancement based on perceptually motivated guided spectrogram filtering

In this paper, a single-channel speech enhancement algorithm is proposed by using guided spectrogram filtering based on masking properties of human auditory system when considering a speech spectrogram as an image. Guided filtering is capable of sharpening details and estimating unwanted textures or background noise from the noisy speech spectrogram. If we consider the noisy spectrogram as a degraded image, we can estimate the spectrogram of the clean speech signal using guided filtering after subtracting noise components. Combined with masking properties of human auditory system, the proposed algorithm adaptively adjusts and reduces the residual noise of the enhanced speech spectrogram according to the corresponding masking threshold. Because the filtering output is a local linear transform of the guidance spectrogram, the local mask window slides can be efficiently implemented via box filter with O(N) computational complexity. Experimental results show that the proposed algorithm can effectively suppress noise in different noisy environments and thus can greatly improve speech quality and speech intelligibility.

Nonlinear Spatial Filtering in Multichannel Speech Enhancement

The majority of multichannel speech enhancement algorithms are two-step procedures that first apply a linear spatial filter, a so-called beamformer, and combine it with a single-channel approach for postprocessing. However, the serial concatenation of a linear spatial filter and a postfilter is not generally optimal in the minimum mean square error (MMSE) sense for noise distributions other than a Gaussian distribution. Rather, the MMSE optimal filter is a joint spatial and spectral nonlinear function. While estimating the parameters of such a filter with traditional methods is challenging, modern neural networks may provide an efficient way to learn the nonlinear function directly from data. To see if further research in this direction is worthwhile, in this work we examine the potential performance benefit of replacing the common two-step procedure with a joint spatial and spectral nonlinear filter. We analyze three different forms of non-Gaussianity: First, we evaluate on super-Gaussian noise with a high kurtosis. Second, we evaluate on inhomogeneous noise fields created by five interfering sources using two microphones, and third, we evaluate on real-world recordings from the CHiME3 database. In all scenarios, considerable improvements may be obtained. Most prominently, our analyses show that a nonlinear spatial filter uses the available spatial information more effectively than a linear spatial filter as it is capable of suppressing more than $D-1$ directional interfering sources with a $D$-dimensional microphone array without spatial adaptation.

DNN-Based Mask Estimation for Distributed Speech Enhancement in Spatially Unconstrained Microphone Arrays

Deep neural network (DNN)-based speech enhancement algorithms in microphone arrays have now proven to be efficient solutions to speech understanding and speech recognition in noisy environments. However, in the context of ad-hoc microphone arrays, many challenges remain and raise the need for distributed processing. In this paper, we propose to extend a previously introduced distributed DNN-based time-frequency mask estimation scheme that can efficiently use spatial information in form of so-called compressed signals which are pre-filtered target estimations. We study the performance of this algorithm named Tango under realistic acoustic conditions and investigate practical aspects of its optimal application. We show that the nodes in the microphone array cooperate by taking profit of their spatial coverage in the room. We also propose to use the compressed signals not only to convey the target estimation but also the noise estimation in order to exploit the acoustic diversity recorded throughout the microphone array.

DeepLPC-MHANet: Multi-Head Self-Attention for Augmented Kalman Filter-Based Speech Enhancement

Current augmented Kalman filter (AKF)-based speech enhancement algorithms utilise a temporal convolutional network (TCN) to estimate the clean speech and noise linear prediction coefficient (LPC). However, the multi-head attention network MHANet) has demonstrated the ability to more efficiently model the long-term dependencies of noisy speech than TCNs. Motivated by this, we investigate the MHANet for LPC estimation. We aim to produce clean speech and noise LPC parameters with the least bias to date. With this, we also aim to produce higher quality and more intelligible enhanced speech than any current KF or AKF-based SEA. To this end, we investigate MHANet within the DeepLPC framework. DeepLPC is a deep learning framework for jointly estimating the clean speech and noise LPC power spectra. DeepLPC is selected as it exhibits significantly less bias than other frameworks, by avoiding the use of whitening filters and post-processing. DeepLPC-MHANet is evaluated on the NOIZEUS corpus using subjective AB listening tests, as well as seven different objective measures (CSIG, CBAK, COVL, PESQ, STOI, SegSNR, and SI-SDR). DeepLPC-MHANet is compared to five existing deep learning-based methods. Compared to other deep learning approaches, DeepLPC-MHANet produced clean speech LPC estimates with the least amount of bias. DeepLPC-MHANet-AKF also produced higher objective scores than any of the competing methods (with an improvement of 0.17 for CSIG, 0.15 for CBAK, 0.19 for COVL, 0.24 for PESQ, 3.70% for STOI, 1.03 dB for SegSNR, and 1.04 dB for SI-SDR over the next best method). The enhanced speech produced by DeepLPC-MHANet-AKF was also the most preferred amongst ten listeners. By producing LPC estimates with the least amount of bias to date, DeepLPC-MHANet enables the AKF to produce enhanced speech at a higher quality and intelligibility than any previous KF or AKF-based method.

Enhancement of Noisy Reverberant Speech Using Polynomial Matrix Eigenvalue Decomposition

Speech enhancement is important for applications such as telecommunications, hearing aids, automatic speech recognition and voice-controlled systems. Enhancement algorithms aim to reduce interfering noise and reverberation while minimizing any speech distortion. In this work for speech enhancement, we propose to use polynomial matrices to model the spatial, spectral and temporal correlations between the speech signals received by a microphone array and polynomial matrix eigenvalue decomposition (PEVD) to decorrelate in space, time and frequency simultaneously. We then propose a blind and unsupervised PEVD-based speech enhancement algorithm. Simulations and informal listening examples involving diverse reverberant and noisy environments have shown that our method can jointly suppress noise and reverberation, thereby achieving speech enhancement without introducing processing artefacts into the enhanced signal.

Evaluation of speech enhancement algorithms applied to electrolaryngeal speech degraded by noise

Electrolaryngeal speech, for persons who have lost their larynx, suffers from the drawback of susceptibility to acoustic noise, which includes inherent electrolarynx motor noise as well as environmental noise. Interactions with electrolarynx users motivated the authors to investigate a crucial drawback of electrolaryngeal speech: degradation of electrolaryngeal speech under noisy environments. The effect of contemporary methods of speech enhancement, viz. noise power spectral density estimation based d-dimensional amplitude trimmed estimation (DATE), and non-negative matrix factorization (NMF) based algorithms, were studied and evaluated for electrolaryngeal speech degraded by noise. Electrolaryngeal speech was corrupted using three types of noisy scenarios at low signal to noise ratios (0, −5 and −10 dB SNR). Objective testing based on the perceptual evaluation of speech quality (PESQ) standard, as well as subjective testing by 14 participants based on the ITU-T P.835 standard were performed. Word-centric intelligibility testing was also performed. The results indicated an improvement in the speech quality. The subjective testing results were further analyzed using one-way analysis of variance (ANOVA) and multiple paired comparison using Tukey’s honest significant difference (HSD) criterion. The overall speech quality of NMF algorithms was found to be significantly higher than that of DATE-based algorithms for the test conditions. The results show that speech enhancement algorithms can aid in improving the electrolaryngeal user experience by reducing the effect of acoustic noise.

Speech enhancement by LSTM-based noise suppression followed by CNN-based speech restoration

Single-channel speech enhancement in highly non-stationary noise conditions is a very challenging task, especially when interfering speech is included in the noise. Deep learning-based approaches have notably improved the performance of speech enhancement algorithms under such conditions, but still introduce speech distortions if strong noise suppression shall be achieved. We propose to address this problem by using a two-stage approach, first performing noise suppression and subsequently restoring natural sounding speech, using specifically chosen neural network topologies and loss functions for each task. A mask-based long short-term memory (LSTM) network is employed for noise suppression and speech restoration is performed via spectral mapping with a convolutional encoder-decoder network (CED). The proposed method improves speech quality (PESQ) over state-of-the-art single-stage methods by about 0.1 points for unseen highly non-stationary noise types including interfering speech. Furthermore, it is able to increase intelligibility in low-SNR conditions and consistently outperforms all reference methods.

Speech Enhancement Based on Modulation-Domain Parametric Multichannel Kalman Filtering

Recently we presented a modulation-domain multichannel Kalman filtering (MKF) algorithm for speech enhancement, which jointly exploits the inter-frame modulation-domain temporal evolution of speech and the inter-channel spatial correlation to estimate the clean speech signal. The goal of speech enhancement is to suppress noise while keeping the speech undistorted, and a key problem is to achieve the best trade-off between speech distortion and noise reduction. In this paper, we extend the MKF by presenting a modulation-domain parametric MKF (PMKF) which includes a parameter that enables flexible control of the speech enhancement behaviour in each time-frequency (TF) bin. Based on the decomposition of the MKF cost function, a new cost function for PMKF is proposed, which uses the controlling parameter to weight the noise reduction and speech distortion terms. An optimal PMKF gain is derived using a minimum mean squared error (MMSE) criterion. We analyse the performance of the proposed MKF, and show its relationship to the speech distortion weighted multichannel Wiener filter (SDW-MWF). To evaluate the impact of the controlling parameter on speech enhancement performance, we further propose PMKF speech enhancement systems in which the controlling parameter is adaptively chosen in each TF bin. Experiments on a publicly available head-related impulse response (HRIR) database in different noisy and reverberant conditions demonstrate the effectiveness of the proposed method.

Affine Projection Algorithm for Speech Enhancement using Controlled Projection Order

This research presents a development of the affine projection algorithm (APA) in voice communication applications. A method of controlling the parameters of the APA is devised to improve the performance in cancelling various types of ambient noise that could possibly corrupt speech signals in voice communication systems. Indicators are used to identify the type of noise accompanying the target signal. Then the corrupted signal is processed in a noise a cancellation setup in such a way that three parameters of algorithm are changed according to the nature of the noise. The spreading of elements in the covariance matrix of the noise is used as an indicator for the type of noise so that the projection order, step-size and filter length are changed at the same time. This way the performance of the canceller is improved rendering lower estimation error with a moderate computational power. The method was tested under various types of noise and showed better convergence performance than the original APA as well as other commonly used algorithms in noise cancellation systems. The MSE of the proposed VPAPA method drops to -65 dB in steady state compared to -20 dB using NLMS and just below -30 dB using standard APA with projection order of 8, while the powerful RLS reaches around -60dB under the same environment. The method can be useful for clearer voice communication in variable environmental noise.

Minimum mean square error estimator for speech enhancement in additive noise assuming Weibull speech priors and speech presence uncertainty

A novel single-channel technique was proposed based on a minimum mean square error (MMSE) estimator to enhance short-time spectral amplitude (STSA) in the Discrete Fourier Transform (DFT) domain. In the present contribution, a Weibull distribution was used to model DFT magnitudes of clean speech signals under the additive Gaussian noise assumption. Moreover, the speech enhancement procedure was conducted with (WSPU) and without speech presence uncertainty (WoSPU). The theoretical spectral gain function was obtained as a weighted geometric mean of hypothetical gains associated with signal presence and absence. Extensive experiments were conducted with clean speech signals taken from the TIMIT database, which had been degraded by various additive non-stationary noise sources, and then enhanced signals were evaluated. The evaluation results demonstrated the outperformance of the proposed method compared to the probability density functions (PDF) of Rayleigh and Gamma distributions in terms of segmental signal-to-noise ratio (segSNR), general SNR, and perceptual evaluation of speech quality (PESQ). The performance in the WSPU case was also significantly improved compared to WoSPU, assuming Weibull speech priors in the MMSE-STSA based speech enhancement algorithm.

Impact of phase distortion and phase-insensitive speech enhancement on speech quality perceived by hearing-impaired listeners

Phase is important for speech since it contributes to the quality and intelligibility during speech perception. Many speech enhancement algorithms lack the ability to predict phase for speech reconstruction and apply the noisy phase instead. In this study, we investigated the influence of phase distortion on the speech-quality ratings of both normal-hearing (NH) and hearing-impaired (HI) listeners by applying different degrees of random phase on speech. In one set of conditions, the speech was embedded in babble noise at 4 different signal-to-noise ratios (SNRs) from -5 to 10 dB, while in another set of conditions, the SNR was fixed at 10 dB and the speech and noise mixture was presented in simulated rooms with reverberation times ranged from 100 to 1000 ms. The speech level was kept at 65 dB SPL for NH listeners; while amplification was applied to ensure the audibility for HI listeners. Ideal ratio mask (IRM) was used for speech enhancement. Speech-quality ratings were collected following the MUSHRA procedure and compared to two objective evaluation metrics (i.e., PESQ and HASQI). Results suggest that phase distortion has negative impact on speech quality for both NH and HI listeners and this effect becomes dominant when speech enhancement was applied.

Research on Speech Enhancement Based on Deep Neural Network

The traditional single-channel speech enhancement algorithm has many unreasonable assumptions, which limit the performance of the algorithm. The speech enhancement based on deep neural network can effectively eliminate the problems such as “music noise” in the traditional methods, thus achieving better results than the traditional single-channel speech enhancement. However, the DNN-based speech enhancement model does not perform well at low SNR. In order to improve the robustness of the model, we used voice activity detection (VAD) to process the training data to obtain a new VAD-DNN speech enhancement model. We added 100 kinds of noise in the training set to improve the ability of the model to deal with the unseen noise. At the same time, in order to prevent the occurrence of overfitting problem, we introduced dropout technology to process the network, and improved the generalization ability of the model by using noise awareness training (NAT). Through experiments, we found that the speech enhancement model based on VAD-DNN improved PESQ index by 0.02 and STOI index by 0.01 on average under the condition of low SNR.