Single-channel Speech Enhancement Method Research Articles

A Single-Channel Speech Enhancement Algorithm Combined with Time-Frequency Mask

Aiming at the problem of noise overestimation when the classical single-channel speech enhancement method suppresses noise, single-channel speech enhancement method on the basis of the time-frequency mask is put forward to improve the quality of speech enhancement and separation. First, we estimate the noise and prior signal-to-noise ratio from the noisy speech, calculate the time-frequency mask and finally combine noisy speech synthesis to enhance the speech. Adaptive time-frequency mask combines IBM and IRM to avoid over-suppressing noise. In this paper, noisy speech is processed by an improved time-frequency mask combined with SNR. The algorithm combines the advantages of the single-channel algorithm and time-frequency mask. The experimental results indicate that this method significantly improves the signal-to-noise ratio, is robust, and is easy to use.

A speech enhancement algorithm based on a non-negative hidden Markov model and Kullback-Leibler divergence

In this paper, we propose a supervised single-channel speech enhancement method that combines Kullback-Leibler (KL) divergence-based non-negative matrix factorization (NMF) and a hidden Markov model (NMF-HMM). With the integration of the HMM, the temporal dynamics information of speech signals can be taken into account. This method includes a training stage and an enhancement stage. In the training stage, the sum of the Poisson distribution, leading to the KL divergence measure, is used as the observation model for each state of the HMM. This ensures that a computationally efficient multiplicative update can be used for the parameter update of this model. In the online enhancement stage, a novel minimum mean square error estimator is proposed for the NMF-HMM. This estimator can be implemented using parallel computing, reducing the time complexity. Moreover, compared to the traditional NMF-based speech enhancement methods, the experimental results show that our proposed algorithm improved the short-time objective intelligibility and perceptual evaluation of speech quality by 5% and 0.18, respectively.

SINGLE CHANNEL SPEECH ENHANCEMENT USING EVOLUTIONARY ALGORITHM WITH LOG-MMSE

Additive noise degrades speech quality and intelligibility. Speech enhancement reduces this noise to make speech more pleasant and intelligible. It plays a significant role in speech recognition or speech-operated systems. In this paper, we propose a single-channel speech enhancement method in which the log-minimum mean square error method (log-MMSE) and modified accelerated particle swarm optimization algorithm are used to design a filter for improving the quality and intelligibility of noisy speech. Accelerated particle swarm optimization (APSO) algorithm is modified in which a single dimension of particle position is changed in a single iteration while obtaining the particle’s new position. Using this algorithm, a filter is designed with multiple passbands and notches for speech enhancement. The modified algorithm converges faster compared with standard particle swarm optimization algorithm (PSO) and APSO giving optimum filter coefficients. The designed filter is used to enhance the speech. The proposed speech enhancement method improves the perceptual estimation of speech quality (PESQ) by 17.05% for 5dB babble noise, 33.92 % for 5dB car noise, 14.96 % for 5dB airport noise, and 39.13 % for 5dB exhibition noise. The average output PESQ for these four types of noise is improved compared to conventional methods of speech enhancement. There is an average of 7.58 dB improvement in segmental SNR for these noise types. The proposed method improves speech intelligibility with minimum speech distortion.

Smartphone-based single-channel speech enhancement application for hearing aids.

This work presents a single-channel speech enhancement (SE) framework based on the super-Gaussian extension of the joint maximum a posteriori (SGJMAP) estimation rule. The developed SE algorithm is an open-source research smartphone-based application for hearing improvement studies. In this algorithm, the SGJMAP-based estimation for noisy speech mixture is smoothed along the frequency axis by a Mel filter-bank, resulting in a Mel-warped frequency-domain SGJMAP estimation. The impulse response of this Mel-warped estimation is obtained by applying a Mel-warped inverse discrete cosine transform (Mel-IDCT). This helps in filtering out the background noise and enhancing the speech signal. The proposed application is implemented on an iPhone (Apple, Cupertino, CA) to operate in real time and tested with normal-hearing (NH) and hearing-impaired (HI) listeners with different types of hearing aids through wireless connectivity. The objective speech quality and intelligibility test results are used to compare the performance of the proposed algorithm to existing conventional single-channel SE methods. Additionally, test results from NH and HI listeners show substantial improvement in speech recognition with the developed method in simulated real-world noisy conditions at different signal-to-noise ratio levels.

Impulsive Noise Detection for Speech Enhancement in HHT Domain

This paper introduces a novel single channel speech enhancement method in the time domain to mitigate the effects of acoustic impulsive noises. The ensemble empirical mode decomposition is applied to analyze the noisy speech signal. The estimation and selection of noise components is based on the impulsiveness index of decomposition modes. An adaptive threshold is proposed to define the criterion to select the noise components. The proposed method is evaluated in speech enhancement experiments considering four acoustic noises with different impulsiveness indices and non-stationarity degrees under various signal-to-noise ratios. Four speech enhancement algorithms are adopted as baseline in the evaluation analysis considering spectral and time domains. Seven objective measures are adopted to compare the proposed and baseline approaches in terms of speech quality and intelligibility. Results show that the proposed solution outperforms the competing algorithms for most of the noisy scenarios. The novel method shows particularly interesting performance when speech signals are corrupted by highly impulsive acoustic noises.

PhaseDCN: A Phase-Enhanced Dual-Path Dilated Convolutional Network for Single-Channel Speech Enhancement

Recent deep neural network (DNN) based single-channel speech enhancement methods have achieved remarkable results in the time-frequency (TF) magnitude domain. To further improve the quality and intelligibility of enhanced speech, the attention to phase enhancement is also increasing. In this paper, we propose a novel dilated convolutional network (DCN) model to simultaneously enhance the magnitude and phase of noisy speech. Unlike the direct complex spectral mapping methods, we take the complex spectrum of the signal as the main target and the ideal ratio mask (IRM) as the auxiliary target in a multi-target learning framework to achieve their complementary advantages. Firstly, a feature extraction module is introduced to achieve the fusion of local and long-term features. Two different targets are learned separately, but share the common feature extraction module, which is helpful to extract more general and suitable features. During the joint learning, the intermediate estimation of the IRM target in the auxiliary path, contributing as the attention gating factors, helps to distinguish the speech or non-speech components of the complex-valued signals in the main path. To leverage more fine-grained long-term contextual information, we introduce a multi-scale dilated convolution approach for feature encoding. Moreover, the proposed model is a causal system, which can fully meet the low latency requirements of real-time speech products. Experimental results show that, compared with other advanced systems, the proposed model not only has better speech denoising performance and phase estimation accuracy, but also generalizes better in the speaker, noise, and channel mismatch cases.

Single‐channel dereverberation and denoising based on lower band trained SA‐LSTMs

The supervised single-channel speech enhancement presents one mixture recording at the input of the neural network and updates network parameters in order to generate an output as the reconstructed speech signal. However, current neural networks-based single-channel speech enhancement methods are not able to fully utilise pertinence with the specific frequency range of speech signals with limited computational complexity. In this study, the authors studied the power spectral density of mixtures with human speech and noise interferences. Based on the theory that the speech signal distributes at the lower band, they proposed a method to train signal approximation (SA) based neural networks with the lower frequency band of the speech mixture to improve the performance. To realise the lower band approach for single-channel speech enhancement, the method uses a long short-term memory (LSTM) block to exploit short-time Fourier transform of the desired frequency range. Furthermore, in order to improve the speech enhancement performance within reverberant room environments, the dereverberation mask and the enhanced ratio mask are exploited as the training targets of two LSTM blocks, respectively. The detailed evaluations confirm that the proposed method outperforms the state-of-the-art methods.

Real-time single-channel deep neural network-based speech enhancement on edge devices.

In this paper, we present a deep neural network architecture comprising of both convolutional neural network (CNN) and recurrent neural network (RNN) layers for real-time single-channel speech enhancement (SE). The proposed neural network model focuses on enhancing the noisy speech magnitude spectrum on a frame-by-frame process. The developed model is implemented on the smartphone (edge device), to demonstrate the real-time usability of the proposed method. Perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) test results are used to compare the proposed algorithm to previously published conventional and deep learning-based SE methods. Subjective ratings show the performance improvement of the proposed model over the other baseline SE methods.

Multi-Task Learning U-Net for Single-Channel Speech Enhancement and Mask-Based Voice Activity Detection

In this paper, a multi-task learning U-shaped neural network (MTU-Net) is proposed and applied to single-channel speech enhancement (SE). The proposed MTU-based SE method estimates an ideal binary mask (IBM) or an ideal ratio mask (IRM) by extending the decoding network of a conventional U-Net to simultaneously model the speech and noise spectra as the target. The effectiveness of the proposed SE method was evaluated under both matched and mismatched noise conditions between training and testing by measuring the perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI). Consequently, the proposed SE method with IRM achieved a substantial improvement with higher average PESQ scores by 0.17, 0.52, and 0.40 than other state-of-the-art deep-learning-based methods, such as the deep recurrent neural network (DRNN), SE generative adversarial network (SEGAN), and conventional U-Net, respectively. In addition, the STOI scores of the proposed SE method are 0.07, 0.05, and 0.05 higher than those of the DRNN, SEGAN, and U-Net, respectively. Next, voice activity detection (VAD) is also proposed by using the IRM estimated by the proposed MTU-Net-based SE method, which is fundamentally an unsupervised method without any model training. Then, the performance of the proposed VAD method was compared with the performance of supervised learning-based methods using a deep neural network (DNN), a boosted DNN, and a long short-term memory (LSTM) network. Consequently, the proposed VAD methods show a slightly better performance than the three neural network-based methods under mismatched noise conditions.

Comparison and real-time implementation of fixed and adaptive beamformers for speech enhancement on smartphones for hearing aid study

In this paper, we compare the performance of fixed and adaptive beamformers as an application to speech enhancement (SE) algorithm. The proposed signal processing pipeline works in real-time for hearing aid devices using only a smartphone with no additional hardware as an assistive tool. The proposed method consists of a Wiener filter based single-channel SE method along with beamformer as a pre-filter using only two built-in microphones of the smartphone. In this work, multiple beamforming approaches like delay and sum, minimum variance distortionless response (MVDR) and generalized sidelobe canceller (GSC) beamformers are considered and compared. The use of beamformers is shown to improve the signal to noise ratio (SNR) of speech signal in real-world noisy conditions through the results presented in this paper. Objective test measures are used to evaluate perceptual quality and intelligibility of enhanced speech achieved by different approaches. Test results show the performance comparison of different approaches as well as the efficiency of proposed method at different SNR levels.

Modulation-Domain Multichannel Kalman Filtering for Speech Enhancement

Compared with single-channel speech enhancement methods, multichannel methods can utilize spatial information to design optimal filters. Although some filters adaptively consider second-order signal statistics, the temporal evolution of the speech spectrum is usually neglected. By using linear prediction (LP) to model the inter-frame temporal evolution of speech, single-channel Kalman filtering (KF) based methods have been developed for speech enhancement. In this paper, we derive a multichannel KF (MKF) that jointly uses both interchannel spatial correlation and interframe temporal correlation for speech enhancement. We perform LP in the modulation domain, and by incorporating the spatial information, derive an optimal MKF gain in the short-time Fourier transform domain. We show that the proposed MKF reduces to the conventional multichannel Wiener filter if the LP information is discarded. Furthermore, we show that, under an appropriate assumption, the MKF is equivalent to a concatenation of the minimum variance distortion response beamformer and a single-channel modulation-domain KF and therefore present an alternative implementation of the MKF. Experiments conducted on a public head-related impulse response database demonstrate the effectiveness of the proposed method.

Single channel speech enhancement based on harmonic estimation combined with statistical based method to improve speech intelligibility for cochlear implant recipients

In this study, we propose a single microphone speech enhancement algorithm by combining harmonic structure estimation and traditional MMSE speech enhancement for a leveraged overall solution. Traditional single channel speech enhancement methods are usually based on the statistic characteristics of noise signals which are effective only for stationary noise, but not for non-stationary noise. In our study, we attempt to estimate noise by exploring the harmonic structure of the target speech combined with temporal noise tracking. In voiced segments, since speech energy is sparsely carried by harmonic partials, the spectrum located between adjacent harmonic partials are considered as noise. We assume that the speech spectrum distributes continuously along the frequency-dimension. Thus, the noise overlapped with speech harmonics can be estimated with an interpolation technique. Next, the estimated noise is incorporated into a traditional MMSE framework for speech enhancement. A listening test is carried out with 6 cochlear implant recipients to evaluate the proposed speech enhancement algorithm. The experimental results show that the proposed algorithm is able to improve the speech intelligibility in terms of word recognition rate for CI listeners.

Single-Channel Online Enhancement of Speech Corrupted by Reverberation and Noise

This paper proposes an online single-channel speech enhancement method designed to improve the quality of speech degraded by reverberation and noise. Based on an autoregressive model for the reverberation power and on a hidden Markov model for clean speech production, a Bayesian filtering formulation of the problem is derived and online joint estimation of the acoustic parameters and mean speech, reverberation, and noise powers is obtained in mel-frequency bands. From these estimates, a real-valued spectral gain is derived and spectral enhancement is applied in the short-time Fourier transform STFT domain. The method yields state-of-the-art performance and greatly reduces the effects of reverberation and noise while improving speech quality and preserving speech intelligibility in challenging acoustic environments.

Push-to-talk 통신을 위한 진폭 및 위상 복원 기반의 단일 채널 음성 향상 방식

Push-to-talk 통신을 위한 진폭 및 위상 복원 기반의 단일 채널 음성 향상 방식

Single-channel speech enhancement method using reconstructive NMF with spectrotemporal speech presence probabilities

In this paper, a novel single microphone channel-based speech enhancement technique is presented. While most of the conventional nonnegative matrix factorization-based approaches focus on generating a basis matrix of speech and noise for enhancement, the proposed algorithm performs an additional process to reconstruct speech from noisy speech when these two elements are highly overlapped in selected spectral bands. This process involves a log-spectral amplitude based estimator, which provides the spectrotemporal speech presence probability to obtain a more accurate reconstruction. Moreover, the proposed algorithm applies an unsupervised learning method to the input noise, so it is adaptable to any type of environmental noise without a pre-trained dictionary. The experimental results demonstrate that the proposed algorithm obtains improved speech enhancement performance compared with conventional single channel-based approaches.

Adaptive Single-Channel Speech Enhancement Method for a Push-To-Talk Enabled Wireless Communication Device

Adaptive Single-Channel Speech Enhancement Method for a Push-To-Talk Enabled Wireless Communication Device

Sparse Hidden Markov Models for Speech Enhancement in Non-Stationary Noise Environments

We propose a sparse hidden Markov model (HMM)-based single-channel speech enhancement method that models the speech and noise gains accurately in non-stationary noise environments. Autoregressive models are employed to describe the speech and noise in a unified framework and the speech and noise gains are modeled as random processes with memory. The likelihood criterion for finding the model parameters is augmented with an ${l_p}$ regularization term resulting in a sparse autoregressive HMM (SARHMM) system that encourages sparsity in the speech- and noise- modeling. In the SARHMM only a small number of HMM states contribute significantly to the model of each particular observed speech segment. As it eliminates ambiguity between noise and speech spectra, the sparsity of speech and noise modeling helps to improve the tracking of the changes of both spectral shapes and power levels of non-stationary noise. Using the modeled speech and noise SARHMMs, we first construct a noise estimator to estimate the noise power spectrum. Then, a Bayesian speech estimator is derived to obtain the enhanced speech signal. The subjective and objective test results indicate that the proposed speech enhancement scheme can achieve a larger segmental SNR improvement, a lower log-spectral distortion and a better speech quality in stationary noise conditions than state-of-the-art reference methods. The advantage of the new method is largest for non-stationary noise conditions .

Speech enhancement based on Bayesian decision and spectral amplitude estimation

In this paper, a single-channel speech enhancement method based on Bayesian decision and spectral amplitude estimation is proposed, in which the speech detection module and spectral amplitude estimation module are included, and the two modules are strongly coupled. First, under the decisions of speech presence and speech absence, the optimal speech amplitude estimators are obtained by minimizing a combined Bayesian risk function, respectively. Second, using the obtained spectral amplitude estimators, the optimal speech detector is achieved by further minimizing the combined Bayesian risk function. Finally, according to the detection results of speech detector, the optimal decision rule is made and the optimal spectral amplitude estimator is chosen for enhancing noisy speech. Furthermore, by considering both detection and estimation errors, we propose a combined cost function which incorporates two general weighted distortion measures for the speech presence and speech absence of the spectral amplitudes, respectively. The cost parameters in the cost function are employed to balance the speech distortion and residual noise caused by missed detection and false alarm, respectively. In addition, we propose two adaptive calculation methods for the perceptual weighted order p and the spectral amplitude order β concerned in the proposed cost function, respectively. The objective and subjective test results indicate that the proposed method can achieve a more significant segmental signal-noise ratio (SNR) improvement, a lower log-spectral distortion, and a better speech quality than the reference methods.

Binary mask based method for enhancement of mixed noise speech of low SNR input

This paper presents a noise reduction method based on binary mask thresholding function for enhancement in single channel speech patterns of mixed highly non-stationary noises with low (negative) input SNR. For this purpose, a mixed highly non-stationary noisy speech database is generated by using noise and clean speech database of AURORA and INDIC speech, respectively. Results are compared with widely used methods such as Daubechies13, Daubechies40, Symlet13, Coiflet5, Wiener, Spectral Subtraction, and log-MMSE for performance evaluation in terms of SNR, PESQ, and Cepstrum distance parameters. In comparison to other methods the proposed single-channel speech enhancement method shows satisfactory results and obtained significant improvement in speech quality and intelligibility.

Speech enhancement based on wavelet packet of an improved principal component analysis

In this paper, we propose a single-channel speech enhancement method, based on the combination of the wavelet packet transform and an improved version of the principal component analysis (PCA). Our method integrates ability of PCA to de-correlate the coefficients by extracting a linear relationship with what of wavelet packet analysis to derive feature vectors used for speech enhancement. This allows us to operate with a convenient shrinkage function on these new coefficients, removing the noise without degrading the speech. Then, the enhanced speech obtained by the inverse wavelet packet transform is decomposed into three subspaces: low rank, sparse, and the remainder noise components. Finally, we calculate the components as a segregation problem. The performance evaluation shows that our method provides a higher noise reduction and a lower signal distortion even in highly noisy conditions without introducing artifacts.