Single-channel Speech Enhancement Algorithms Research Articles

Single-channel speech enhancement algorithm based on ME-MGCRN in low signal-to-noise scenario

Single-channel speech enhancement algorithm based on ME-MGCRN in low signal-to-noise scenario

Performance of single-channel speech enhancement algorithms on Mandarin listeners with different immersion conditions in New Zealand English

Speech enhancement (SE) is a widely used technology to improve the quality and intelligibility of noisy speech. So far, SE algorithms were designed and evaluated on native listeners only, but not on non-native listeners who are known to be more disadvantaged when listening in noisy environments. This paper investigates the performance of five widely used single-channel SE algorithms on early-immersed New Zealand English (NZE) listeners and native Mandarin listeners with different immersion conditions in NZE under negative input signal-to-noise ratio (SNR) by conducting a subjective listening test in NZE sentences. The performance of the SE algorithms in terms of speech intelligibility in the three participant groups was investigated. The result showed that the early-immersed group always achieved the highest intelligibility. The late-immersed group outperformed the non-immersed group for higher input SNR conditions, possibly due to the increasing familiarity with the NZE accent, whereas this advantage disappeared at the lowest tested input SNR conditions. The SE algorithms tested in this study failed to improve and rather degraded the speech intelligibility, indicating that these SE algorithms may not be able to reduce the perception gap between early-, late- and non-immersed listeners, nor able to improve the speech intelligibility under negative input SNR in general. These findings have implications for the future development of SE algorithms tailored to Mandarin listeners, and for understanding the impact of language immersion on speech perception in noise.

Performance of speech enhancement algorithms on the speech intelligibility of native Mandarin listeners immersed in English-speaking environment

Speech enhancement algorithms have been developed to improve speech intelligibility for listeners under noisy conditions. However, all existing algorithms were evaluated only by native listeners, the performance of such algorithms on non-native listeners was rarely investigated. This study conducts a subjective listening test on native New Zealand English listeners and native Mandarin listeners who have been immersed in New Zealand English-speaking environment for more than one year. The participants were asked to transcribe noisy English sentences processed by five widely used single-channel speech enhancement algorithms. The speech intelligibility of the two groups was quantified and compared to investigate the effectiveness of the speech enhancement algorithms on non-native listeners who are familiar with the target language.

Deep Neural Network for Supervised Single-Channel Speech Enhancement

Speech enhancement is fundamental for various real time speech applications and it is a challenging task in the case of a single channel because practically only one data channel is available. We have proposed a supervised single channel speech enhancement algorithm in this paper based on a deep neural network (DNN) and less aggressive Wiener filtering as additional DNN layer. During the training stage the network learns and predicts the magnitude spectrums of the clean and noise signals from input noisy speech acoustic features. Relative spectral transform-perceptual linear prediction (RASTA-PLP) is used in the proposed method to extract the acoustic features at the frame level. Autoregressive moving average (ARMA) filter is applied to smooth the temporal curves of extracted features. The trained network predicts the coefficients to construct a ratio mask based on mean square error (MSE) objective cost function. The less aggressive Wiener filter is placed as an additional layer on the top of a DNN to produce an enhanced magnitude spectrum. Finally, the noisy speech phase is used to reconstruct the enhanced speech. The experimental results demonstrate that the proposed DNN framework with less aggressive Wiener filtering outperforms the competing speech enhancement methods in terms of the speech quality and intelligibility.

A Single-Channel Speech Enhancement Algorithm Combined with Time-Frequency Mask

Aiming at the problem of noise overestimation when the classical single-channel speech enhancement method suppresses noise, single-channel speech enhancement method on the basis of the time-frequency mask is put forward to improve the quality of speech enhancement and separation. First, we estimate the noise and prior signal-to-noise ratio from the noisy speech, calculate the time-frequency mask and finally combine noisy speech synthesis to enhance the speech. Adaptive time-frequency mask combines IBM and IRM to avoid over-suppressing noise. In this paper, noisy speech is processed by an improved time-frequency mask combined with SNR. The algorithm combines the advantages of the single-channel algorithm and time-frequency mask. The experimental results indicate that this method significantly improves the signal-to-noise ratio, is robust, and is easy to use.

A Two-stage Speech Enhancement Method Based on Deep Learning Network

The conventional single-channel speech enhancement algorithms require only a small amount of computation to implement, but have problems with “musical noise” and speech distortion. The learning-based enhancement method achieves a relatively good performance, but it requires a right training data set and a large computational load. This paper proposed a method to combine the two types of algorithms. This method includes two stages. In the first stage, the Berouti spectral subtraction was used to subtract part of noise from the noisy speech. In the second stage, a deep neural network was built to remove residual noise and improve the clarity of speech obtained in the first stage. The experimental results indicate that, training with the same data set, the method proposed in this paper clearly outperforms the baseline model in PESQ and FWSegSNR.

End-to-End Brain-Driven Speech Enhancement in Multi-Talker Conditions

Single-channel speech enhancement algorithms have seen great improvements over the past few years. Despite these improvements, they still lack the efficiency of the auditory system in extracting attended auditory information in the presence of competing speakers. Recently, it has been shown that the attended auditory information can be decoded from the brain activity of the listener. In this paper, we propose two novel end-to-end deep learning methods referred to as the Brain Enhanced Speech Denoiser (BESD) and the U-shaped Brain Enhanced Speech Denoiser (U-BESD) respectively, that take advantage of this fact to denoise a multi-talker speech mixture without considering further background noises or reverberations. We use a Feature-wise Linear Modulation (FiLM) between the brain activity and the sound mixture, to better extract the features of the attended speaker to perform speech enhancement. We show, using electroencephalography (EEG) signals recorded from the listener, that both BESD and U-BESD successfully extract the attended speaker without any prior information about this speaker. Moreover, U-BESD also outperforms a current state-of-the-art approach that also uses brain activity to perform enhancement. The proposed neural network-based methods would thus make great candidates for realistic applications where no prior information about the attended speaker is available, such as hearing aids, cellphones, or noise cancelling headphones.

Smartphone-based single-channel speech enhancement application for hearing aids.

This work presents a single-channel speech enhancement (SE) framework based on the super-Gaussian extension of the joint maximum a posteriori (SGJMAP) estimation rule. The developed SE algorithm is an open-source research smartphone-based application for hearing improvement studies. In this algorithm, the SGJMAP-based estimation for noisy speech mixture is smoothed along the frequency axis by a Mel filter-bank, resulting in a Mel-warped frequency-domain SGJMAP estimation. The impulse response of this Mel-warped estimation is obtained by applying a Mel-warped inverse discrete cosine transform (Mel-IDCT). This helps in filtering out the background noise and enhancing the speech signal. The proposed application is implemented on an iPhone (Apple, Cupertino, CA) to operate in real time and tested with normal-hearing (NH) and hearing-impaired (HI) listeners with different types of hearing aids through wireless connectivity. The objective speech quality and intelligibility test results are used to compare the performance of the proposed algorithm to existing conventional single-channel SE methods. Additionally, test results from NH and HI listeners show substantial improvement in speech recognition with the developed method in simulated real-world noisy conditions at different signal-to-noise ratio levels.

Speech enhancement based on perceptually motivated guided spectrogram filtering

In this paper, a single-channel speech enhancement algorithm is proposed by using guided spectrogram filtering based on masking properties of human auditory system when considering a speech spectrogram as an image. Guided filtering is capable of sharpening details and estimating unwanted textures or background noise from the noisy speech spectrogram. If we consider the noisy spectrogram as a degraded image, we can estimate the spectrogram of the clean speech signal using guided filtering after subtracting noise components. Combined with masking properties of human auditory system, the proposed algorithm adaptively adjusts and reduces the residual noise of the enhanced speech spectrogram according to the corresponding masking threshold. Because the filtering output is a local linear transform of the guidance spectrogram, the local mask window slides can be efficiently implemented via box filter with O(N) computational complexity. Experimental results show that the proposed algorithm can effectively suppress noise in different noisy environments and thus can greatly improve speech quality and speech intelligibility.

Research on Speech Enhancement Based on Deep Neural Network

The traditional single-channel speech enhancement algorithm has many unreasonable assumptions, which limit the performance of the algorithm. The speech enhancement based on deep neural network can effectively eliminate the problems such as “music noise” in the traditional methods, thus achieving better results than the traditional single-channel speech enhancement. However, the DNN-based speech enhancement model does not perform well at low SNR. In order to improve the robustness of the model, we used voice activity detection (VAD) to process the training data to obtain a new VAD-DNN speech enhancement model. We added 100 kinds of noise in the training set to improve the ability of the model to deal with the unseen noise. At the same time, in order to prevent the occurrence of overfitting problem, we introduced dropout technology to process the network, and improved the generalization ability of the model by using noise awareness training (NAT). Through experiments, we found that the speech enhancement model based on VAD-DNN improved PESQ index by 0.02 and STOI index by 0.01 on average under the condition of low SNR.

Real-time speech enhancement algorithm for transient noise suppression

To effectively restrain stationary noise and transient noise, a real-time single-channel speech enhancement algorithm is proposed. First, to evaluate stationary noise, the quantile noise estimation method is used to obtain the spectrum of stationary noise. Then, based on the normalized variance and gravity center of the signal, the transient noise detection method is proposed to modify the spectrum of stationary noise. Next, the speech presence probability is estimated based on the speech features and harmonic analysis. Finally, the optimized-modified log-spectral amplitude (OM-LSA) estimator is adopted for speech enhancement. The experimental noise contains 115 environmental sounds with the SNR of −10 to 10 dB. The experimental results show that the performance of the proposed algorithm is comparable to the OM-LSA algorithm which has good denoising performance, but the real-time performance of the former is much better. Compared with the Webrtc real-time algorithm, under the overall performance of stationary noise and transient noise, the overall speech quality indicators of the improved algorithm increased by 7.5%, 7.8% and 5.0%, respectively. And the short-time objective intelligibility increased by 2.4%, 2.4% and 2.0%, respectively. Even compared with the recurrent neural network(RNN) algorithm, the suppression performance of the transient noise is better. Besides, the real-time experiment base on the hardware platform shows that the runtime of processing a 10 ms frame is 4.3 ms.

Single-channel speech enhancement with correlated spectral components: Limits-potential

In this paper, we investigate single-channel speech enhancement algorithms that operate in the short-time Fourier transform and take into account dependencies w.r.t. frequency. As a result of allowing for inter-frequency dependencies, the minimum mean square error optimal estimates of the short-time Fourier transform expansion coefficients are functions of complex-valued covariance matrices in general. The covariance matrices are not known a priori and have to be estimated from the observed data. This work is dedicated to analyzing how this affects the respective single-channel speech enhancement algorithms. We propose a statistical model that circumvents the need to estimate complex-valued second order statistics and derive a linear multidimensional short-time spectral amplitude estimator that is motivated by these assumptions. Further, we provide empirical evidence for the assumptions that form the basis of this model. We evaluate the potential of taking into account inter-frequency dependencies for single-channel speech enhancement and subsequently compare the estimator resulting from the proposed statistical model to relevant benchmark methods. The results indicate that estimators that consider inter-frequency dependencies are capable of pushing the limits of standard approaches in terms of joint speech quality and intelligibility improvement when the second order statistics are estimated from isolated speech data. The proposed linear multidimensional short-time spectral amplitude estimator preserves this trend in fully blind scenarios.

Supervised single channel dual domains speech enhancement using sparse non-negative matrix factorization

In this paper, we propose a novel single-channel speech enhancement algorithm that applies dual-domain transforms comprising of dual-tree complex wavelet transform (DTCWT) and short-time Fourier transform (STFT) with a sparse non-negative matrix factorization (SNMF). The first domain belongs to the DTCWT, which is utilized on the time domain signals to conquer the weakness of signal distortions brought about by the downsampling of the discrete wavelet packet transform (DWPT) and delivered a set of subband signals. The second domain alludes to the STFT, which is exploited to each subband signal and built a complex spectrogram. At last, we apply the SNMF to the magnitude spectrogram for extracting speech components. In short, the DTCWT decomposes the time-domain noisy signal into a set of subband signals and afterward applied STFT to each subband signal, and we get nonnegative matrices by taking the absolute value of the complex matrix. From this point forward, we apply SNMF to each nonnegative matrix and identify the speech components. Finally, the estimated signal can be achieved through a subband binary ratio mask (SBRM) by applying the inverse STFT (ISTFT) and, subsequently, the inverse DTCWT (IDTCWT). The proposed approach is assessed utilizing the GRID audio-visual and IEEE databases, and diverse kinds of noises such as stationary, non-stationary, and quasi-stationary. The exploratory outcomes demonstrate that the proposed algorithm improved objective speech quality and intelligibility altogether at all considered signal to noise ratios (SNRs), compared to the other seven speech enhancement methods of STFT-SNMF, STFT-SNMFSE, MLD-STFT-SNMF, STFT-GDL, STFT-CJSR, DTCWT-SNMF, and DWPT-STFT-SNMF.

Single Channel Speech Enhancement Algorithm based on BLSTM-DNN Bidirectional Optimized Hybrid Model

The performance of existing speech enhancement algorithms based on deep learning is not ideal in complex noise environment. To improve the problem, a bidirectional optimized hybrid network named BLSTM-DNN is constructed based on bidirectional long-short term memory (BLSTM) network and fully-connected deep neural network (DNN). This structure uses BLSTM to extract high-level information including past and future temporal context of noisy speech. Next, fully-connected DNN fits the high-level information to ideal ratio mask (IRM). Finally, the IRMs estimated by the BLSTM-DNN are used to enhance the noisy speech. Experimental results show that the proposed method can effectively improve the speech quality and intelligibility under unknown noise conditions.

On Improvement of Speech Intelligibility and Quality: A Survey of Unsupervised Single Channel Speech Enhancement Algorithms

On Improvement of Speech Intelligibility and Quality: A Survey of Unsupervised Single Channel Speech Enhancement Algorithms

Real-Time Speech Enhancement Algorithm Based on Attention LSTM

Because traditional single-channel speech enhancement algorithms are sensitive to the environment and perform poorly, a speech enhancement algorithm based on attention-gated long short-term memory (LSTM) is proposed. To simulate human auditory perceptual characteristics, the algorithm divides the frequency band according to the Bark scale. Based on these bands, bark frequency cepstral coefficients (BFCCs), their derivative features and pitch-based features are extracted. Furthermore, considering that different noises have different influence on the clean speech, the attention mechanism is applied to screen out the information less polluted by noise, which is helpful to reconstruct the clean speech. To adaptively reallocate the power ratio of the speech and noise during the construction of the ratio mask, the ideal ratio mask (IRM) with the inter-channel correlation (ICC) is adopted as the learning target. In addition, to improve the performance of the network, the algorithm introduces a multiobjective learning strategy to jointly optimize the networks by using a voice activity detector (VAD). Subjective and objective experiments show that the proposed algorithm outperforms other baseline algorithms. In real-time experiment, the proposed algorithm maintains high real-time performance and fast convergence speed.

Word order detection in English classroom teaching based on improved genetic algorithm of block coding

The performance of the speech recognition system for English classroom teaching is largely affected by the surrounding environment. These interference signals will seriously reduce the quality and intelligibility of the speech signal, thereby greatly reducing the performance of the far-field speech recognition system. Aiming at word order detection in English classroom teaching, this paper proposes an analysis model based on block coding and improved genetic algorithm. Moreover, for DNN-based single-channel speech enhancement algorithms, this paper proposes PDNNs and PLSTMs to solve the problem of serious performance degradation of prototype DNN speech enhancement under low signal-to-noise ratio. This method decomposes the entire enhancement task into multiple subtasks to complete, and the previously completed subtasks will provide prior knowledge for the subsequent subtasks, so that the subsequent subtasks can learn its goals better. In general, the experimental results prove the reliability of the model constructed in this paper.

A review of supervised learning algorithms for single channel speech enhancement

Reducing interfering noise in a noisy speech recording has been a difficult task in many applications related to the voice. From hands-free communication to human–machine interaction, a speech signal of the interest captured by a microphone is always mixed with the interfering noise. The interfering noise appends new frequency components and masks a large portion of the time-varying spectra of the desired speech. This significantly affects our perception of the desired speech when listening to the noisy observations. Therefore, it is extremely desirable and sometimes even crucial to clean the noisy speech signals. This clean-up process is referred to as the speech enhancement (SE). SE aims to improve the speech intelligibility and quality of the voice for the communication. We present a comprehensive review on the supervised single channel speech enhancement (SCSE) algorithms. First, a classification based overview of the supervised SCSE algorithms is provided and the related works is outlined. The recent literature on the SCSE algorithms in supervised perspective is reviewed. Finally, some open research problems are identified that need further research.

Speech intelligibility enhancement for Thai-speaking cochlear implant listeners

Cochlear implant (CI) listeners encounter difficulties in communicating with other persons in noisy listening environments. However, most CI research has been carried out using the English language. In this study, single-channel speech enhancement (SE) strategies as a pre-processing approach for the CI system were investigated in terms of Thai speech intelligibility improvement. Two SE algorithms, namely multi-band spectral subtraction (MBSS) and Weiner filter (WF) algorithms, were evaluated. Speech signals consisting of monosyllabic and bisyllabic Thai words were degraded by speech-shaped noise and babble noise at SNR levels of 0, 5, and 10 dB. Then the noisy words were enhanced using SE algorithms. The enhanced words were fed into the CI system to synthesize vocoded speech. The vocoded speech was presented to twenty normal-hearing listeners. The results indicated that speech intelligibility was marginally improved by the MBSS algorithm and significantly improved by the WF algorithm in some conditions. The enhanced bisyllabic words showed a noticeably higher intelligibility improvement than the enhanced monosyllabic words in all conditions, particularly in speech-shaped noise. Such outcomes may be beneficial to Thai-speaking CI listeners.

Phase-Sensitive Joint Learning Algorithms for Deep Learning-Based Speech Enhancement

This letter presents a phase-sensitive joint learning algorithm for single-channel speech enhancement. Although a deep learning framework that estimates the time-frequency (T-F) domain ideal ratio masks demonstrates a strong performance, it is limited in the sense that the enhancement process is performed only in the magnitude domain, while the phase spectra remain unchanged. Thus, recent studies have been conducted to involve phase spectra in speech enhancement systems. A phase-sensitive mask (PSM) is a T-F mask that implicitly represents phase-related information. However, since the PSM has an unbounded value, the networks are trained to target its truncated values rather than directly estimating it. To effectively train the PSM, we first approximate it to have a bounded dynamic range under the assumption that speech and noise are uncorrelated. We then propose a joint learning algorithm that trains the approximated value through its parameterized variables in order to minimize the inevitable error caused by the truncation process. Specifically, we design a network that explicitly targets three parameterized variables: 1) speech magnitude spectra; 2) noise magnitude spectra; and 3) phase difference of clean to noisy spectra. To further improve the performance, we also investigate how the dynamic range of magnitude spectra controlled by a warping function affects the final performance in joint learning algorithms. Finally, we examined how the proposed additional constraint that preserves the sum of the estimated speech and noise power spectra affects the overall system performance. The experimental results show that the proposed learning algorithm outperforms the conventional learning algorithm with the truncated phase-sensitive approximation.