Short-time Spectral Amplitude Estimator Research Articles

Comparative Study on Noisy Speech Preprocessing Algorithms

Speech Enhancement and noise reduction have wide applications in speech processing. They are often employed as pre-processing stage in various applications. The work to be presented in this paper is denoising a single-channel speech signal at the presence of a highly non-stationary background noise in order to improve the perceptible quality and intelligibility of the speech. Real world noise is mostly highly non-stationary and does not affect the speech signal uniformly over the spectrum. This paper investigates various Discrete Fourier Transform-based algorithms as single-channel pre-processing techniques consisting of: Spectral Subtraction using over-subtraction and spectral floor, Multi-Band Spectral Subtraction (MBSS), Wiener Filter, MMSE of Short-Time Spectral Amplitude (MMSE-STSA) estimator with, and without using SPU modifier, MMSE Log-Spectral Amplitude Estimator with, and without using SPU modifier, Optimally-Modified Log-Spectral Amplitude estimator (OM-LSA). The processed speeches from these algorithms are compared at the same set of conditions using visual examinations of signals in the time domain and the spectrograms, and also the objective and subjective tests for quality and perceptual evaluation. All the implemented algorithms provide considerable, different degrees of flexibility and control on noise elimination levels that reduces artifacts in the enhanced speech, resulting in the improved quality, and intelligibility.

Log-Spectral Amplitude and Spectral Polarity Estimation in Short-Time Discrete Cosine Transform Domain

Single-channel speech enhancement based on short-time spectral amplitude (STSA) estimation often uses the unmodified phase spectrum for speech re-synthesis, thereby introducing undesired artifacts to the enhanced speech. Using discrete Cosine transform (DCT) instead of discrete Fourier transform (DFT) reduces the effects of such issues because the consequences of using noisy DCT polarities for speech re-synthesis are less severe than using the noisy DFT phases. Although DFT-based STSA estimators have been adequately studied in the past, such estimators have not sufficiently been developed for the DCT domain. This study aims to demonstrate the superiority of DCT representation in STSA estimation-based speech enhancement. To achieve this, we first derive the DCT-based STSA estimator which minimizes the mean squared error (MSE) of the log-spectral amplitudes (LSA). We then propose a novel DCT polarity estimator to be used in combination with the STSA estimator. The clean speech DCT coefficients are modeled by a Gaussian or a Laplace density and the noise DCT coefficients are modeled by a Gaussian density. To assess the enhanced speech, objective and subjective quality measures are employed. Results show that the new estimators performed better and are widely preferred by listeners over the corresponding DFT-based estimators. Moreover, the proposed STSA estimators can be expressed in the closed-form, whereas the DFT-based estimator with super-Gaussian speech prior has no closed-form solutions.

Research on Speech Enhancement Algorithm of Multiresolution Cochleagram Based on Skip Connection Deep Neural Network

The speech enhancement effect of traditional deep learning algorithms is not ideal under low signal-to-noise ratios (SNR). Skip connections-deep neural network (Skip-DNN) improves the traditional deep neural network (DNN) by adding skip connections between each layer of the neural network to solve the degradation problem of DNN. In this paper, the Multiresolution Cochleagram (MRCG) features in the gammachirp transform domain are denoised to obtain the improved MRCG (I-MRCG). The noise reduction method adopts the Minimum Mean-Square Error Short-Time Spectral Amplitude Estimator (MMSE-STSA) and takes I-MRCG as the input feature and Skip-DNN as the training network to improve the speech enhancement effect of the model. This paper also proposes an improved source-to-distortion ratio (SDR) loss function. When the loss function uses the improved SDR, it will improve the performance of Skip-DNN speech enhancement model. The experiments in this paper are performed on the Edinburgh dataset. When using I-MRCG as the input feature of Skip-DNN, the average perceptual evaluation of speech quality (PESQ) is 2.9137, and the average short-time objective intelligibility (STOI) is 0.8515. Compared with MRCG as Skip-DNN input features, the improvements are 0.91% and 0.71%, respectively. When the improved SDR is used as the loss function of the speech model, the average PESQ is 2.9699 and the average STOI is 0.8547. Compared with other loss functions, the improved SDR has a better enhancement effect when used as the loss function of the speech enhancement model.

Spectral distortion level resulting in a just-noticeable difference between an a priori signal-to-noise ratio estimate and its instantaneous case.

Minimum mean-square error (MMSE) approaches to speech enhancement are widely used in the literature. The quality of enhanced speech produced by an MMSE approach is directly impacted by the accuracy of the employed a priori signal-to-noise ratio (SNR) estimator. In this paper, the a priori SNR estimate spectral distortion (SD) level that results in a just-noticeable difference (JND) in the perceived quality of MMSE approach enhanced speech is found. The JND SD level is indicative of the accuracy that an a priori SNR estimator must exceed to have no impact on the perceived quality of MMSE approach enhanced speech. To measure the JND SD level, listening tests are conducted across five SNR levels, five noise sources, and two MMSE approaches [the MMSE short-time spectral amplitude (MMSE-STSA) estimator and the Wiener filter]. A statistical analysis of the results indicates that the JND SD level increases with the SNR level, is higher for the MMSE-STSA estimator, and is not impacted by the type of background noise. Following the literature, a significant improvement in a priori SNR estimation accuracy is required to reach the JND SD level.

Musical-noise-free noise reduction by using biased harmonic regeneration and considering relationship between a priori SNR and sound quality

This paper focuses on two representative single-microphone noise reduction problems: speech distortion and musical noise. Many noise reduction methods have been proposed for each problem. Harmonic regeneration noise reduction (HRNR) was introduced for the improvements of speech distortion and the a priori signal-to-noise ratio (SNR) estimator. HRNR involves using a unique signal to regenerate harmonics, which had been eliminated. Musical-noise-free noise reduction based on the minimum-mean square error short-time spectral amplitude estimator (musical-noise-free MMSE-STSA estimator) has also been proposed. This method can suppress a noisy signal without generating musical noise by introducing a bias into the classical a priori SNR estimator. We propose a noise reduction method for addressing these problems simultaneously by improving the classical a priori SNR estimator. We investigated the behavior of the internal parameters for the proposed and conventional methods with regard to speech quality and show the effectiveness of the proposed method in terms of speech distortion and musical noise. We consider and discuss the relationship between the estimation accuracy of an a priori SNR and speech quality. Specifically, we consider the factors to improve speech quality in terms of biasing.

Single-channel speech enhancement with correlated spectral components: Limits-potential

In this paper, we investigate single-channel speech enhancement algorithms that operate in the short-time Fourier transform and take into account dependencies w.r.t. frequency. As a result of allowing for inter-frequency dependencies, the minimum mean square error optimal estimates of the short-time Fourier transform expansion coefficients are functions of complex-valued covariance matrices in general. The covariance matrices are not known a priori and have to be estimated from the observed data. This work is dedicated to analyzing how this affects the respective single-channel speech enhancement algorithms. We propose a statistical model that circumvents the need to estimate complex-valued second order statistics and derive a linear multidimensional short-time spectral amplitude estimator that is motivated by these assumptions. Further, we provide empirical evidence for the assumptions that form the basis of this model. We evaluate the potential of taking into account inter-frequency dependencies for single-channel speech enhancement and subsequently compare the estimator resulting from the proposed statistical model to relevant benchmark methods. The results indicate that estimators that consider inter-frequency dependencies are capable of pushing the limits of standard approaches in terms of joint speech quality and intelligibility improvement when the second order statistics are estimated from isolated speech data. The proposed linear multidimensional short-time spectral amplitude estimator preserves this trend in fully blind scenarios.

Deep learning for minimum mean-square error approaches to speech enhancement

Recently, the focus of speech enhancement research has shifted from minimum mean-square error (MMSE) approaches, like the MMSE short-time spectral amplitude (MMSE-STSA) estimator, to state-of-the-art masking- and mapping-based deep learning approaches. We aim to bridge the gap between these two differing speech enhancement approaches. Deep learning methods for MMSE approaches are investigated in this work, with the objective of producing intelligible enhanced speech at a high quality. Since the speech enhancement performance of an MMSE approach improves with the accuracy of the used a priori signal-to-noise ratio (SNR) estimator, a residual long short-term memory (ResLSTM) network is utilised here to accurately estimate the a priori SNR. MMSE approaches utilising the ResLSTM a priori SNR estimator are evaluated using subjective and objective measures of speech quality and intelligibility. The tested conditions include real-world non-stationary and coloured noise sources at multiple SNR levels. MMSE approaches utilising the proposed a priori SNR estimator are able to achieve higher enhanced speech quality and intelligibility scores than recent masking- and mapping-based deep learning approaches. The results presented in this work show that the performance of an MMSE approach to speech enhancement significantly increases when utilising deep learning.Availability: The proposed a priori SNR estimator is available at: https://github.com/anicolson/DeepXi.

DNN-Supported Speech Enhancement With Cepstral Estimation of Both Excitation and Envelope

In this paper, we propose and compare various techniques for the estimation of clean spectral envelopes in noisy conditions. The source-filter model of human speech production is employed in combination with a hidden Markov model and/or a deep neural network approach to estimate clean envelope-representing coefficients in the cepstral domain. The cepstral estimators for speech spectral envelope-based noise reduction are both evaluated alone and also in combination with the recently introduced cepstral excitation manipulation CEM technique for a priori SNR estimation in a noise reduction framework. Relative to the classical MMSE short time spectral amplitude estimator, we obtain more than 2 dB higher noise attenuation, and relative to our recent CEM technique still 0.5 dB more, in both cases maintaining the quality of the speech component and obtaining considerable SNR improvement.

Spectral Restoration Based Speech Enhancement for Robust Speaker Identification

Spectral restoration based speech enhancement algorithms are used to enhance quality of noise masked speech for robust speaker identification. In presence of background noise, the performance of speaker identification systems can be severely deteriorated. The present study employed and evaluated the Minimum Mean-SquareError Short-Time Spectral Amplitude Estimators with modified a priori SNR estimate prior to speaker identification to improve performance of the speaker identification systems in presence of background noise. For speaker identification, Mel Frequency Cepstral coefficient and Vector Quantization is used to extract the speech features and to model the extracted features respectively. The experimental results showed significant improvement in speaker identification rates when spectral restoration based speech enhancement algorithms are used as a pre-processing step. The identification rates are found to be higher after employing the speech enhancement algorithms.

Automatic and Efficient Denoising of Bioacoustics Recordings Using MMSE STSA

Automatic recording and analysis of bird calls is becoming an important way to understand changes in bird populations and assess environmental health. An issue currently proving problematic with the automatic analysis of bird recordings is interference from noise that can mask vocalizations of interest. As such, noise reduction can greatly increase the accuracy of automatic analyses and reduce processing work for subsequent steps in bioacoustics analyses. However, only limited work has been done in the context of bird recordings. Most semiautomatic methods either manually apply sound enhancement methods available in audio processing systems such as SoX and Audacity or apply preliminary filters such as low- and high-pass filters. These methods are insufficient both in terms of how generically they can be applied and their integration with automatic systems that need to process large amounts of data. Some other work applied more sophisticated denoising methods or combinations of different methods such as minimum mean square error short-time spectral amplitude estimator (MMSE STSA) and spectral subtraction for other species such as anurans. However, their effectiveness is not tested on bird recordings. In this paper, we analyze the applicability of the MMSE STSA algorithm to remove noise from environmental recordings containing bird sounds, particularly focusing on its quality and processing time. The experimental evaluation using real data clearly shows that MMSE STSA can reduce noise with similar effectiveness [using objective metrics such as predicted signal quality (SIG)] to a previously recommended wavelet-transform-based denoising technique while executing between approximately 5–300 times faster depending on the audio files tested.

Identification of modal parameters from noisy transient response signals

Summary In the process of impact testing of large-scale mechanical equipment, the measured forced response signals are often polluted by strong background noise. The forced response signal has a low signal-to-noise ratio, and this makes it difficult to accurately estimate the modal parameters. To solve this problem, the mean averaging of repeatedly measured frequency response function estimates is often employed in practical applications. However, a large number of impact tests are not practical for the modal testing of large-scale mechanical equipment. The primary objective of this paper is to reduce the averaging operation and improve the accuracy of the modal identification by using a noise removal technique. A hybrid denoising method is proposed by combining the Wiener and improved minimum mean-square-error short-time spectral amplitude estimators. The proposed method can effectively remove both stationary and highly nonstationary noise while preserving the important features of the true forced response signals. The simulation results show that the proposed noise removal technique improves the accuracy of the estimated modal parameters using only one impulse response signal. The experimental results show that the proposed method can accurately identify a natural frequency that is very close to a strong interference frequency in the modal test of a 600-MW generator casing.

Automatic anuran identification using noise removal and audio activity detection

The use of bioacoustics to identify animal species has huge potential for use in biology and conservation research. Fields that could be greatly enhanced by the use of bioacoustical techniques include the study of animal behavior, soundscape ecology, species diversity assessments, and long-term monitoring - for example to further our understanding of the conservation status of numerous species and their vulnerability to different threats. In this study, we focus primarily, but not exclusively, on the identification of anuran vocalizations. We chose anurans both because they tend to be quite vocal and because they are considered indicators of environmental health. We present a system for semi-automated segmentation of anuran calls, based on sound enhancement method that uses Minimum-Mean Square Error (MMSE) Short-Time Spectral Amplitude (STSA) estimator and noise suppression algorithm using Spectral Subtraction (SS), and an automated classification system for 17 anuran species based on Mel-Frequency Cepstrum Coefficients (MFCC) and the Gaussian Mixture Model (GMM). To our knowledge this is the first study that applies this combination of methods for animal identification. This technique achieves accuracies of between 96.1% and 100% per species. Experimental results show that the semi-automated segmentation technique performs better than automated segmentation systems, improving the average success rate to 98.61%. The effectiveness of the proposed anuran identification system in natural environment is thus verified. This work presents a first approach to future tools which can signify a significant advance in the procedures to analysis in a semiautomatic or even in an automatic way to analysis indicators of environmental health based on expert and intelligent systems.

Fast distributed multichannel speech enhancement using novel frequency domain estimators of magnitude-squared spectrum

This paper proposes two novel frequency domain estimators for fast distributed multichannel speech enhancement in background of white and colored noise. The proposed two frequency domain estimators are maximum a posterior (MAP) and minimum mean square error (MMSE) estimators, respectively. They significantly generalize two single channel optimal frequency domain estimators of magnitude-squared spectrum. Compared with the optimal multichannel frequency domain estimator generalizing the single channel short-time spectral amplitude and log-spectral amplitude estimators, the proposed two frequency domain estimators have a very low computational cost. Computed results show that the proposed two estimators reduce both colored background noise and speech distortion. Furthermore, the proposed two multichannel algorithms have a much faster computational speed than conventional multichannel algorithms for distributed multichannel speech enhancement.

Minimum Mean-Square Error Estimation of Mel-Frequency Cepstral Features–A Theoretically Consistent Approach

In this work, we consider the problem of feature enhancement for noise-robust automatic speech recognition (ASR). We propose a method for minimum mean-square error (MMSE) estimation of mel-frequency cepstral features, which is based on a minimum number of well-established, theoretically consistent statistical assumptions. More specifically, the method belongs to the class of methods relying on the statistical framework proposed in Ephraim and Malah's original work (“Speech enhancement using a minimum mean-square error short-time spectral amplitude estimator,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-32, no. 6, 1984). The method is general in that it allows MMSE estimation of mel-frequency cepstral coefficients (MFCC's), cepstral-mean subtracted (CMS-) MFCC's, autoregressive-moving-average (ARMA)-filtered CMS-MFCC's, velocity, and acceleration coefficients. In addition, the method is easily modified to take into account other compressive non-linearities than the logarithm traditionally used for MFCC computation. In terms of MFCC estimation performance, as measured by MFCC mean-square error, the proposed method shows performance which is identical to or better than other state-of-the-art methods. In terms of ASR performance, no statistical difference could be found between the proposed method and the state-of-the-art methods. We conclude that existing state-of-the-art MFCC feature enhancement algorithms within this class of algorithms, while theoretically suboptimal or based on theoretically inconsistent assumptions, perform close to optimally in the MMSE sense.

Speech enhancement based on β-order MMSE estimation of Short Time Spectral Amplitude and Laplacian speech modeling

This paper addresses the problem of speech enhancement employing the Minimum Mean-Square Error (MMSE) of β-order Short Time Spectral Amplitude (STSA). The motivation has been to take advantages of both Laplacian speech modeling and β-order cost function in MMSE estimation of clean speech. We present an analytical solution for β-order MMSE STSA estimator assuming Laplacian prior for the real and imaginary parts of the Discrete Fourier Transform (DFT) coefficients of (clean) speech. We also assume Gaussian distribution for the real and imaginary parts of the DFT coefficients of the noise. The analytical solution, named β-order LapMMSE, does not have a closed form and is highly non-linear and computationally complex. Using some approximations for the joint probability density function and the Bessel function, we also present an improved closed-form version of the estimator (called β-order ImpLapMMSE). The value of β is adapted as a function of frame Signal to Noise Ratio (SNR). We have compared the performance of the proposed estimator with the state-of-the-art estimators that assume either Gaussian or Laplacian probability density functions for the real and imaginary parts of the DFT coefficients of clean speech. To this end, the input noisy signal and the outputs of MMSE STSA, β-order STSA, and ImpLapMMSE estimators have been compared with the output of the proposed estimator. Our comparative evaluations in terms of Segmental SNR (SegSNR), Perceptual Evaluation of Speech Quality (PESQ), and Log-Likelihood Ratio (LLR) distance demonstrate the superior performance of the proposed β-order ImpLapMMSE estimator.

Use of baseband phase structure to improve the performance of current speech enhancement algorithms

In this study we propose a noise estimation technique based on spectral sparsity, detected by using the harmonic property of voiced segments of the speech. We estimate the frame to frame phase difference for clean speech, in the baseband Short Time Fourier Transform (STFT) domain, from corrupted speech. This estimated frame-to-frame phase difference is used as a means of detecting the noise and harmonic dominant frequency bins in voiced frames. In the unvoiced frames noise is estimated using a Voice Activity Detector (VAD). Using this approach gives better noise estimation for the highly non-stationary noises like babble, restaurant and subway noise. The proposed noise estimation algorithm along with the phase difference as an additional prior is used to extend the standard spectral subtraction algorithm. Estimated baseband phase difference is used to prevent over-suppression of harmonic dominant speech in the voiced frames by adjusting the over-attenuation factor of the spectral subtraction algorithm. We verify the effectiveness of the proposed noise estimation technique when used with the log-Minimum Mean Squared Error Short Time Spectral Amplitude Estimator (log-MMSE STSA) speech enhancement algorithm. Finally, spectral subtraction and log-MMSE STSA are combined to achieve better noise suppression with minimum musical noise and speech distortion. This combination results in further improvement of speech quality.

Speech Prior Estimation for Generalized Minimum Mean-Square Error Short-Time Spectral Amplitude Estimator

SUMMARY In this paper, we introduce a generalized minimum meansquare error short-time spectral amplitude estimator with a new prior estimation of the speech probability density function based on momentcumulant transformation. From the objective and subjective evaluation experiments, we show the improved noise reduction performance of the proposed method.

A Unified Framework for Designing Optimal STSA Estimators Assuming Maximum Likelihood Phase Equivalence of Speech and Noise

In this paper, we present a stochastic framework for designing optimal short-time spectral amplitude (STSA) estimators for speech enhancement assuming phase equivalence of speech and noise. By assuming additive superposition of speech and noise, which is implied by the maximum-likelihood (ML) phase estimate, we effectively project the optimal spectral amplitude estimation problem onto a 1-D subspace of the complex spectral plane, thus simplifying the problem formulation. Assuming generalized Gamma distributions (GGDs) for a priori distributions of both speech and noise STSAs, we derive separate families of novel estimators according to either the maximum-likelihood (ML), the minimum mean-square error (MMSE), or the maximum a posteriori (MAP) criterion. The use of GGDs allows optimal estimators to be determined in a generalized form, so that particular solutions can be obtained by substituting statistical shape parameters corresponding to expected speech and noise priors. It is interesting to note that several of the proposed estimators exhibit strong similarities to well-known STSA solutions. For example, the magnitude spectral subtracter (MSS) and Wiener filter (WF) are obtained for specific cases of GGD shape parameters. Quantitative analysis of a selected subset of the proposed estimators shows improvement over the traditional log-spectral MMSE estimator of Ephraim and Malah, in terms of segmental signal-to-noise ratio (SNR) and the COSH distance measure, when applied to the Noizeus database. Although single-channel speech enhancement is offered as an illustrative example, the theory presented here could be applicable to other signals, such as music and images.

Distributed multichannel speech enhancement with minimum mean-square error short-time spectral amplitude, log-spectral amplitude, and spectral phase estimation

In this paper, the authors present optimal multichannel frequency domain estimators for minimum mean-square error (MMSE) short-time spectral amplitude (STSA), log-spectral amplitude (LSA), and spectral phase estimation in a widely distributed microphone configuration. The estimators utilize Rayleigh and Gaussian statistical models for the speech prior and noise likelihood with a diffuse noise field for the surrounding environment. Based on the Signal-to-Noise Ratio (SNR) and Segmental Signal-to-Noise Ratio (SSNR) along with the Log-Likelihood Ratio (LLR) and Perceptual Evaluation of Speech Quality (PESQ) as objective metrics, the multichannel LSA estimator decreases background noise and speech distortion and increases speech quality compared to the baseline single channel STSA and LSA estimators, where the optimal multichannel spectral phase estimator serves as a significant quantity to the improvements, and demonstrates robustness due to time alignment and attenuation factor estimation. Overall, the optimal distributed microphone spectral estimators show strong results in noisy environments with application to many consumer, industrial, and military products.

Multidimensional STSA Estimators for Speech Enhancement With Correlated Spectral Components

Speech enhancement algorithms are used to remove background noise in a speech signal. In Bayesian short-time spectral amplitude (STSA) estimation for single-channel speech enhancement, the spectral components are traditionally assumed uncorrelated. However, this assumption is inexact since some correlation is present in practice. In this paper, we investigate a multidimensional Bayesian STSA estimator that assumes correlated spectral components. Since the closed-form solution of this optimum estimator is not readily available, we alternatively derive closed-form expressions for an upper and a lower bound on the desired estimator. Using these bounds, we propose a new family of speech enhancement estimators that are characterized by a scalar parameter 0 ≤ γ ≤ 1, with γ = 0 corresponding to the lower bound and γ = 1 to the upper bound. An appropriate estimator for the correlation matrix of the clean speech is further derived. Evaluation results from both objective and subjective speech quality measures show that at moderate to high SNR values, where spectral correlation of speech is most noticeable, the proposed estimators can achieve significant improvements over the traditional STSA and Wiener filter estimators.