Speech Power Spectral Density Research Articles

An Analysis of Traditional Noise Power Spectral Density Estimators Based on the Gaussian Stochastic Volatility Model

Many single- and multi-channel speech enhancement techniques, old and new, rely in one way or another on estimates of the noise power spectral density (PSD). For example, the classical Wiener filter requires that either the speech or noise PSD be estimated. Typically, the noise PSD is estimated, as it is often easier to model and estimate than the speech. As a result, much attention has been paid to this important problem over the past couple of decades, with important scientific milestones being the minimum statistics (MS), the minima controlled recursive averaging (IMCRA), and the minimum mean squared (MMSE) estimators. Despite leading to major progress, these estimators are rather ad hoc, making them difficult to tune and improve in a systematic manner. In this article, we analyse some of the common heuristics employed in such noise PSD estimators to put them on firmer mathematical ground. More specifically, we use the Gaussian stochastic volatility model and show that the MMSE noise PSD estimator can be interpreted as a special case thereof. Moreover, we analyze the related problem of speech presence probability (SPP) estimation and show that the SPP estimation performed in the MMSE noise PSD estimator can be interpreted as an SNR estimator in the context of the Gaussian stochastic volatility model.

Improved Speech Spatial Covariance Matrix Estimation for Online Multi-Microphone Speech Enhancement.

Online multi-microphone speech enhancement aims to extract target speech from multiple noisy inputs by exploiting the spatial information as well as the spectro-temporal characteristics with low latency. Acoustic parameters such as the acoustic transfer function and speech and noise spatial covariance matrices (SCMs) should be estimated in a causal manner to enable the online estimation of the clean speech spectra. In this paper, we propose an improved estimator for the speech SCM, which can be parameterized with the speech power spectral density (PSD) and relative transfer function (RTF). Specifically, we adopt the temporal cepstrum smoothing (TCS) scheme to estimate the speech PSD, which is conventionally estimated with temporal smoothing. Furthermore, we propose a novel RTF estimator based on a time difference of arrival (TDoA) estimate obtained by the cross-correlation method. Furthermore, we propose refining the initial estimate of speech SCM by utilizing the estimates for the clean speech spectrum and clean speech power spectrum. The proposed approach showed superior performance in terms of the perceptual evaluation of speech quality (PESQ) scores, extended short-time objective intelligibility (eSTOI), and scale-invariant signal-to-distortion ratio (SISDR) in our experiments on the CHiME-4 database.

Improved Speech Enhancement Considering Speech PSD Uncertainty

Speech enhancement based on statistical models has been studied for several decades. Recently, the speech enhancement adopting a speech power spectral density (PSD) uncertainty model has been proposed. This approach distinguishes the true speech PSD from its estimate and considers both as random variables. It incorporates a prior distribution of speech spectra and speech PSD estimators to derive the PSD uncertainty-aware counterpart to conventional clean speech estimators, which results in performance improvement. However, the speech PSD uncertainty model has not yet been adopted for parameter estimations such as <inline-formula><tex-math notation="LaTeX"><?TeX $\mathit{a posteriori}$?></tex-math></inline-formula> speech presence probability, noise PSD, and speech power spectra estimations in the speech enhancement framework. In this paper, we incorporate the speech PSD uncertainty model to all the components of the statistical model-based speech enhancement framework by deriving PSD uncertainty-aware counterparts to conventional parameter estimators. Specifically, we derive the <inline-formula><tex-math notation="LaTeX"><?TeX $\mathit{a posteriori}$?></tex-math></inline-formula> speech presence probability (SPP) where the likelihood function for each hypothesis is based on the speech PSD uncertainty. With this <inline-formula><tex-math notation="LaTeX"><?TeX $\mathit{a posteriori}$?></tex-math></inline-formula> SPP, a novel SPP-based noise PSD estimator is derived. Also, we derive the minimum mean-square error (MMSE) estimator for the power spectrum of the clean speech in the current frame under speech PSD uncertainty which is exploited to refine the speech PSD estimator. Finally, the refined speech PSD estimator is incorporated into the spectral gain function based on the speech PSD uncertainty model. The proposed approach showed improved noise PSD estimation performance in terms of the averaged logarithmic error distance, and improved speech enhancement performance in terms of the noise reduction, segmental signal-to-noise ratio, perceptual evaluation of speech quality (PESQ) scores and short-time objective intelligibility in our experiments. It also exhibited comparable performance with a real-time deep learning-based speech enhancement system in terms of the PESQ scores and composite measures for the VoiceBank-DEMAND dataset.

Differentiable Measures for Speech Spectral Modeling

Autoregressive models for the envelope of speech power spectral densities (PSDs) are refined by the self-supervised spectral learning machine (S3LM) provided with differentiable spectral objective functions, including the Itakura-Saito divergence (ISD), the Kullback-Leibler divergence (KLD), the reverse KLD (RKLD) and the log spectral distortion (LSD), which display more significant results. However, in order to assess the models more perceptually, a method is proposed based upon perturbations around perfect reconstruction analysis-synthesis configurations. In the cross-excitation analysis-synthesis assessment (CEASA) method, the residual signals generated by analysis filters of the spectral models are injected as excitation into the synthesis filters derived from the same and other models in order to be evaluated by the perceptual evaluation of speech quality (PESQ) and Itakura divergence (ID), which are averaged over a set of models obtained using the objective functions mentioned above. The results lead to a superior performance when the RKLD is used as the loss function for the estimation of the spectral models with the ISD ranking close behind. The focus of these divergences on the spectral peaks is argued and pointed as the most important factor for this behavior. Specifically, using the PESQ scores obtained with CEASA, the RKLD loss is found to improve the performance by 1.0%, 4.0% and 19.3% with respect to the open-loop analysis, the KLD and the LSD models, respectively, while the corresponding improvements for the ISD loss are 0.1%, 3.0% and 18.2%, and the RKLD models excel the ISD models by 1.0% on average. Even though the spectral measures alone are not able to unequivocally distinguish the better of the two, CEASA is shown to have enough sensitivity to distinguish their performances. In summary, the learning machine S3LM fits models for the short-term spectral envelope of speech and, for the evaluation of its performance under several differentiable loss functions, the CEASA assessment tool has been developed. In addition, CEASA may be used for other assessments connected with speech analysis and synthesis.

Autoregressive Moving Average Jointly-Diagonalizable Spatial Covariance Analysis for Joint Source Separation and Dereverberation

This paper describes a computationally-efficient statistical approach to joint (semi-)blind source separation and dereverberation for multichannel noisy reverberant mixture signals. A standard approach to source separation is to formulate a generative model of a multichannel mixture spectrogram that consists of source and spatial models representing the time-frequency power spectral densities (PSDs) and spatial covariance matrices (SCMs) of source images, respectively, and find the maximum-likelihood estimates of these parameters. A state-of-the-art blind source separation method in this thread of research is fast multichannel nonnegative matrix factorization (FastMNMF) based on the low-rank PSDs and jointly-diagonalizable full-rank SCMs. To perform mutually-dependent separation and dereverberation jointly, in this paper we integrate both moving average (MA) and autoregressive (AR) models that represent the early reflections and late reverberations of sources, respectively, into the FastMNMF formalism. Using a pretrained deep generative model of speech PSDs as a source model, we realize semi-blind joint speech separation and dereverberation. We derive an iterative optimization algorithm based on iterative projection or iterative source steering for jointly and efficiently updating the AR parameters and the SCMs. Our experimental results showed the superiority of the proposed ARMA extension over its AR- or MA-ablated version in a speech separation and/or dereverberation task.

ML Estimation and CRBs for Reverberation, Speech, and Noise PSDs in Rank-Deficient Noise Field

Speech communication systems are prone to performance degradation in reverberant and noisy acoustic environments. Dereverberation and noise reduction algorithms typically require several model parameters, e.g. the speech, reverberation and noise power spectral densities (PSDs). A commonly used assumption is that the noise PSD matrix is known. However, in practical acoustic scenarios, the noise PSD matrix is unknown and should be estimated along with the speech and reverberation PSDs. In this article, we consider the case of rank-deficient noise PSD matrix, which arises when the noise signal consists of multiple directional noise sources, whose number is less than the number of microphones. We derive two closed-form maximum likelihood estimators (MLEs). The first is a non-blocking-based estimator which jointly estimates the speech, reverberation and noise PSDs, and the second is a blocking-based estimator, which first blocks the speech signal and then jointly estimates the reverberation and noise PSDs. Both estimators are analytically compared and analyzed, and mean square errors (MSEs) expressions are derived. Furthermore, Cramer-Rao Bounds (CRBs) on the estimated PSDs are derived. The proposed estimators are examined using both simulation and real reverberant and noisy signals, demonstrating the advantage of the proposed method compared to competing estimators.

Spatial Coherence-Aware Multi-Channel Wind Noise Reduction

Outdoor recording is particularly challenging in the presence of wind, which induces highly non-stationary noise in the microphone signals. To enhance a desired signal, e.g., speech, a dedicated noise reduction processing is required. The reduction is usually performed by estimating an unknown set of parameters, e.g., the noise and the speech power spectral densities (PSDs). In contrast to the commonly used assumption of uncorrelated wind noise in multi-channel recordings, we assume the spatial correlation of wind noise contributions to be non-zero at low frequencies when closely-spaced microphones are employed. In our earlier work, we assumed that the spatial coherence was known, i.e., given by a model which depends on the free-field speed and direction of the air stream. As these quantities are unknown in practice, we propose in this work a method to recursively estimate the spatial coherence matrix based on the microphone observations. In addition, we prove the equivalence of two recently developed noise PSD estimation methods when uncorrelated wind noise is assumed, and we propose an approximation of both estimators which is independent of the propagation vector of the speech source at sufficiently low frequency and for a small inter-microphone distance. An evaluation in terms of improvements in speech quality, signal-to-noise ratio and intelligibility is carried out using both simulated and measured wind noise samples.

Scoring-Based ML Estimation and CRBs for Reverberation, Speech, and Noise PSDs in a Spatially Homogeneous Noise Field

Hands-free speech systems are subject to performance degradation due to reverberation and noise. Common methods for enhancing reverberant and noisy speech require the knowledge of the speech, reverberation and noise power spectral densities (PSDs). Most literature on this topic assumes that the noise power spectral density (PSD) matrix is known. However, in many practical acoustic scenarios, the noise PSD is unknown and should be estimated along with the speech and the reverberation PSDs. In this article, the noise is modeled as a spatially homogeneous sound field, with an unknown time-varying PSD multiplied by a known time-invariant spatial coherence matrix. We derive two maximum likelihood estimators (MLEs) for the various PSDs, including the noise: The first is a non-blocking-based estimator, that jointly estimates the PSDs of the speech, reverberation and noise components. The second MLE is a blocking-based estimator, that blocks the speech signal and estimates the reverberation and noise PSDs. Since a closed-form solution does not exist, both estimators iteratively maximize the likelihood using the Fisher scoring method. In order to compare both methods, the corresponding Cramer-Rao Bounds (CRBs) are derived. For both the reverberation and the noise PSDs, it is shown that the non-blocking-based CRB is lower than the blocking-based CRB. Performance evaluation using both simulated and real reverberant and noisy signals, shows that the proposed estimators outperform competing estimators, and greatly reduce the effect of reverberation and noise.

Bayesian MMSE Filtering of Noisy Speech by SNR Marginalization With Global PSD Priors

MMSE filtering of speech with additive noise and latent speech power-spectral density (PSD) is addressed. This problem is strong in single-channel speech enhancement and restricts the utility of stationary Wiener filters or other statistical estimators based on PSDs. The issue typically manifests itself in residual noise after filtering, despite the availability of the noise PSD. Our paper therefore incorporates the latent speech PSD state via marginalization into the MMSE estimation framework of complex speech spectral amplitudes. The hence involved joint posterior distribution of the complex speech amplitude and speech PSD, conditioned on just the noisy observations, is then resolved in the Bayesian sense into a speech and a speech-PSD posterior. The latter is expressed via the local data likelihood and a hyper-prior of the local speech PSD or a-priori SNR—i.e., a global distribution across the entire speech signal. Marginalization, in this way, turns into expectation over a latent Wiener filter, such that explicit estimation of local a-priori SNR is eliminated. The local input data in the form of the a-posteriori SNR and the global SNR value as a descriptor of the overall speech-in-noise condition turns out sufficient to control our resulting MMSE spectral gain function, and, potentially, can be provided much easier than the latent and time-varying a-priori SNR. An improved balance of residual noise and speech quality in the enhancement of noisy speech is demonstrated by objective experimental evaluation.

On Speech Enhancement Under PSD Uncertainty

Many well-known and frequently employed Bayesian clean speech estimators have been derived under the assumption that the true power spectral densities (PSDs) of speech and noise are exactly known. In practice, however, only power spectral density (PSD) estimates are available. Simply neglecting PSD estimation errors and handling the estimates as true values leads to speech estimation errors causing musical noise and undesired suppression of speech. In this paper, the uncertainty of the available speech PSD estimates is addressed. The main contributions are the following. First, we summarize and examine ways to model and incorporate the uncertainty of PSD estimates for a more robust speech enhancement performance. Second, a novel nonlinear clean speech estimator is derived that takes into account prior knowledge about the absolute value of typical speech PSDs. Third, we show that the derived statistical framework provides uncertainty-aware counterparts to a number of well-known conventional clean speech estimators such as the Wiener filter and Ephraim and Malah's amplitude estimators. Fourth, we show how modern PSD estimators can be incorporated into the theoretical framework and propose to employ frequency dependent priors. Finally, the effects and benefits of considering the uncertainty of speech PSD estimates are analyzed, discussed, and evaluated via instrumental measures and a listening experiment.

Analysis of Eigenvalue Decomposition-Based Late Reverberation Power Spectral Density Estimation

Many speech dereverberation techniques require an estimate of the late reverberation power spectral density (PSD). State-of-the-art multichannel methods for estimating the late reverberation PSD typically rely on first, an estimate of the relative transfer functions (RTFs) of the target signal; second, a model for the spatial coherence matrix of the late reverberation; and finally, an estimate of the reverberant speech or reverberant and noisy speech PSD matrix. The RTFs, the spatial coherence matrix, and the speech PSD matrix are all prone to modeling and estimation errors in practice, with the RTFs being particularly difficult to estimate accurately, especially in highly reverberant and noisy scenarios. Recently, we proposed an eigenvalue decomposition (EVD)-based late reverberation PSD estimator, which does not require an estimate of the RTFs. In this paper, this EVD-based PSD estimator is further analyzed and its estimation accuracy and computational complexity are analytically compared to a state-of-the-art maximum likelihood (ML) based PSD estimator. It is shown that for perfect knowledge of the RTFs, spatial coherence matrix, and reverberant speech PSD matrix, the ML-based and the EVD-based PSD estimates are both equal to the true late reverberation PSD. In addition, it is shown that for erroneous RTFs but perfect knowledge of the spatial coherence matrix and reverberant speech PSD matrix, the ML-based PSD estimate is larger than or equal to the true late reverberation PSD, whereas the EVD-based PSD estimate is obviously still equal to the true late reverberation PSD. Finally, it is shown that when modeling and estimation errors occur in all quantities, the ML-based PSD estimate is larger than or equal to the EVD-based PSD estimate. Simulation results for several realistic acoustic scenarios demonstrate the advantages of using the EVD-based PSD estimator in a multichannel Wiener filter, yielding a significantly better performance than the ML-based PSD estimator.

Cramér–Rao Bound Analysis of Reverberation Level Estimators for Dereverberation and Noise Reduction

The reverberation power spectral density (PSD) is often required for dereverberation and noise reduction algorithms. In this work, we compare two maximum likelihood (ML) estimators of the reverberation PSD in a noisy environment. In the first estimator, the direct path is first blocked. Then, the ML criterion for estimating the reverberation PSD is stated according to the probability density function of the blocking matrix (BM) outputs. In the second estimator, the speech component is not blocked. Instead, the ML criterion for estimating the speech and reverberation PSD is stated according to the probability density function of the microphone signals. To compare the expected mean square error (MSE) between the two ML estimators of the reverberation PSD, the Cramer–Rao Bounds (CRBs) for the two ML estimators are derived. We show that the CRB for the joint reverberation and speech PSD estimator is lower than the CRB for estimating the reverberation PSD from the BM outputs. Experimental results show that the MSE of the two estimators indeed obeys the CRB curves. Experimental results of multimicrophone dereverberation and noise reduction algorithm show the benefits of using the ML estimators in comparison with another baseline estimators.

Maximum-Likelihood Approach With Bayesian Refinement for Multichannel-Wiener Postfiltering

Multichannel signal enhancement is an important task, e.g., in modern speech communication. It can be approached in a mean-square error sense by multichannel Wiener filtering, which is often implemented as a cascade of spatial beamforming and spectral enhancement postfiltering. In this paper, we propose a novel Bayesian perspective on the problem of postfilter adaptation. Specifically, we present a two-step procedure for postfilter estimation. We first derive maximum-likelihood (ML) optimal estimators of the target signal and noise power spectral densities (PSD) employing the high-dimensional receiver data in conjunction with blindly estimated channels. When constructing a Wiener postfilter for noisy speech based on the respective speech and noise PSD estimates, this ML-based postfilter is capable of performing good speech enhancement, while leaving musical noise. Hence, we extend to a Bayesian estimation approach in a second step. By statistically modeling the ML-based postfilter estimation and further proposing a prior for the ideal Wiener postfilter, we derive a Bayesian refinement of the ML-based postfilter. This refinement then solely depends on the ML-based postfilter and is independent of the underlying high-dimensional data. Our Bayesian refinement is therefore implemented as a lightweight table look-up. Both proposed postfilters perform well in terms of segmental signal-to-noise ratio and perceptual evaluation of speech quality in various acoustical setups with nonstationary noise. The Bayes-refined postfilter offers consistent improvement over the ML-based postfilter and a single-channel speech enhancement baseline.

Decision-directed speech power spectral density matrix estimation for multichannel speech enhancement

In this letter, a multichannel decision-directed approach to estimate the speech power spectral density (PSD) matrix for multichannel speech enhancement is proposed. There have been attempts to build multichannel speech enhancement filters which depend only on the speech and noise PSD matrices, for which the accurate estimate of the clean speech PSD matrix is crucial for a successful noise reduction. In contrast to the maximum likelihood estimator which has been applied conventionally, the proposed decision-directed method is capable of tracking the time-varying speech characteristics more robustly and improves the noise reduction performance under various noise environments.

A Priori SNR Estimation Using Air- and Bone-Conduction Microphones

This paper proposes an a priori signal-to-noise ratio (SNR) estimator using an air-conduction (AC) and a bone-conduction (BC) microphone. Among various ways of combining AC and BC microphones for speech enhancement, it is shown that the total enhancement performance can be maximized if the BC microphone is utilized for estimating the power spectral density (PSD) of the desired speech signal. Considering the fact that a small deviation in the speech PSD estimation process brings severe spectral distortion, this paper focuses on controlling weighting factors while estimating the a priori SNR with the decision-directed approach framework. The time--frequency varying weighting factor that is determined by taking a minimum mean square error criterion improves the capability of eliminating residual noise and minimizing speech distortion. Since the weighting factors are also adjusted by measuring the usefulness of the AC and BC microphones, the proposed approach is suitable for tracking the parameter even if the characteristic of environment changes rapidly. The simulation results confirm the superiority of the proposed algorithm to conventional algorithms in high noise environments.

A speech enhancement method based on sparse reconstruction of power spectral density

Using sparse representation of power spectral density (PSD) approximated by magnitude-squared spectrum, a new speech enhancement method is presented. The approximation K-singular value decomposition (K-SVD) algorithm with nonnegative constraint is used to train an overcomplete dictionary of the clean speech PSD. The least angle regression algorithm (LARS) with a termination rule based on the ℓ2 norm of the sum of the noise PSD and cross term between the clean speech and noise spectra is applied to estimate the clean speech PSD. Combining the estimated PSD with the signal subspace approach based on the short-time spectral amplitude (SSB-STSA), the enhanced speech signal is obtained. The simulation results show that the new method can yield better performance in most of noise conditions.

The Effect of Spectral Estimation on Speech Enhancement Performance

It has long been observed that accuracy in spectral estimation greatly affects the quality of enhanced speech. A small decrease in the bias and variance of the estimator can greatly reduce the amount of residual noise and distortion in the recovered speech. To date, however, there has been little interest in a rigorous analysis quantifying such observations. In this paper, we analyze the effect of spectral estimate variance on enhanced speech as measured by quantitative and qualitative means. The performance analysis is derived for the signal subspace and the minimum mean square error short-time spectral amplitude estimators. Error is defined as the random function of frequency given by the difference between the estimated and the true power spectral density (PSD) functions. It is measured by its variance as a fraction of the clean speech PSD squared: a norm called the variance quality factor (VQF). The error VQF is derived in terms of the VQF of measurable quantities such as noisy speech and noise alone. It is shown that reducing the PSD estimate variance reduces significantly the VQF of the enhancement error. We provide analytical derivations to establish the results and accompanying simulations to confirm the theoretical analysis. Simulations test the periodogram, Blackman-Tukey, Bartlett-Welch, and Multitaper spectral estimation methods.

Letter: Optimal echo path modelling using chaotic modulated speech

AbstractA novel modulating scheme is presented that allows optimal echo path modelling using, real time least mean square (LMS) system identification techniques. The systems of interest are usually based on a class of digital adaptive filters (DAFs). The modelling convergence rate derived from the optimal Wiener weights defines the performance criterion. A novel chaotic based modulation regime utilising the logistic mapping is exploited to whiten the speech power spectral density (PSD) whilst preserving the signal bandwidth requirements. The whitened signal optimises the convergence rate and allows adaptation times to be calculated without a‐priori knowledge of the speech signal's second order statistics. Software simulations are reported for an ensemble of conversational speech realisations and it is shown that the described modulation scheme significantly improves the convergence rate and guarantees optimal modelling independent of the input speech signal statistics. Furthermore, the noise whitened input signal circumvents control supervisory problems usually associated with uncoded speech inputs.