Dereverberation Method Research Articles

An improved two-stage dereverberation method based on bayesian estimation of a speech source

It is not easy to extract speech emanating from a specific direction in a large reverberant room with additional sound sources. Many of the existing methods work well only if there is a single sound source, and their performance degrades if additional sound sources are present. In this work, we have addressed this problem by maximizing the posterior probability of speech signal as computed by use of Bayes' Theorem. Our de-reverb method is two-staged. We have evaluated the efficacy of our method against several popular methods in terms of five objective measures; Signal-to-Interference Ratio, Log-Likelihood Ratio, Cepstrum Distance, Frequency-Weighted Segment SNR, and Speechto-Reverberation Modulation Energy Ratio. We show that on each of these parameters, the method proposed in this work performs better than the one we have benchmarked it against.

Tracking of Moving Sources in a Reverberant Environment Using Evolutionary Algorithms

This paper describes a source tracking technique in a reverberant environment using a new combination of an adaptive species-based particle swarm optimization (ASPSO) algorithm and a multisignal classification (MUSIC) algorithm. To mitigate the effects of reverberation, an insightful dereverberation method based on an online autoregressive (AR) array and a minimum variance distortionless response (MVDR) beamformer is developed to dereverberate the microphone signal prior to direction of arrival (DOA) estimation using MUSIC. On the basis of several evolutionary schemes, ASPSO enables rapid tracking by finding local maxima in the MUSIC pseudospectrum. In the ASPSO algorithm, particles are divided into different species, where each species is associated with a sound source. As the sound source moves, the DOA information is dynamically updated using ASPSO, in which the inertia weight decreases progressively to prevent premature convergence. Two update rules for adapting the filter coefficients are employed for drastically moving sources. Simulations and experiments are conducted using a circular microphone array to validate the proposed ASPSO with AR (ASPSO-AR) algorithm. The results demonstrate that ASPSO-AR requires one-third of the processing time of the grid search (GS) method. In addition, the root-mean-square error (RMSE) of the ASPSO-AR algorithm is 10° less than that of the GS method.

VACE-WPE: Virtual Acoustic Channel Expansion Based on Neural Networks for Weighted Prediction Error-Based Speech Dereverberation

Speech dereverberation is an important issue for many real-world speech processing applications. Among the techniques developed, the weighted prediction error (WPE) algorithm has been widely adopted and advanced over the last decade, which blindly cancels out the late reverberation component from the reverberant mixture of microphone signals. In this study, we extend the neural-network-based virtual acoustic channel expansion (VACE) framework for the WPE-based speech dereverberation, a variant of the WPE that we recently proposed to enable the use of dual-channel WPE algorithm in a single-microphone speech dereverberation scenario. Based on the previous study, some ablation studies are conducted regarding the constituents of the VACE-WPE in an offline processing scenario. These studies reveal the characteristics of the system, thereby simplifying the architecture and leading to the introduction of new strategies for training the neural network for the VACE. Experimental results demonstrate that VACE-WPE (our PyTorch implementation and pre-trained models are available from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/dreadbird06/vace_wpe</uri> ) considerably outperforms its single-channel counterpart in simulated noisy reverberant environments in terms of objective speech quality and is superior to the single-channel WPE as well as several fully neural speech dereverberation methods when employed as the front-end for the far-field automatic speech recognizer.

Generative adversarial networks for single channel separation of convolutive mixed speech signals

Abstract The suppression of interference for speech recognition is of great significance in noisy situation, especially in single channel receiving mode, the suppression of interference is much more difficult. In this paper, we propose a generative adversarial network (GAN) based method for single channel dereverberation and speech separation. Different from the existing methods, our method considers the influence of strong reverberation on the observed signals. The proposed network involves two parts: reverberation suppression and target speech enhancement. Firstly, we use an improved CyclyGAN to compensate the multi-path effect on both target speech and interference. Secondly, we propose a differentialGAN to extract both target speech and interference while the interference enhancement network can indirectly improve the performance of target speech enhancement network. We use the real and imaginary parts of the complex spectrum as the feature vector, which avoids the phase mismatch during signal recovery. Simulation results show that our method is superior to its competitors in terms of multiple metrics in severe reverberation environment.

Independent Vector Extraction for Fast Joint Blind Source Separation and Dereverberation

We address a blind source separation (BSS) problem in a noisy reverberant environment in which the number of microphones $M$ is greater than the number of sources of interest, and the other noise components can be approximated as stationary and Gaussian distributed. Conventional BSS algorithms for the optimization of a multi-input multi-output convolutional beamformer have suffered from a huge computational cost when $M$ is large. We here propose a computationally efficient method that integrates a weighted prediction error (WPE) dereverberation method and a fast BSS method called independent vector extraction (IVE), which has been developed for less reverberant environments. We show that, given the power spectrum for each source, the optimization problem of the new method can be reduced to that of IVE by exploiting the stationary condition, which makes the optimization easy to handle and computationally efficient. An experiment of speech signal separation shows that, compared to a conventional method that integrates WPE and independent vector analysis, our proposed method achieves much faster convergence while maintaining its separation performance.

Split Bregman Approach to Linear Prediction Based Dereverberation With Enforced Speech Sparsity

The recordings of speech in enclosures are corrupted by reverberation caused by multipath wave propagation from the speaker to the distant microphones. In this letter, we address the problem of reducing the late part of room reverberation. The presented blind dereverberation method consists in multichannel linear prediction (MCLP) and enforces sparsity of the dereverberated speech by adopting the split Bregman approach. The proposed algorithm alternately solves two optimization problems, where the former cost function is derived by assuming that speech is modelled using a sparse prior distribution, while the latter optimization emphasizes speech sparsity by an additional incorporation of a weighted $\ell _1$ -norm of the output signal to the standard linear prediction based cost function. The results of experiments performed using simulated and measured room impulse responses for various reverberation time values indicate superior performance of the proposed sparse split Bregman (SSB) method over state-of-the-art non-sparse and sparse MCLP-based dereverberation methods in terms of standard evaluation measures and as pre-processsing to the automatic speech recognition.

Online Speech Dereverberation Using Mixture of Multichannel Linear Prediction Models

We extend the state-of-the-art online dereverberation method, online weighted prediction error (WPE), which predicts late reverberation components using a multichannel linear prediction (MCLP) filter. The multi-input/output inverse theorem states that in general such an MCLP filter for WPE exists only if the number of sources is less than that of microphones M and there is no additive noise; otherwise, the WPE model has error, degrading its performance especially when M is small. To mitigate this WPE drawback, we recently developed an offline dereverberation method called switching WPE (SwWPE), which has multiple MCLP filters and switches them for each time-frequency bin. We here propose a recursive least squares (RLS) algorithm for the online optimization of SwWPE. Our proposed RLS has forgetting weights for each MCLP filter, and their decay rates are controlled based on a switching mechanism of SwWPE to stably improve the dereverberation performance of the online WPE. Experimental results show that our online SwWPE clearly outperforms online WPE in speech dereverberation tasks.

Blind Signal Dereverberation Based on Mixture of Weighted Prediction Error Models

We extend the linear prediction-based dereverberation method called weighted prediction error (WPE). WPE optimizes a causal finite impulse response (FIR) filter that predicts the late reverberation components of an observed signal. However, by the multi-input/output inverse (MINT) theorem, in general, such FIR filters exist only when the number of sources is fewer than that of the microphones and no ambient noise exists. To mitigate the model error of WPE in adverse environments, we propose a mixture model of multiple WPEs in which the time frames are divided into clusters in each frequency bin and the WPE's causal FIR filter is optimized in each cluster. Experimental results show that our proposed method significantly improves the dereverberation performance of WPE when the noise level is high or the number of microphones is small.

Speech Dereverberation Based on Improved Wasserstein Generative Adversarial Networks

In reality, the sound we hear is not only disturbed by noise, but also the reverberant, whose effects are rarely taken into account. Recently, deep learning has shown great advantages in speech signal processing. But among the existing dereverberation approaches, very few methods apply deep learning at the waveform level. In addition, in the case of sever reverberation, the conventional speech dereverberation methods perform poorly, such as MCLP (multi-channel linear prediction). We proposed a new speech dereverberation method in this paper, which is based on improved WGAN (Wasserstein Generative Adversarial Networks), called WGAN-GP, whose generator uses strided-convolutional networks and the discriminator is structured on DNNs. Due to the addition of the gradient penalty item, WGAN-GP improves the stability of training and the generalization of the model. In the case of severe reverberation, according to the experimental results, the proposed system can perform better than MCLP. As the proposed method based on WAGN-GP can improve speech quality, speech signal processing systems may be able to apply it to pre-processing stage.

Harmonic beamformers for speech enhancement and dereverberation in the time domain

Abstract This paper presents a framework for parametric broadband beamforming that exploits the frequency-domain sparsity of voiced speech to achieve more noise reduction than traditional nonparametric broadband beamforming without introducing additional distortion. In this framework, the harmonic model is used to parametrize the signal of interest by a single parameter, the fundamental frequency, whereby both speech enhancement and derevereration can be performed. This framework thus exploits both the spatial and temporal properties of speech signals simultaneously and includes both fixed and adaptive beamformers, such as (1) delay-and-sum, (2) null forming, (3) Wiener, (4) minimum variance distortionless response (MVDR), and (5) linearly constrained minimum variance beamformers. Moreover, the framework contains standard broadband beamforming as a special case, whereby the proposed beamformers can also handle unvoiced speech. The reported experimental results demonstrate the capabilities of the proposed framework to perform both speech enhancement and dereverberation simultaneously. The proposed beamformers are evaluated in terms of speech distortion and objective measures for speech quality and speech intelligibility, and are compared to nonparametric broadband beamformers. The results show that the proposed beamformers perform well compared to traditional methods, including a state-of-the-art dereverberation method, particularly in adverse conditions with high amounts of noise and reverberation.

Blind speech dereverberation using sparse decomposition and multi-channel linear prediction

In this study, a blind speech dereverberation method in a noiseless single input multiple output acoustic channel is proposed. The method is based on multichannel linear prediction (MCLP) in STFT domain assuming sparsity in both residual speech and channel coefficients. The proposed speech dereverberation algorithm assumes that both the residual speech signal and the linear prediction coefficients is sparse. The optimization was performed by convex optimization using ADMM and CVX. The proposed model was compared with state of the art methods with lp norm optimization criteria. Simulations were evaluated in different room models with various reverberation times, numbers of microphones and parameter adjustments. The results show that the performance of the proposed method is superior in terms of speech dereverberation assessment criteria.

A Unified Convolutional Beamformer for Simultaneous Denoising and Dereverberation

This paper proposes a method for estimating a convolutional beamformer that can perform denoising and dereverberation simultaneously in an optimal way. The application of dereverberation based on a weighted prediction error (WPE) method followed by denoising based on a minimum variance distortionless response (MVDR) beamformer has conventionally been considered a promising approach, however, the optimality of this approach cannot be guaranteed. To realize the optimal integration of denoising and dereverberation, we present a method that unifies the WPE dereverberation method and a variant of the MVDR beamformer, namely a minimum power distortionless response (MPDR) beamformer, into a single convolutional beamformer, and we optimize it based on a single unified optimization criterion. The proposed beamformer is referred to as a Weighted Power minimization Distortionless response (WPD) beamformer. Experiments show that the proposed method substantially improves the speech enhancement performance in terms of both objective speech enhancement measures and automatic speech recognition (ASR) performance.

Multi-channel adaptive dereverberation robust to abrupt change of target speaker position.

Adaptive algorithm based on multi-channel linear prediction (MCLP) is an effective dereverberation method. However, the abrupt change of the target speech source position makes it difficult to guarantee both the fast convergence speed and the optimal steady-state behavior. In this letter, the recursive-least-squares (RLS)-based and Kalman-filter-based adaptive MCLP method for speech dereverberation are investigated. Based on the relative weighted change of the adaptive filter coefficients, a time-varying forgetting factor for the RLS algorithm and a re-initialization mechanism for the Kalman filter are proposed to make the algorithm robust to the abrupt change of the target speaker positions. The advantages of the proposed scheme are demonstrated in the experiments.

Blind reverberation energy estimation using exponential averaging with attack and release time constants for hearing aids

Abstract Dereverberation signal processing is necessary for hearing-aid systems because reverberation degrades speech intelligibility in reverberant environments. In general, hearing-aid systems require low computational cost and real-time signal processing. The spectral subtraction (SS) method is a simple and frequently used technique which is used not only for noise-reduction but for dereverberation as well. To perform dereverberation methods based on SS for hearing aids, the reverberation energy must be estimated or measured. In this paper, the SS-based blind estimation of reverberation energy in a single-channel speech signal is proposed by using exponential averaging with attack and release time constants. The estimation error, which is the difference between the true and estimated reverberation energy, was used for evaluation. The estimation error of the proposed method was compared with the results of the method proposed by Lebart et al., which is a well-known non-blind SS-based dereverberation method. According to the results, the reverberation energy was more accurately estimated when the reverberation time was longer than 0.6 s, and the estimation error of the proposed method was approximately half of that of the well-known non-blind method by Lebart.

Guided spectrogram filtering for speech dereverberation

Abstract Guided filtering is a computationally efficient and powerful technique used in image processing applications, such as edge-preserving smoothing, details enhancing and single image dehazing. In this paper, we propose a novel single channel speech dereverberation method using guided spectrogram filtering by considering a speech spectrogram as an image. The proposed method requires neither room acoustic parameter estimation nor late reverberant spectral variance estimation. Objective test results show the validity of the guided spectrogram filtering method for speech dereverberation. Compared with state-of-the-art speech dereverberation methods, the proposed method has better performance in terms of perceptual evaluation of speech quality (PESQ), speech-to-reverberation modulation energy ratio (SRMR) and short-time objective intelligibility (STOI) in most cases.

Adaptive Speech Dereverberation Using Constrained Sparse Multichannel Linear Prediction

In this letter, we present an adaptive speech dereverberation method based on constrained sparse multichannel linear prediction (MCLP), minimizing the mixed $\ell _{2,p}$ norm of the desired component. In order to prevent overestimation of the undesired reverberant component, possibly leading to severe distortions of the output, we propose to use a statistical model for late reverberation to limit the power of the MCLP-based estimate. The resulting constrained optimization problem is solved by using the alternating direction method of multipliers, resulting in two variants of the dereverberation algorithm. Simulation results show that the proposed constraint increases the robustness with respect to parameter selection and improves the usability for dynamic scenarios in comparison to the unconstrained method.

Low latency dereverberation with considering speech naturalness for hearing aids

It is well known that long reverberation makes speech unclear especially for hearing impaired. For a long time, a lot of researches on dereverberation have been studied. However, the computational cost of most of those methods are too expensive for hearing aids. Therefore, we have proposed the simplified dereverberation method, which can estimate reverberation energy from observed signal. In addition, it is also necessary implementing the processing with low latency to avoid an annoyance for hearing aid wearer. Processed signal have to be natural as much as possible to reduce listening effort, although there is performance limit because of simplicity. In this study, we make two proposals in our previous method. First, we apply Frequency-Warped Filterbank (FWF) in analysis and synthesis systems in order to make frequency bands as Bark scaled alignment, and to take the tolerable latency all over frequency for hearing. Second, we introduce the gain control in each frequency to maintain naturalness of processed speech, because reverberation energy varies depending on frequency bands in actual reverberant environments. From subjective experiments, there was a tendency that they prefer this proposal to the previous method.

Anti-Forensics of Environmental-Signature-Based Audio Splicing Detection and Its Countermeasure via Rich-Features Classification

Numerous methods for detecting audio splicing have been proposed. Environmental-signature-based methods are considered to be the most effective forgery detection methods. The performance of existing audio forensic analysis methods is generally measured in the absence of any anti-forensic attack. Effectiveness of these methods in the presence of anti-forensic attacks is therefore unknown. In this paper, we propose an effective anti-forensic attack for environmental-signature-based splicing detection method and countermeasures to detect the presence of the anti-forensic attack. For anti-forensic attack, dereverberation-based processing is proposed. Three dereverberation methods are considered to tamper with the acoustic environment signature. Experimental results indicate that the proposed dereverberation-based anti-forensic attack significantly degrades the performance of the selected splicing detection method. The proposed countermeasures exploit artifacts introduced by the anti-forensic processing. To detect the presence of potential anti-forensic processing, a machine learning-based framework is proposed. In particular, the proposed anti-forensic detection method uses a rich-feature model consisting of Fourier coefficients, spectral properties, high-order statistics of musical noise residuals, and modulation spectral coefficients to capture traces of dereverberation attacks. The performance of the proposed framework is evaluated on both synthetic data and real-world speech recordings. The experimental results show that the proposed rich-feature model can detect the presence of anti-forensic processing with an average accuracy of 95%.

Speech dereverberation method based on spectral subtraction and spectral line enhancement

Abstract Speech signals recorded with a distant microphone usually are interfered by the spatial reverberation in the room, which severely degrades the clarity and intelligibility of speech. A speech dereverberation method based on spectral subtraction and spectral line enhancement is proposed in this paper. Following the generalized statistical reverberation model, the power spectrum of late reverberation is estimated and removed from the reverberation speech by the spectral subtraction method. Then, according to the human auditory model, a spectral line enhancement technique based on adaptive post-filtering is adopted to further eliminate the reverberant components between adjacent speech formants. The proposed method can effectively suppress the spatial reverberation and improve the auditory perception of speech. The subjective and objective evaluation results reveal that the perceptual quality of speech is greatly improved by the proposed method.

Speech Dereverberation Using Non-Negative Convolutive Transfer Function and Spectro-Temporal Modeling

This paper presents two single channel speech dereverberation methods to enhance the quality of speech signals that have been recorded in an enclosed space. For both methods, the room acoustics are modeled using a nonnegative approximation of the convolutive transfer function (NCTF), and to additionally exploit the spectral properties of the speech signal, such as the low rank nature of the speech spectrogram, the speech spectrogram is modeled using nonnegative matrix factorization (NMF). Two methods are described to combine the NCTF and NMF models. In the first method, referred to as the integrated method, a cost function is constructed by directly integrating the speech NMF model into the NCTF model, while in the second method, referred to as the weighted method, the NCTF and NMF based cost functions are weighted and summed. Efficient update rules are derived to solve both optimization problems. In addition, an extension of the integrated method is presented, which exploits the temporal dependencies of the speech signal. Several experiments are performed on reverberant speech signals with and without background noise, where the integrated method yields a considerably higher speech quality than the baseline NCTF method and a state of the art spectral enhancement method. Moreover, the experimental results indicate that the weighted method can even lead to a better performance in terms of instrumental quality measures, but that the optimal weighting parameter depends on the room acoustics and the utilized NMF model. Modeling the temporal dependencies in the integrated method was found to be useful only for highly reverberant conditions.