Late Reverberation Research Articles

Denoising Directional Room Impulse Responses with Spatially Anisotropic Late Reverberation Tails

Directional room impulse responses (DRIR) measured with spherical microphone arrays (SMA) enable the reproduction of room reverberation effects on three-dimensional surround-sound systems (e.g., Higher-Order Ambisonics) through multichannel convolution. However, such measurements inevitably contain a nondecaying noise floor that may produce an audible “infinite reverberation effect” upon convolution. If the late reverberation tail can be considered a diffuse field before reaching the noise floor, the latter may be removed and replaced with an extension of the exponentially-decaying tail synthesized as a zero-mean Gaussian noise. This has previously been shown to preserve the diffuse-field properties of the late reverberation tail when performed in the spherical harmonic domain (SHD). In this paper, we show that in the case of highly anisotropic yet incoherent late fields, the spatial symmetry of the spherical harmonics is not conducive to preserving the energy distribution of the reverberation tail. To remedy this, we propose denoising in an optimized spatial domain obtained by plane-wave decomposition (PWD), and demonstrate that this method equally preserves the incoherence of the late reverberation field.

Early reflection detection using autocorrelation to improve robustness of speaker verification in reverberant conditions

A speech signal captured by a distant microphone is generally smeared by reverberation, which severely degrades automatic speaker recognition performance. To improve system performance, an effective and robust method is proposed to extract features for speech processing. In this paper, a room impulse response is presumed to comprise of three parts: a direct-path response, early reflections and late reverberations. Since late reverberations are known to be a major cause of system performance degradation, this paper focuses on dealing with the effect of early reflection because the early reflections and their properties play a necessary role within the acoustics of an enclosure. The proposed method first estimates the early reflection using autocorrelation function from the presentation of speech signals in the first stage, the estimates are combined with an anechoic signal for use into training the system in the second stage. The employed method looks to be promising, achieving a substantial improvement in system performance relating to reduced equal error rate and detection trade-off, especially at longer reverberation time.

A method to virtually extend reverberation time of measured impulse responses without losing room coloration

For various audio and acoustical applications, it is useful to modify the reverberation time of a measured room impulse response. For example, it can be interesting for creating various modified impulse responses for convolution reverbs in video games or sound designs. With this in mind, it is proposed to modify an impulse response reverberation time while keeping the other reverberation characteristics, such as the frequency response, unaffected. Knowing how a typical impulse response is related to the room characteristics, we propose a new method. First, the impulse response of a room is measured. Then, the reverberation time is evaluated. This reverberation time is next approximated by Sabine-Eyring equation, resulting in an absorption coefficient. A new absorption coefficient is then manually provided to reach a new target reverberation time. Finally, the late reverberation of the original impulse response is convolved with a gaussian white noise to the desired length and stitched with appropriate decaying envelopes to the end of the initially measured impulse response. This paper illustrates that the method works based on simulations and that the modified impulse response is realistic and can be related to the actual room.For various audio and acoustical applications, it is useful to modify the reverberation time of a measured room impulse response. For example, it can be interesting for creating various modified impulse responses for convolution reverbs in video games or sound designs. With this in mind, it is proposed to modify an impulse response reverberation time while keeping the other reverberation characteristics, such as the frequency response, unaffected. Knowing how a typical impulse response is related to the room characteristics, we propose a new method. First, the impulse response of a room is measured. Then, the reverberation time is evaluated. This reverberation time is next approximated by Sabine-Eyring equation, resulting in an absorption coefficient. A new absorption coefficient is then manually provided to reach a new target reverberation time. Finally, the late reverberation of the original impulse response is convolved with a gaussian white noise to the desired length and stitched with appropriate dec...

A Novel Scheme for Single-Channel Speech Dereverberation

This paper presents a novel scheme for speech dereverberation. The core of our method is a two-stage single-channel speech enhancement scheme. Degraded speech obtains a sparser representation of the linear prediction residual in the first stage of our proposed scheme by applying orthogonal matching pursuit on overcomplete bases, trained by the K-SVD algorithm. Our method includes an estimation of reverberation and mixing time from a recorded hand clap or a simulated room impulse response, which are used to create a time-domain envelope. Late reverberation is suppressed at the second stage by estimating its energy from the previous envelope and removed with spectral subtraction. Further speech enhancement is applied on minimizing the background noise, based on optimal smoothing and minimum statistics. Experimental results indicate favorable quality, compared to two state-of-the-art methods, especially in real reverberant environments with increased reverberation and background noise.

Estimation of Late Reverberation Characteristics from a Single Two-Dimensional Environmental Image Using Convolutional Neural Networks

Estimation of Late Reverberation Characteristics from a Single Two-Dimensional Environmental Image Using Convolutional Neural Networks

Robust Joint Estimation of Multimicrophone Signal Model Parameters

One of the biggest challenges in multimicrophone applications is the estimation of the parameters of the signal model, such as the power spectral densities (PSDs) of the sources, the early (relative) acoustic transfer functions of the sources with respect to the microphones, the PSD of late reverberation, and the PSDs of microphone-self noise. Typically, existing methods estimate subsets of the aforementioned parameters and assume some of the other parameters to be known a priori . This may result in inconsistencies and inaccurately estimated parameters and potential performance degradation in the applications using these estimated parameters. So far, there is no method to jointly estimate all the aforementioned parameters. In this paper, we propose a robust method for jointly estimating all the aforementioned parameters using confirmatory factor analysis. The estimation accuracy of the signal-model parameters thus obtained outperforms existing methods in most cases. We experimentally show significant performance gains in several multimicrophone applications over state-of-the-art methods.

Subjective assessment of the intelligibility of noised speech in lecture room

Subjective assessment of the speech intelligibility is of great practical interest, since this parameter can be used in many engineering and natural-mathematical fields, such as bioengineering, design of lecture rooms and concert halls, mathematical modeling and medicine. By this time, several attempts were made to assess the intelligibility of the room. The results of such experiments became conclusions: 1) early reflections improve the intelligibility of distorted noise and reverb speech; 2) using binaural type of listening we will have better intelligibility than with monoural. But to re-implement such experiments, it is necessary to use a lot of expensive equipment, which is not always possible. Therefore, the purpose of this work is to study the influence of the characteristics of the room on the speech intelligibility without the use of a large number of equipment. The idea of the experiment is as follows: distorted by noise and reverb signals must be listened to by the participants in the experiment, the perceived sound is fixed and compared with the undistorted signal. The clear signal was recorded using a microphone, sound card and audio file software. Synthesis of distorted signals took place in two stages: 1) adding white noise; 2) filtering the resulting mix through a non-recursive digital filter using the impulse characteristics of the room for six distances to the source of sound. The mathematical model of the distorted (output) signal was the convolution of an additive mix of clear signal and white noise with a pulse characteristic. To ensure the necessary signal-to-noise ratio, the noise was weighed by the corresponding coefficients at the stage of adding it to the clear signal. Testing was carried out in five stages: 1) simulating of distorted signals; 2) voicing signal to participant; 3) signal listening; 4) fixation of the perceived sound; 5) intelligibility calculating. For testing, six binaural impulse characteristics of the room with known parameters from the Aachen database of impulse characteristics were used. Each of the impulse characteristics corresponds to a certain value of the distance from the sound source to the microphone. After the completion of the experiment, the overall result was obtained by averaging by the number of participants. The overall result has shown that for small values of the signal-to-noise ratio over long distances, the intelligibility is greater than for small distances. For high values, the signal-to-noise ratio is better for small distances. Such results may be explained by the fact that, at long distances, the combination of the effects of early reflections and binaural listening positively affects the intelligibility. The main reason for the low intelligibility of the distorted by noise speech in the room is a late reverberation. As a conclusion we can say that the computer modeling method makes it easy to create a distorted by the noise and influence of the room signal; It is shown that, for small values, the signal-to-noise ratio is more readable than for small distances.

Late Reverberation Cancellation Using Bayesian Estimation of Multi-Channel Linear Predictors and Student's t-Source Prior

Multi-channel linear prediction MCLP can model the late reverberation in the short-time Fourier transform domain using a delayed linear predictor and the prediction residual is taken as the desired early reflection component. Traditionally, a Gaussian source model with time-dependent precision inverse of variance is considered for the desired signal. In this paper, we propose a Student's t-distribution model for the desired signal, which is realized as a Gaussian source with a Gamma distributed precision. Further, since the choice of a proper MCLP order is critical, we also incorporate a Gaussian distribution prior for the prediction coefficients and a higher order. We consider a batch estimation scenario and develop variational Bayes expectation maximization VBEM algorithm for joint posterior inference and hyper-parameter estimation. This has lead to more accurate and robust estimation of the late reverb component and hence its cancellation, benefitting the desired residual signal estimation. Along with these stochastic models, we formulate single channel output MISO and multi channel output MIMO schemes using shared priors for the desired signal precision and the estimated MCLP coefficients at each microphone. Experiments using real room impulse responses show improved late reverberation suppression with the proposed VBEM approach over the traditional methods, for different room conditions. Additionally, we achieve a sparse coefficient vector for the MCLP avoiding the criticality of manually choosing the model order. The MIMO formulation is easily extended to include spatial filtering of the enhanced signals, which further improves the estimation of the desired signal.

Auditory Feedback for Navigation with Echoes in Virtual Environments: Training Procedure and Orientation Strategies.

Being able to hear objects in an environment, for example using echolocation, is a challenging task. The main goal of the current work is to use virtual environments (VEs) to train novice users to navigate using echolocation. Previous studies have shown that musicians are able to differentiate sound pulses from reflections. This paper presents design patterns for VE simulators for both training and testing procedures, while classifying users' navigation strategies in the VE. Moreover, the paper presents features that increase users' performance in VEs. We report the findings of two user studies: a pilot test that helped improve the sonic interaction design, and a primary study exposing participants to a spatial orientation task during four conditions which were early reflections (RF), late reverberation (RV), early reflections-reverberation (RR) and visual stimuli (V). The latter study allowed us to identify navigation strategies among the users. Some users (10/26) reported an ability to create spatial cognitive maps during the test with auditory echoes, which may explain why this group performed better than the remaining participants in the RR condition.

Common mathematical framework for stochastic reverberation models.

In the field of room acoustics, it is well known that reverberation can be characterized statistically in a particular region of the time-frequency domain (after the transition time and above Schroeder's frequency). Since the 1950s, various formulas have been established, focusing on particular aspects of reverberation: exponential decay over time, correlations between frequencies, correlations between sensors at each frequency, and time-frequency distribution. In this paper, the author introduces a stochastic reverberation model, which permits us to retrieve all these well-known results within a common mathematical framework. To the best of the author's knowledge, this is the first time that such a unification work is presented. The benefits are multiple: several formulas generalizing the classical results are established that jointly characterize the spatial, temporal, and spectral properties of late reverberation.

Raking early reflection signals for late reverberation and noise reduction.

In acoustic environments the microphones capture a source signal corrupted by room reverberation and noise. This letter presents two optimum spatial filters which jointly reduce late reverberation and noise for the sound source recorded from distance in a reverberant room. The proposed rake filters are derived by incorporating spatial information about early reflection paths into the multichannel filter design. The results of performed numerical and real experiments indicate that the proposed rake filtering improves the reduction of detrimental late reverberation and noise, and increases the perceptual quality of the recorded speech.

Scene-aware audio for 360° videos

Although 360° cameras ease the capture of panoramic footage, it remains challenging to add realistic 360° audio that blends into the captured scene and is synchronized with the camera motion. We present a method for adding scene-aware spatial audio to 360° videos in typical indoor scenes, using only a conventional mono-channel microphone and a speaker. We observe that the late reverberation of a room's impulse response is usually diffuse spatially and directionally. Exploiting this fact, we propose a method that synthesizes the directional impulse response between any source and listening locations by combining a synthesized early reverberation part and a measured late reverberation tail. The early reverberation is simulated using a geometric acoustic simulation and then enhanced using a frequency modulation method to capture room resonances. The late reverberation is extracted from a recorded impulse response, with a carefully chosen time duration that separates out the late reverberation from the early reverberation. In our validations, we show that our synthesized spatial audio matches closely with recordings using ambisonic microphones. Lastly, we demonstrate the strength of our method in several applications.

An Advanced Expectation-Maximization Algorithm for Multimicrophone Speech Dereverberation and Noise Reduction

In speech communication scenes, the microphone signals are often weakened by reverberation and ambient noise. Reverberation can be divided into an early part that is comprised of the direct speech and some early ones and a late part. In this paper, an advanced expectation-maximization (EM) algorithm for multimicrophone speech dereverberation and noise reduction, extending EM by taking the noise statistics into account, is proposed. This method aims at estimating the early parts filtered by early transform functions (ETFs). The EM algorithm is executed in two steps. In the E-step, the filtered early speech is calculated. Then, in the M-step, the ETFs, the power spectral density (PSD) of the filtered early speech and the late reverberation, and the spatial coherence matrix of the late reverberation are estimated. This algorithm is evaluated using an open-source room impulse responses (RIRs) database with a reverberation time of 0.36s and 0.61s under different signal-to-noise (SNR) conditions. It is proved an improvement compared with the original EM method.

Analysis of Eigenvalue Decomposition-Based Late Reverberation Power Spectral Density Estimation

Many speech dereverberation techniques require an estimate of the late reverberation power spectral density (PSD). State-of-the-art multichannel methods for estimating the late reverberation PSD typically rely on first, an estimate of the relative transfer functions (RTFs) of the target signal; second, a model for the spatial coherence matrix of the late reverberation; and finally, an estimate of the reverberant speech or reverberant and noisy speech PSD matrix. The RTFs, the spatial coherence matrix, and the speech PSD matrix are all prone to modeling and estimation errors in practice, with the RTFs being particularly difficult to estimate accurately, especially in highly reverberant and noisy scenarios. Recently, we proposed an eigenvalue decomposition (EVD)-based late reverberation PSD estimator, which does not require an estimate of the RTFs. In this paper, this EVD-based PSD estimator is further analyzed and its estimation accuracy and computational complexity are analytically compared to a state-of-the-art maximum likelihood (ML) based PSD estimator. It is shown that for perfect knowledge of the RTFs, spatial coherence matrix, and reverberant speech PSD matrix, the ML-based and the EVD-based PSD estimates are both equal to the true late reverberation PSD. In addition, it is shown that for erroneous RTFs but perfect knowledge of the spatial coherence matrix and reverberant speech PSD matrix, the ML-based PSD estimate is larger than or equal to the true late reverberation PSD, whereas the EVD-based PSD estimate is obviously still equal to the true late reverberation PSD. Finally, it is shown that when modeling and estimation errors occur in all quantities, the ML-based PSD estimate is larger than or equal to the EVD-based PSD estimate. Simulation results for several realistic acoustic scenarios demonstrate the advantages of using the EVD-based PSD estimator in a multichannel Wiener filter, yielding a significantly better performance than the ML-based PSD estimator.

Evaluation and Comparison of Late Reverberation Power Spectral Density Estimators

Reduction of late reverberation can be achieved using spatio-spectral filters, such as the multichannel Wiener filter. To compute this filter, an estimate of the late reverberation power spectral density (PSD) is required. In recent years, a multitude of late reverberation PSD estimators have been proposed. In this paper, these estimators are categorized into several classes, their relations and differences are discussed, and a comprehensive experimental comparison is provided. To compare their performance, simulations in controlled as well as practical scenarios are conducted. It is shown that a common weakness of spatial coherence-based estimators is their performance in high direct-to-diffuse ratio conditions. To mitigate this problem, a correction method is proposed and evaluated. It is shown that the proposed correction method can decrease the speech distortion without significantly affecting the reverberation reduction.

Speech intelligibility in rooms: Disrupting the effect of prior listening exposure.

It has been demonstrated that prior listening exposure to reverberant environments can improve speech understanding in that environment. Previous studies have shown that the buildup of this effect is brief (less than 1 s) and seems largely to be elicited by exposure to the temporal modulation characteristics of the room environment. Situations that might be expected to cause a disruption in this process have yet to be demonstrated. This study seeks to address this issue by showing what types of changes in the acoustic environment cause a breakdown of the room exposure phenomenon. Using speech carrier phrases featuring sudden changes in the acoustic environment, breakdown in the room exposure effect was observed when there was change in the late reverberation characteristics of the room that signaled a different room environment. Changes in patterns of early reflections within the same room environment did not elicit breakdown. Because the environmental situations that resulted in breakdown also resulted in substantial changes to the broadband temporal modulation characteristic of the signal reaching the ears, results from this study provide additional support for the hypothesis that the room exposure phenomenon is linked to the temporal modulation characteristics of the environment.

A perceptually motivated LP residual estimator in noisy and reverberant environments

Both reverberation and additive noise can degrade the quality of recorded speech and thus should be suppressed simultaneously. Previous studies have shown that the generalized singular value decomposition (GSVD) has the capability of suppressing the additive noise effectively, but it is not often applied for speech dereverberation since reverberation is considered to be convolutive as well as colored noise. Recently, we revealed that late reverberation is also additive and relatively white interference component in the linear prediction (LP) residual domain. To suppress both late reverberation and additive noise, we have proposed an optimal filter for LP residual estimator (LPRE) based on a constrained minimum mean square error (CMMSE) by using GSVD in single channel speech enhancement, where the algorithm is referred as CMMSE-GSVD-LPRE. Experimental results have shown a better performance of the CMMSE-GSVD-LPRE than spectral subtraction methods, but some residual noise and reverberation components are still audible and annoying. To solve this problem, this paper incorporates the masking properties of the human auditory system in the LP residual domain to further suppress these residual noise and reverberation components while reducing speech distortion at the same time. Various simulation experiments are conducted, and the results show an improved performance of the proposed algorithm. Experimental results with speech recorded in noisy and reverberant environments further confirm the effectiveness of the proposed algorithm in real-world environments.

Self-masking and overlap-masking from reverberation using the speech-evoked auditory brainstem response.

This study introduces an improved method to investigate the effects of reverberation using the speech-evoked auditory brainstem response (ABR) that more realistically captures the influence of self- and overlap-masking induced by room reverberation. Speech-evoked ABR was measured under three acoustic scenarios: anechoic, mild reverberation with dominance of early reflections, and severe reverberation with dominance of late reverberation. Responses were significantly weaker and had longer latencies with severe reverberation relative to anechoic and mild reverberation. Although larger responses and shorter latencies were observed with mild reverberation than anechoic, possibly due to early reflections, these reached significance in only one of six ABR response measures.

Binaural Sound Localization Based on Reverberation Weighting and Generalized Parametric Mapping

Binaural sound source localization is an important technique for speech enhancement, video conferencing, and human-robot interaction, etc. However, in realistic scenarios, the reverberation and environmental noise would degrade the precision of sound direction estimation. Therefore, reliable sound localization is essential to practical applications. To deal with these disturbances, this paper presents a novel binaural sound source localization approach based on reverberation weighting and generalized parametric mapping. First, the reverberation weighting as a preprocessing stage, is used to separately suppress the early and late reverberation, while preserving interaural cues. Then, two binaural cues, i.e., interaural time and intensity differences, are extracted from the frequency-domain representations of dereverberated binaural signals for the online localization. Their corresponding templates are established using the training data. Furthermore, the generalized parametric mapping is proposed to build a generalized parametric model for describing relationships between azimuth and binaural cues analytically. Finally, a two-step sound localization process is introduced to refine azimuth estimation based on the generalized parametric model and template matching. Experiments in both simulated and real scenarios validate that the proposed method can achieve better localization performance compared to state-of-the-art methods.

Interactive sound propagation and rendering for large multi-source scenes

We present an approach to generate plausible acoustic effects at interactive rates in large dynamic environments containing many sound sources. Our formulation combines listener-based backward ray tracing with sound source clustering and hybrid audio rendering to handle complex scenes. We present a new algorithm for dynamic late reverberation that performs high-order ray tracing from the listener against spherical sound sources. We achieve sublinear scaling with the number of sources by clustering distant sound sources and taking relative visibility into account. We also describe a hybrid convolution-based audio rendering technique that can process hundreds of thousands of sound paths at interactive rates. We demonstrate the performance on many indoor and outdoor scenes with up to 200 sound sources. In practice, our algorithm can compute more than 50 reflection orders at interactive rates on a multicore PC, and we observe a 5x speedup over prior geometric sound propagation algorithms.