Reverberant Environments Research Articles

Is acoustic space “learned” in the buildup of the precedence effect?

Most of what is known about sound source localization in reverberant environments rests upon knowledge of how the auditory system processes a pair of brief stimuli presented over headphones that simulate a direct sound and a single reflection. In everyday environments, however, sound sources often emit relatively continuous sounds or repeat them often in succession—for example, speech. Also, listeners often move. Dynamic changes in source/listener positions may present a unique opportunity to investigate the effects of “learning” that seem to result in the “buildup of the precedence effect—listeners' ability to effectively localize reverberant sounds after repeated presentation that would be poorly localized after only one presentation. The implicated learning may constitute learning of the spatial acoustics of the environment or learning of the temporal order and timing of reflections (or both). In this study, we sought to disambiguate these two possible learning strategies by comparing listener behavior in response to presentation of repeated pairs of lead/lag noise stimuli presented in a soundfield. Listeners’ perceived sound-source localization and the rate of fusion were measured.

Measuring angle-dependent sound absorption in ordinary rooms

The effective use of sound absorbing materials in room acoustical design requires the use of angle-dependent coefficients, as opposed to random incidence coefficients, to properly account for the dissipation of sound energy at the room's boundary. In this study, a method is proposed for measuring the angle-dependent surface impedance and absorption coefficient of a boundary material in situ, in an ordinary room. The proposed method relies on the reconstruction of the pressure and normal particle velocity at the boundary using a plane-wave decomposition of the sound field measured with an array of microphones. In sufficiently reverberant environments, where sound strikes the material from nearly all directions in space, the angle-dependent properties of the boundary can be obtained simultaneously for all angles of incidence, from a single source position. In addition, the proposed methodology enables us to identify the specific directions of incidence on the material and makes it possible to estimate its operational acoustic performance. The validity of the method is examined experimentally in a reverberation chamber and in a classroom, using a robotic arm to scan the sound field in the vicinity of an absorbing boundary. Consistent absorption data are measured for various source positions and room configurations.

Global Sound Field Reconstruction in the Room Environment Based on Inverse Wave-Based Simulation

An inverse wave modeling-based method is proposed for globally reconstructing the sound field in room environments. The method builds the wave model of the sound field as prior knowledge to support the reconstruction under strong reverberation. In this method, the whole space is divided into a set of subdomains. Based on the theory of discretization-based numerical simulation, a wave model that can describe the transfer characteristic between any subdomain and the source is built. Supported by this model, the sound source is recovered based on spatial sound pressure sampling and the global sound field reconstruction can be further accomplished in the reverberant environment. In particular, the shape function with the property of sparsity is constructed in building the wave model. Then, the intensity on the point source is represented by a sparse vector over the subdomains, and then, the sparse method can be used to achieve the recovery of this vector, which reduces the sampling burden in the space. Numerical verifications are performed to evaluate the performances of the proposed method. It demonstrates that the proposed method is capable of obtaining accurate reconstructions in a strong reverberant environment. It also shows that the method is applicable to problems with complicated excitations in the low-frequency range.

The precedence effect in spatial hearing manifests in cortical neural population responses

BackgroundTo localize sound sources accurately in a reverberant environment, human binaural hearing strongly favors analyzing the initial wave front of sounds. Behavioral studies of this “precedence effect” have so far largely been confined to human subjects, limiting the scope of complementary physiological approaches. Similarly, physiological studies have mostly looked at neural responses in the inferior colliculus, the main relay point between the inner ear and the auditory cortex, or used modeling of cochlear auditory transduction in an attempt to identify likely underlying mechanisms. Studies capable of providing a direct comparison of neural coding and behavioral measures of sound localization under the precedence effect are lacking.ResultsWe adapted a “temporal weighting function” paradigm previously developed to quantify the precedence effect in human for use in laboratory rats. The animals learned to lateralize click trains in which each click in the train had a different interaural time difference. Computing the “perceptual weight” of each click in the train revealed a strong onset bias, very similar to that reported for humans. Follow-on electrocorticographic recording experiments revealed that onset weighting of interaural time differences is a robust feature of the cortical population response, but interestingly, it often fails to manifest at individual cortical recording sites.ConclusionWhile previous studies suggested that the precedence effect may be caused by early processing mechanisms in the cochlea or inhibitory circuitry in the brainstem and midbrain, our results indicate that the precedence effect is not fully developed at the level of individual recording sites in the auditory cortex, but robust and consistent precedence effects are observable only in the auditory cortex at the level of cortical population responses. This indicates that the precedence effect emerges at later cortical processing stages and is a significantly “higher order” feature than has hitherto been assumed.

Blast trauma affects production and perception of mouse ultrasonic vocalizations.

Blast trauma from explosions affects hearing and communication in a significant proportion of soldiers. Many veterans report difficulty communicating, especially in noisy and reverberant environments, which contributes to complex mental health problems including anxiety and depression. However, the relationship between communication and perceptual problems after a blast has received little scientific attention. In the current studies, the effects of blast trauma on the production and perception of ultrasonic vocalizations (USVs) by CBA/CaJ mice, a common animal model for hearing and communication disorders, was explored. Overall, mice change the total number of vocalizations, the proportion produced of each syllable category, and the peak frequency, bandwidth, and duration of their vocalizations after blast exposure. Further, the perception of USVs is affected after blast trauma, with an immediate worsening of detection for most USV categories in the first 1-5 days after blasts, which later recovers. This study is the first to examine changes in the production and perception of communication signals after blast traumas in mice and is an important step towards developing treatments for blast-induced hearing and communication disorders.

3D Multiple Sound Source Localization by Proposed T-Shaped Circular Distributed Microphone Arrays in Combination with GEVD and Adaptive GCC-PHAT/ML Algorithms.

Multiple simultaneous sound source localization (SSL) is one of the most important applications in the speech signal processing. The one-step algorithms with the advantage of low computational complexity (and low accuracy), and the two-step methods with high accuracy (and high computational complexity) are proposed for multiple SSL. In this article, a combination of one-step-based method based on the generalized eigenvalue decomposition (GEVD), and a two-step-based method based on the adaptive generalized cross-correlation (GCC) by using the phase transform/maximum likelihood (PHAT/ML) filters along with a novel T-shaped circular distributed microphone array (TCDMA) is proposed for 3D multiple simultaneous SSL. In addition, the low computational complexity advantage of the GCC algorithm is considered in combination with the high accuracy of the GEVD method by using the distributed microphone array to eliminate spatial aliasing and thus obtain more appropriate information. The proposed T-shaped circular distributed microphone array-based adaptive GEVD and GCC-PHAT/ML algorithms (TCDMA-AGGPM) is compared with hierarchical grid refinement (HiGRID), temporal extension of multiple response model of sparse Bayesian learning with spherical harmonic (SH) extension (SH-TMSBL), sound field morphological component analysis (SF-MCA), and time-frequency mixture weight Bayesian nonparametric acoustical holography beamforming (TF-MW-BNP-AHB) methods based on the mean absolute estimation error (MAEE) criteria in noisy and reverberant environments on simulated and real data. The superiority of the proposed method is presented by showing the high accuracy and low computational complexity for 3D multiple simultaneous SSL.

Deep Learning-Based Estimation of Reverberant Environment for Audio Data Augmentation.

This paper proposes an audio data augmentation method based on deep learning in order to improve the performance of dereverberation. Conventionally, audio data are augmented using a room impulse response, which is artificially generated by some methods, such as the image method. The proposed method estimates a reverberation environment model based on a deep neural network that is trained by using clean and recorded audio data as inputs and outputs, respectively. Then, a large amount of a real augmented database is constructed by using the trained reverberation model, and the dereverberation model is trained with the augmented database. The performance of the augmentation model was verified by a log spectral distance and mean square error between the real augmented data and the recorded data. In addition, according to dereverberation experiments, the proposed method showed improved performance compared with the conventional method.

Tracking of Moving Sources in a Reverberant Environment Using Evolutionary Algorithms

This paper describes a source tracking technique in a reverberant environment using a new combination of an adaptive species-based particle swarm optimization (ASPSO) algorithm and a multisignal classification (MUSIC) algorithm. To mitigate the effects of reverberation, an insightful dereverberation method based on an online autoregressive (AR) array and a minimum variance distortionless response (MVDR) beamformer is developed to dereverberate the microphone signal prior to direction of arrival (DOA) estimation using MUSIC. On the basis of several evolutionary schemes, ASPSO enables rapid tracking by finding local maxima in the MUSIC pseudospectrum. In the ASPSO algorithm, particles are divided into different species, where each species is associated with a sound source. As the sound source moves, the DOA information is dynamically updated using ASPSO, in which the inertia weight decreases progressively to prevent premature convergence. Two update rules for adapting the filter coefficients are employed for drastically moving sources. Simulations and experiments are conducted using a circular microphone array to validate the proposed ASPSO with AR (ASPSO-AR) algorithm. The results demonstrate that ASPSO-AR requires one-third of the processing time of the grid search (GS) method. In addition, the root-mean-square error (RMSE) of the ASPSO-AR algorithm is 10° less than that of the GS method.

Convolutive Transfer Function-Based Multichannel Nonnegative Matrix Factorization for Overdetermined Blind Source Separation

Most multichannel blind source separation (BSS) approaches rely on a spatial model to encode the transfer functions from sources to microphones and a source model to encode the source power spectral density. The rank-1 spatial model has been widely exploited in independent component analysis (ICA), independent vector analysis (IVA), and independent low-rank matrix analysis (ILRMA). The full-rank spatial model is also considered in many BSS approaches, such as full-rank spatial covariance matrix analysis (FCA), multichannel nonnegative matrix factorization (MNMF), and FastMNMF, which can improve the separation performance in the case of long reverberation times. This paper proposes a new MNMF framework based on the convolutive transfer function (CTF) for overdetermined BSS. The time-domain convolutive mixture model is approximated by a frequency-wise convolutive mixture model instead of the widely adopted frequency-wise instantaneous mixture model. The iterative projection algorithm is adopted to estimate the demixing matrix, and the multiplicative update rule is employed to estimate nonnegative matrix factorization (NMF) parameters. Finally, the source image is reconstructed using a multichannel Wiener filter. The advantages of the proposed method are twofold. First, the CTF approximation enables us to use a short window to represent long impulse responses. Second, the full-rank spatial model can be derived based on the CTF approximation and slowly time-variant source variances, and close relationships between the proposed method and ILRMA, FCA, MNMF and FastMNMF are revealed. Extensive experiments show that the proposed algorithm achieves a higher separation performance than ILRMA and FastMNMF in reverberant environments.

Supervised Learning Approach for Explicit Spatial Filtering of Speech

Spatial filtering of speech based on neural networks has been widely studied. However, existing approaches focus on improving signal extraction or separation performance, and how to define the signal in the direction-of-interest (DOI) for spatial filtering has not been investigated in detail. This study proposes a method to train neural networks for extracting directivity components of speech signals in the DOI. To this end, we formulate the problem by defining the DOI and its corresponding desired signal in a reverberant environment. Moreover, we demonstrate an on-the-fly training data generation procedure to feed the spatially diverse data to train the networks. The proposed method was evaluated with regard to spatial speech extraction and localization performance. In particular, it has been confirmed that the network trained with the proposed method using simulated datasets also functions for real recordings.

Microphone Array-Based Sound Source Localization Using Convolutional Residual Network

Microphone array-based sound source localization (SSL) is widely used in a variety of occasions such as video conferencing, robotic hearing, speech enhancement, speech recognition and so on. The traditional SSL methods cannot achieve satisfactory performance in adverse noisy and reverberant environments. In order to improve localization performance, a novel SSL algorithm using convolutional residual network (CRN) is proposed in this paper. The spatial features including time difference of arrivals (TDOAs) between microphone pairs and steered response power-phase transform (SRP-PHAT) spatial spectrum are extracted in each Gammatone sub-band. The spatial features of different sub-bands with a frame are combine into a feature matrix as the input of CRN. The proposed algorithm employ CRN to fuse the spatial features. Since the CRN introduces the residual structure on the basis of the convolutional network, it reduce the difficulty of training procedure and accelerate the convergence of the model. A CRN model is learned from the training data in various reverberation and noise environments to establish the mapping regularity between the input feature and the sound azimuth. Through simulation verification, compared with the methods using traditional deep neural network, the proposed algorithm can achieve a better localization performance in SSL task, and provide better generalization capacity to untrained noise and reverberation.

Decoupled Multiple Speaker Direction-of-Arrival Estimator Under Reverberant Environments

Direction-of-arrival (DOA) estimation for multiple simultaneous speakers in reverberant environments is still one of the challenging tasks in the audio signal processing field. A recent approach addresses this problem using a spherical harmonics domain feature named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">relative harmonic coefficients</i> (RHC). Based on a bin-wise operation across the STFT (short-time Fourier transform) domain, this method detects the direct-path RHC in the first stage, followed by single source localization in the second stage. However, the method is computationally expensive as each STFT bin requires an exhaustive grid search over the two-dimensional (2-D) directional space. In this paper, we propose a significantly more computationally efficient alternative that decouples the azimuth and elevation 2-D search to two separate one-dimensional (1-D) search. The proposed multi-speaker localization algorithm comprises of two main steps, responsible for: (i) achieving a joint direct-path RHC detection and decoupled DOA estimation using 1-D search; and (ii) counting the number of speakers and estimating their DOAs based on the estimates from direct-path dominated STFT bins. Experiments using both simulated and real-life reverberant recordings confirm the significant computational complexity reduction while achieving competitive localization accuracy, compared to the baseline approaches. Although our proposed method performs in an unsupervised manner, it proves to be applicable even under unfavorable acoustic environments with a high reverberation level (e.g., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$T_{60}=1$</tex-math></inline-formula> second).

Acoustic Source Localization in the Circular Harmonic Domain Using Deep Learning Architecture

The problem of direction of arrival (DOA) estima- tion with a circular microphone array has been addressed with classical source localization methods, such as the model-based methods and the parametric methods. These methods have an ad- vantage in estimating the DOAs in a blind manner, i.e. with no (or limited) prior knowledge about the sound sources. However, their performance tends to degrade rapidly in noisy and reverberant environments or in the presence of sensor array limitations, such as sensor gain and phase errors. In this paper, we present a new approach by leveraging the strength of a convolutional neural network (CNN)-based deep learning approach. In particular, we design new circular harmonic features that are frequency- invariant as inputs to the CNN architecture, so as to offer improvements in DOA estimation in unseen adverse environments and obtain good adaptation to array imperfections. To our knowledge, such a deep learning approach has not been used in the circular harmonic domain. Experiments performed on both simulated and real-data show that our method gives significantly better performance, than the recent baseline methods, in a variety of noise and reverberation levels, in terms of the accuracy of the DOA estimation.

Design of Generalized Sinusoidal Frequency Modulated Pulse Train Waveform to Improve Tracking Performance of High Duty Cycle Sonar Systems

High duty cycle sonar (HDCS) systems have a high potential for improving tracking performance compared to conventional pulsed active sonar systems, but their implementation has been challenging to achieve. This is because conventional studies had focused primarily on solving direct blast interference problems caused by continuous transmission and reception of pulse train waveforms. In this paper, we propose a design scheme for a generalized sinusoidal frequency modulated (GSFM) pulse train waveform to improve HDCS tracking performance. The proposed optimization scheme utilizes the trade-off relationship between detection performance in a reverberation environment and measurement uncertainty according to the parameter ρ of the GSFM waveform. The detection probability and measurement noise covariance matrix of the Kalman filter are calculated using the HCDS tracking framework, which reflects the designed GSFM pulse train waveform in the tracking process. The simulation using the HDCS tracking framework demonstrated that optimal tracking performance was obtained when the parameter ρ was 1.07.

Ensemble Learning Approach With Class Rotation for Three-Dimensional Classification on Direction-of-Arrival Estimation

Over the past decade, neural networks have been widely used for direction-of-arrival (DoA) estimation owing to their high accuracy in noisy and reverberant environments. Classes of singular-model classifiers generally correspond to discretized DoA candidate angles, which in the case of a three-dimensional estimation, are bounded by the grid derived from uniform sampling over the unit sphere. Motivated by this, we propose an ensemble learning approach for classification tasks to improve estimation accuracy, as the ensemble criterion outperforms the individual criterion. First, individual networks that make up the ensemble differ slightly by grid rotation according to the Euler rotation theorem to complement discrete directional information. Score fusion was performed in the spherical harmonic domain for a more stable ensemble classification because the grid rotation also involves an angular mismatch. Moreover, to achieve a more accurate DoA estimation, interpolation over the fused scores was performed. Performance analysis and a comparison of state-of-the-art parametric and deep learning-based methods in several acoustic situations were conducted to determine the accuracy of the ensemble and to analyze the gradual angular error reduction as a function of noise and reverberation levels as more networks were added.

Weighted pressure matching with windowed targets for personal sound zones.

Personal sound zones (PSZ) systems use an array of loudspeakers to render independent audio signals to multiple listeners within a room. The performance of a PSZ system, designed using weighted pressure matching, depends on the selected target responses for the bright zone. In reverberant environments, the target responses are generally chosen to be the room impulse responses from one of the loudspeakers to the control points in the selected bright zone. This approach synthesizes the direct propagation component and all the reverberant components in the bright zone, while minimizing the energy in the dark zone. We present a theoretical analysis to show that high energy differences cannot be achieved for the diffuse reverberant components in the bright and dark zones, and so trying to synthesize these components in the bright zone does not lead to the best performance. It is then shown that the performance can be improved by using windowed versions of these measured impulse responses as target signals, in order to control which reverberant components are synthesized in the bright zone and which are not. This observation is supported by experimental measurements in two scenarios with different levels of reverberation.

VACE-WPE: Virtual Acoustic Channel Expansion Based on Neural Networks for Weighted Prediction Error-Based Speech Dereverberation

Speech dereverberation is an important issue for many real-world speech processing applications. Among the techniques developed, the weighted prediction error (WPE) algorithm has been widely adopted and advanced over the last decade, which blindly cancels out the late reverberation component from the reverberant mixture of microphone signals. In this study, we extend the neural-network-based virtual acoustic channel expansion (VACE) framework for the WPE-based speech dereverberation, a variant of the WPE that we recently proposed to enable the use of dual-channel WPE algorithm in a single-microphone speech dereverberation scenario. Based on the previous study, some ablation studies are conducted regarding the constituents of the VACE-WPE in an offline processing scenario. These studies reveal the characteristics of the system, thereby simplifying the architecture and leading to the introduction of new strategies for training the neural network for the VACE. Experimental results demonstrate that VACE-WPE (our PyTorch implementation and pre-trained models are available from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/dreadbird06/vace_wpe</uri> ) considerably outperforms its single-channel counterpart in simulated noisy reverberant environments in terms of objective speech quality and is superior to the single-channel WPE as well as several fully neural speech dereverberation methods when employed as the front-end for the far-field automatic speech recognizer.

Suppressing reverberation in cochlear implant stimulus patterns using time-frequency masks based on phoneme groups.

Cochlear implant (CI) users experience considerable difficulty in understanding speech in reverberant listening environments. This issue is commonly addressed with time-frequency masking, where a time-frequency decomposed reverberant signal is multiplied by a matrix of gain values to suppress reverberation. However, mask estimation is challenging in reverberant environments due to the large spectro-temporal variations in the speech signal. To overcome this variability, we previously developed a phoneme-based algorithm that selects a different mask estimation model based on the underlying phoneme. In the ideal case where knowledge of the phoneme was assumed, the phoneme-based approach provided larger benefits than a phoneme-independent approach when tested in normal-hearing listeners using an acoustic model of CI processing. The current work investigates the phoneme-based mask estimation algorithm in the real-time feasible case where the prediction from a phoneme classifier is used to select the phoneme-specific mask. To further ensure real-time feasibility, both the phoneme classifier and mask estimation algorithm use causal features extracted from within the CI processing framework. We conducted experiments in normal-hearing listeners using an acoustic model of CI processing, and the results showed that the phoneme-specific algorithm benefitted the majority of subjects.

Preliminary Archaeoacoustic Study of Kanheri Caves in Mumbai (Maharashtra, India)

Here we report first ever study on acoustical evaluation of Kanheri Caves located in Sanjay Gandhi National Park, Mumbai (Maharashtra, India). These caves are dated to a period between 2nd century BCE to 7th century CE. In this study we used an ambisonic recorder to capture Impulse Response, which carries acoustic signature of the place. Out of total 109 caves 41 were surveyed in available time. Out of those reverberant environment was noted in 12 caves. Measurements were made only in 3 caves (Cave Nos. 1, 3, 11) which are important. In the beginning we carried out an FFT analysis. We then studied room acoustic parameters like Reverberation Time, Early Decay Time, Clarity, Definition, etc., based on the measurement of Impulse response. Cave No. 3 have high value of reverberation time, compared to other. Therefore it also have lower clarity compared to others. It is properties needs to be compared with similar structures (chaityas) in Maharashtra (at Ajanta, Ellora, Nashik, Junnar, etc.) and elsewhere in India. It is worthwhile to carry out further research in Cave No. 3 with more sophisticated instruments as well as 3D modeling. Since the experiment was performed with receiver at only one position, we also suggest to carry out experiment with receiver at multiple positions and then comparing them.

Sparse DNN Model for Frequency Expanding of Higher Order Ambisonics Encoding Process

The performance of higherorder Ambisonics (HOA) signals obtained using spherical harmonics decomposition method is disturbed by two primary sources of errors, the noise pollution in low-frequency band and the spatial aliasing in high-frequency band. Inspired by the HOA signals upscale method, which is performed using the sparse character of the sound field, this paper propose a sound field decomposition model based on a sparse deep neural network that offers HOA signals with wider frequency bandwidth. We use the frequency domain multi-scale convolutional network to realize the spherical harmonics decomposition, as well as learning the spatial aliasing pattern, based on which the aliasing-free HOA signals can be derived. Besides, we apply a sparse encoding network to cpature the sparse feature of the sound field which will improve the model performance when the sparse condition is satisfied. The experiments results prove that the proposed model can obtain HOA signals with wider frequency range of operation under multiple sources (up to 10 sources) and low reverberant environments (<inline-formula><tex-math notation="LaTeX">$T_{60}\le$</tex-math></inline-formula> 400 ms). When the sparsity feature cannot be satisfied (<inline-formula><tex-math notation="LaTeX">$T_{60} =$</tex-math></inline-formula> 800 ms), the proposed network model still maintain the same performance as the traditional methods.