Localization In Reverberant Environments Research Articles

A High-Resolution Time Reversal Method for Target Localization in Reverberant Environments

Reverberation in real environments is an important factor affecting the high resolution of target sound source localization (SSL) methods. Broadband low-frequency signals are common in real environments. This study focuses on the localization of this type of signal in reverberant environments. Because the time reversal (TR) method can overcome multipath effects and realize adaptive focusing, it is particularly suitable for SSL in a reverberant environment. On the basis of the significant advantages of the sparse Bayesian learning algorithm in the estimation of wave direction, a novel SSL is proposed in reverberant environments. First, the sound propagation model in a reverberant environment is studied and the TR focusing signal is obtained. We then use the sparse Bayesian framework to locate the broadband low-frequency sound source. To validate the effectiveness of the proposed method for broadband low-frequency targeting in a reverberant environment, simulations and real data experiments were performed. The localization performance under different bandwidths, different numbers of microphones, signal-to-noise ratios, reverberation times, and off-grid conditions was studied in the simulation experiments. The practical experiment was conducted in a reverberation chamber. Simulation and experimental results indicate that the proposed method can achieve satisfactory spatial resolution in reverberant environments and is robust.

A grid-free global optimization algorithm for sound sources localization in three-dimensional reverberant environments

In this work, a new grid-free global optimization algorithm that combined the image source model(ISM) and state transition algorithm(STA) was proposed to realize the sound source localization in reverberant environments. The simulation results show that only first-order mirror sources being kept can ensure localization accuracy. Compared with de-reverberation beamforming, the localization results of the proposed algorithm have no sidelobes and smaller mainlobes. Compared with the differential evolution(DE) algorithm, STA can converge to smaller fitness values, which can achieve more accurate sound source localization in reverberant environments. Meanwhile, the proposed algorithm can be applied to both single-source and multi-source localization in practical scenarios. The experimental results show that the proposed algorithm can locate the sound source accurately in reverberant environments.

Is acoustic space “learned” in the buildup of the precedence effect?

Most of what is known about sound source localization in reverberant environments rests upon knowledge of how the auditory system processes a pair of brief stimuli presented over headphones that simulate a direct sound and a single reflection. In everyday environments, however, sound sources often emit relatively continuous sounds or repeat them often in succession—for example, speech. Also, listeners often move. Dynamic changes in source/listener positions may present a unique opportunity to investigate the effects of “learning” that seem to result in the “buildup of the precedence effect—listeners' ability to effectively localize reverberant sounds after repeated presentation that would be poorly localized after only one presentation. The implicated learning may constitute learning of the spatial acoustics of the environment or learning of the temporal order and timing of reflections (or both). In this study, we sought to disambiguate these two possible learning strategies by comparing listener behavior in response to presentation of repeated pairs of lead/lag noise stimuli presented in a soundfield. Listeners’ perceived sound-source localization and the rate of fusion were measured.

Differences in sound-source localization in reverberant environments that depend on stimulus-type and mode of presentation

One interesting aspect of sound-source localization in reverberant environments is that different stimuli (e.g., speech versus noise) can elicit different spatial sensations at the same delays between the direct sound and its reflections. For example, a given delay between a single simulated direct sound and a delayed copy presented from a different location may elicit the sensation of either a single sound located at or near its source for speech while a noise stimulus under otherwise the same conditions might be perceived as a direct sound with echoes. Furthermore, presenting stimuli over headphones versus loudspeakers can further modify spatial perception for the same stimulus conditions. This modeling study will explore reasons for these two interesting differences of behavioral outcome, using simulation tools developed in Montagne and Zhou (JASA, 2018) and Pastore and Braasch (JASA, 2020).

Residual physical modeling and semi-supervised source localization

Source localization in reverberant environments remains an open challenge which ML techniques have shown promise in addressing. Real-world acoustic environments contain irregular boundaries, scattering, and diffracting elements (e.g., furniture and uneven surfaces) which are impractical to model using acoustic propagation software. We develop a hybrid approach to acoustic source localization which utilizes both analytical, signal processing-based localization, and machine learning (ML) to provide a data-driven source localization model. In this approach, the output of an analytical source localization model (e.g., SRP-PHAT or MUSIC) is augmented by a neural network (NN) system. This augmented system consists of a variational autoencoder and localization network, which is designed to correct the analytic estimate. The networks are trained via semi-supervised learning, on both labeled and unlabeled inputs. The NNs in this approach parameterize stochastic functions, which facilitate a statistically principled learning approach via variational inference (VI). The stochasticity also helps quantify source location uncertainty under the VI approximate posterior. For simulated and real acoustic data, the hybrid approach generalizes better and is more efficient than ML or analytic approaches alone.

Underwater Acoustic Localization Based on IR-GCC-PHAT in Reverberant Environments

This paper presents a method of underwater acoustic localization in reverberant environments. Time difference of arrival (TDOA) measurement algorithm is a key technology for estimating direction of arrival (DOA) of an underwater sound source. In strong multipath interference, the pseudo-peaks in a correlation function disturb the detection of a correct time position and lead to a large TDOA measurement error. The proposed algorithm computes a time difference by taking cross-correlation of two impulse responses and improves robustness to multipath interference. The comparison of TDOA algorithms is done by evaluating the position accuracy of underwater sound source in both simulation and experiment.

Semi-Supervised Source Localization in Reverberant Environments With Deep Generative Modeling

Localization in reverberant environments remains an open challenge. Recently, supervised learning approaches have demonstrated very promising results in addressing reverberation. However, even with large data volumes, the number of labels available for supervised learning in such environments is usually small. We propose to address this issue with a semi-supervised learning (SSL) approach, based on deep generative modeling. Our chosen deep generative model, the variational autoencoder (VAE), is trained to generate the phase of relative transfer functions (RTFs) between microphones. In parallel, a direction of arrival (DOA) classifier network based on RTF-phase is also trained. The joint generative and discriminative model, deemed VAE-SSL, is trained using labeled and unlabeled RTF-phase sequences. In learning to generate and classify the sequences, the VAE-SSL extracts the physical causes of the RTF-phase (i.e., source location) from distracting signal characteristics such as noise and speech activity. This facilitates effective end-to-end operation of the VAE-SSL, which requires minimal preprocessing of RTF-phase. VAE-SSL is compared with two signal processing-based approaches, steered response power with phase transform (SRP-PHAT) and MUltiple SIgnal Classification (MUSIC), as well as fully supervised CNNs. The approaches are compared using data from two real acoustic environments - one of which was recently obtained at Technical University of Denmark specifically for our study. We find that VAE-SSL can outperform the conventional approaches and the CNN in label-limited scenarios. Further, the trained VAE-SSL system can generate new RTF-phase samples which capture the physics of the acoustic environment. Thus, the generative modeling in VAE-SSL provides a means of interpreting the learned representations. To the best of our knowledge, this paper presents the first approach to modeling the physics of acoustic propagation using deep generative modeling.

Robust Indoor Localization in Reverberant Environment using Cross Correlation Functions as Time-of-Arrival Likelihood

Robust Indoor Localization in Reverberant Environment using Cross Correlation Functions as Time-of-Arrival Likelihood

Semi-supervised source localization in reverberant environments using deep generative modeling

We present a method for acoustic source localization in reverberant environments based on semi-supervised machine learning (ML) with deep generative models. Source localization in the presence of reverberation remains a major challenge, which recent ML techniques have shown promise in addressing. Despite often large data volumes, the number of labels available for supervised learning in reverberant environments is usually small. In semi-supervised learning, ML systems are trained using many examples with only few labels, with the goal of exploiting the natural structure of the data. We use variational autoencoders (VAEs), which are generative neural networks (NNs) that rely on explicit probabilistic representations, to model the latent distribution of reverberant acoustic data. VAEs consist of an encoder NN, which maps complex input distributions to simpler parametric distributions (e.g., Gaussian), and a decoder NN which approximates the training examples. The VAE is trained to generate the phase of relative transfer functions (RTFs) between two microphones in reverberant environments, in parallel with a DOA classifier, on both labeled and unlabeled RTF samples. The performance this VAE-based approach is compared with conventional and ML-based localization in simulated and real-world scenarios.

Source Localization Using Distributed Microphones in Reverberant Environments Based on Deep Learning and Ray Space Transform

In this article we present a methodology for source localization in reverberant environments from Generalized Cross Correlations (GCCs) computed between spatially distributed individual microphones. Reverberation tends to negatively affect localization based on Time Differences of Arrival (TDOAs), which become inaccurate due to the presence of spurious peaks in the GCC. We therefore adopt a data-driven approach based on a convolutional neural network, which, using the GCCs as input, estimates the source location in two steps. It first computes the Ray Space Transform (RST) from multiple arrays. The RST is a convenient representation of the acoustic rays impinging on the array in a parametric space, called Ray Space. Rays produced by a source are visualized in the RST as patterns, whose position is uniquely related to the source location. The second step consists of estimating the source location through a nonlinear fitting, which estimates the coordinates that best approximate the RST pattern obtained through the first step. It is worth noting that training can be accomplished on simulated data only, thus relaxing the need of actually deploying microphone arrays in the acoustic scene. The localization accuracy of the proposed techniques is similar to the one of SRP-PHAT, however our method demonstrates an increased robustness regarding different distributed microphones configurations. Moreover, the use of the RST as an intermediate representation makes it possible for the network to generalize to data unseen during training.

Diagonal Unloading Beamforming in the Spherical Harmonic Domain for Acoustic Source Localization in Reverberant Environments

Spherical microphone arrays allow the sound field analysis in three dimensions with the advantage of having the same resolution in all directions. By considering the frequency-independent character of the steering vectors in the spherical harmonic (SH) domain, we propose a very low-complexity SH diagonal unloading (DU) beamforming with a novel frequency smoothing power transform (FSPT) of the covariance matrices. We consider the direction of arrival (DOA) estimation problem of acoustic sources in reverberant conditions. The DU beamforming provides high resolution directional response since it exploits the subspace orthogonality property of the covariance matrix by the removal or the attenuation of the signal subspaces, obtained through the subtraction of an opportune diagonal matrix from the covariance matrix. The FSPT aims at smoothing the narrowband covariance matrices of the entire set of frequency domain components, and it pursues this goal by minimizing the narrowband error contributions due to reverberation in the broadband frequency smoothing covariance matrix. We analyze the DOA estimation performance using speech signals with simulations and real acoustic data in reverberant conditions. The results show that the proposed SH-DU-FSPT has a DOA estimation performance comparable to that of high resolution state-of-the-art methods with a significant reduction of the computational cost, since the steering directional responses are computed on the broadband frequency smoothing covariance matrix.

Sound Source Localization in Reverberant Environments Based on Structural Sparse Bayesian Learning

Sound Source Localization in Reverberant Environments Based on Structural Sparse Bayesian Learning

Distributed Expectation-Maximization Algorithm for Speaker Localization in Reverberant Environments

Localization of acoustic sources has attracted a considerable amount of research attention in recent years. A major obstacle to achieving high localization accuracy is the presence of reverberation, the influence of which obviously increases with the number of active speakers in the room. Human hearing is capable of localizing acoustic sources even in extreme conditions. In this study, we propose to combine a method based on human hearing mechanisms and a modified incremental distributed expectation-maximization (IDEM) algorithm. Rather than using phase difference measurements that are modeled by a mixture of complex-valued Gaussians, as proposed in the original IDEM framework, we propose to use time difference of arrival measurements in multiple subbands and model them by a mixture of real-valued truncated Gaussians. Moreover, we propose to first filter the measurements in order to reduce the effect of the multipath conditions. The proposed method is evaluated using both simulated data and real-life recordings.

Reverberation Robust Feature Extraction for Sound Source Localization Using a Small-Sized Microphone Array

Conventional methods for sound source localization using microphone arrays are usually addressed from the signal processing viewpoint, where the sound source location is treated as a continuous parameter to be estimated over some spatial space. Actually, in some practical scenarios, such as in conference rooms and cars, sound source locations are only confined to some predefined areas. Therefore, it is more reasonable to deal with the problem from a machine learning point of view. By incorporating the prior information available about sound environments, machine learning-based methods have the potential to better deal with sound source localization in the presence of room reverberation. The key to machine learning-based sound source localization methods is how to extract effective source location features. The existing feature extraction schemes, such as the popular time-difference-of-arrival features, however, are not suitable for small-sized sensor arrays, due to the fact that sound source localization in reverberant environments become much challenging for small-sized arrays. To combat the problem, in this paper, we propose a reverberation robust feature extraction method for sound source localization based on sound intensity (SI) estimation using a small-sized microphone array. In particular, three robust feature extraction procedures have been employed in the proposed features, including normalization, phase transform weighting, and fully incorporating the redundancies in SI estimation. Simulation and real-world experimental results both show that the proposed sound source location features are more effective for small-sized arrays in reverberant environments when compared with the existing features.

Perpendicular Cross-Spectra Fusion for Sound Source Localization With a Planar Microphone Array

Multiple sound source localization in reverberant environments stands as one of the most difficult challenges for many applications related to microphone array signal processing. In this paper, we describe perpendicular cross-spectra fusion (PCSF), a new direction-of-arrival (DOA) estimation algorithm, which utilizes an analytic formula for direction estimation in the time–frequency (TF) domain. Inherent to this technique is the presence of multiple direction estimation subsystems which operate in parallel, producing a multiplicity of candidate DOAs at each TF point. We define a metric of coherence, based on the property of divergence of the different DOA estimators, for assessing the reliability of different signal portions, so that only TF bins with a high quality of directional information are exploited for local DOA estimation. The resulting collection of local DOAs is provided as input to a recently proposed histogram processing approach, which is based on matching pursuit. Results based on simulation and real recordings illustrate the advantages of PCSF compared to other DOA estimation techniques subject to the same histogram-based processing, in the context of real-time multiple source localization and counting; improved performance in reverberant conditions and high tolerance to diffuse and common mode noise.

Sound Source and Microphone Localization From Acoustic Impulse Responses

This letter proposes a new method for source and microphone localization in reverberant environments using a randomly arranged sensor array, under the hypothesis that the position of one reference sensor and the geometry of the environment are known, and the other microphone positions are unknown. A minimum mean square error (MMSE) estimator that exploits early reflections is proposed. The MMSE estimator is solved by a grid search method that combines the information on early reflections estimated using a multichannel blind system identification and the time difference of arrivals between the reflections and the direct-path calculated with the image-source model. Simulations under different reverberant scenarios demonstrate the ability of the proposed approaches in localizing source and microphone.

Acoustic cavitation localization in reverberant environments

Cavitation detection and localization techniques generally require visual access to the fluid field, multiple high-speed cameras, and appropriate illumination to locate cavitation. This can be costly and is not always suitable for all test environments, particularly when the bubble diameter is small or duration is short. Acoustic detection and localization of cavitation can be more robust and more easily implemented, without requiring visual access to the site in question. This research utilizes the distinct acoustic signature of cavitation events to both detect and localize cavitation during experimental water tunnel testing. Using 22 hydrophones and the processing techniques plane-wave beamforming and Matched-Field Processing (MFP), cavitation is accurately and quickly localized during testing in a 12” diameter water tunnel. Cavitation is induced using a Nd:YAG laser for precise control of bubble location and repeatability. Accounting for and overcoming the effects of reflections on acoustic localization in acoustically small environments is paramount in water tunnels, and the techniques employed to minimize error will be discussed.

Auditory localization in realistic environments by normal-hearing and hearing-impaired listeners

The ability to correctly localize sounds is important for general awareness of the auditory scene and communication in adverse acoustic conditions. However, most localization studies are performed in rather simple and artificial conditions. In particular, very few studies have considered localization in reverberant environments or in the presence of complex interferers, and no studies have systematically investigated the effect of distance. In the present study, localization performance was therefore measured as a function of source-receiver distance using a virtual auditory environment. With increasing source-receiver distance the direct-to-reverberation energy ratio decreases and the auditory system increasingly relies on mechanisms related to the precedence effect. Both aspects may be particularly problematic for hearing-impaired listeners. The acoustics of a cafeteria was simulated with the room acoustic software ODEON for a large number of source-receiver locations. Signals were generated for a 3D ar...

Multiple-Hypothesis Extended Particle Filter for Acoustic Source Localization in Reverberant Environments

Particle filtering has been shown to be an effective approach to solving the problem of acoustic source localization in reverberant environments. In reverberant environment, the direct- arrival of the single source is accompanied by multiple spurious arrivals. Multiple-hypothesis model associated with these arrivals can be used to alleviate the unreliability often attributed to the acoustic source localization problem. Until recently, this multiple- hypothesis approach was only applied to bootstrap-based particle filter schemes. Recently, the extended Kalman particle filter (EPF) scheme which allows for an improved tracking capability was proposed for the localization problem. The EPF scheme utilizes a global extended Kalman filter (EKF) which strongly depends on prior knowledge of the correct hypotheses. Due to this, the extension of the multiple-hypothesis model for this scheme is not trivial. In this paper, the EPF scheme is adapted to the multiple-hypothesis model to track a single acoustic source in reverberant environments. Our work is supported by an extensive experimental study using both simulated data and data recorded in our acoustic lab. Various algorithms and array constellations were evaluated. The results demonstrate the superiority of the proposed algorithm in both tracking and switching scenarios. It is further shown that splitting the array into several sub-arrays improves the robustness of the estimated source location.

Localization dominance and the effect of frequency in the Mongolian Gerbil, Meriones unguiculatus

Due to its good low-frequency hearing, the Mongolian Gerbil (Meriones unguiculatus) has become a well-established animal model for human hearing. In humans, sound localization in reverberant environments is facilitated by the precedence effect, i.e., the perceptual suppression of spatial information carried by echoes. The current study addresses the question whether gerbils are a valid animal model for such complex spatial processing. Specifically, we quantify localization dominance, i.e., the fact that in the context of precedence, only the directional information of the sound which reaches the ear first dominates the perceived position of a sound source whereas directional information of the delayed echoes is suppressed. As localization dominance is known to be stimulus-dependent, we quantified the extent to which the spectral content of transient sounds affects localization dominance in the gerbil. The results reveal that gerbils show stable localization dominance across echo delays, well comparable to humans. Moreover, localization dominance systematically decreased with increasing center frequency, which has not been demonstrated in an animal before. These findings are consistent with an important contribution of peripheral-auditory processing to perceptual localization dominance. The data show that the gerbil is an excellent model to study the neural basis of complex spatial-auditory processing.