Sound Source Localization Research Articles

The effect of right versus left long-term single-sided deafness on sound source localization

The effect of right versus left long-term single-sided deafness on sound source localization

Super-resolution localization and orientation estimation of multiple dipole sound sources: From a maximum likelihood framework to wind tunnel validation

This paper investigates the identification of multiple dipole sound sources using sound pressures measured from a microphone array. The problem is addressed in the maximum likelihood (ML) framework, where the locations, orientations, and powers of multiple dipole sound sources are unknown parameters to be estimated. By the consistency property of ML, the estimated parameters converge to their actual values, which implies an asymptotically perfect spatial resolution, if a sufficiently high signal-to-noise ratio can be achieved. In order to reduce the dimension of the optimization problem of ML, the contribution of each dipole source to the measured pressures is assumed to be a latent variable and the ML problem is equivalently solved via the expectation–maximization (EM) algorithm, which iteratively and sequentially updates each source contribution and the associated sound source parameters. The number of sound sources can also be determined by the model selection approaches which add a penalty of model dimension to the ML objective function. The proposed method is assessed via a laboratory experiment where the sound field is produced by dipole speakers and a wind tunnel experiment where airframe aerodynamic noise is generated at a high Reynolds number. Experimental results show that the proposed method outperforms existing approaches in the sense of higher spatial resolution, more accurate localization, and the capacity to identify the orientations of multiple dipole sound sources.

Sound source localization via distance metric learning with regularization

Sound source localization (SSL) or simply direction of arrival (DOA) classification is an important ingredient in acoustic applications. Traditional model-based algorithms are susceptible to the effects of noise and reverberation, while data-driven deep learning algorithms maintain strong performance across a variety of acoustic circumstances, but typically require a large amount of labeled data. Nevertheless, the existing datasets for SSL are not sufficiently big and diverse to achieve the full potential of deep learning algorithms. Then, it is an imperative work to develop a non-data-hungry algorithm of SSL using small or medium data volume. To this end, we propose a regularized distance metric learning algorithm, that is, by means of the kernel method, we design a nonlinear feature transformation from two aspects: feature points and feature distributions. It transforms the data into a new feature space that brings features of the same class as close as possible and removes features of different classes as far away as possible, which can significantly improve the output of a DOA classifier that follows. Experimental results show that the proposed algorithm outperforms deep learning algorithms in diverse acoustic conditions.

Vocal Call Locator Benchmark (VCL) for localizing rodent vocalizations from multi-channel audio.

Understanding the behavioral and neural dynamics of social interactions is a goal of contemporary neuroscience. Many machine learning methods have emerged in recent years to make sense of complex video and neurophysiological data that result from these experiments. Less focus has been placed on understanding how animals process acoustic information, including social vocalizations. A critical step to bridge this gap is determining the senders and receivers of acoustic information in social interactions. While sound source localization (SSL) is a classic problem in signal processing, existing approaches are limited in their ability to localize animal-generated sounds in standard laboratory environments. Advances in deep learning methods for SSL are likely to help address these limitations, however there are currently no publicly available models, datasets, or benchmarks to systematically evaluate SSL algorithms in the domain of bioacoustics. Here, we present the VCL Benchmark: the first large-scale dataset for benchmarking SSL algorithms in rodents. We acquired synchronized video and multi-channel audio recordings of 767,295 sounds with annotated ground truth sources across 9 conditions. The dataset provides benchmarks which evaluate SSL performance on real data, simulated acoustic data, and a mixture of real and simulated data. We intend for this benchmark to facilitate knowledge transfer between the neuroscience and acoustic machine learning communities, which have had limited overlap.

Population coding of auditory space in the dorsal inferior colliculus persists with altered binaural cues.

Sound localization is critical for real-world hearing, such as segregating overlapping sound streams. For optimal flexibility, central representations of auditory space must adapt to peripheral changes in binaural cue availability, such as following asymmetric hearing loss in adulthood. However, whether the mature auditory system can reliably encode spatial auditory representations upon abrupt changes in binaural input is unclear. Here we use 2-photon Ca2+ imaging in awake head-fixed mice to determine how the higher-order "shell" layers of the inferior colliculus (IC) encode sound source location in the frontal azimuth, under binaural conditions and after acute monaural hearing loss induced by an ear plug ipsilateral to the imaged hemisphere. Spatial receptive fields were typically broad and not exclusively contralateral: Neurons responded reliably to multiple positions in the contra- and ipsi-lateral hemifields, with preferred positions tiling the entire frontal azimuth. Ear plugging broadened receptive fields and reduced spatial selectivity in a subset of neurons, in agreement with an inhibitory influence of ipsilateral sounds. However ear plugging also enhanced spatial tuning and/or unmasked receptive fields in other neurons, shifting the distribution of preferred angles ipsilaterally with minimal impact on the neuronal population's overall spatial resolution; these effects occurred within 2 hours of ear plugging. Consequently, linear classifiers trained on fluorescence data from control and ear-plugged conditions had similar classification accuracy when tested on held out data from within, but not across hearing conditions. Spatially informative neuronal population codes therefore arise rapidly following monaural hearing loss, in absence of overt experience.

Improved Binaural Hearing Effects and Sound Source Localization in Listeners with Simulated Unilateral Conductive Hearing Loss: Effect of a Bone Conduction Device.

Objectives: It has been proven that patients with unilateral conductive hearing loss (UCHL) may encounter typical problems associated with asymmetric hearing, especially in challenging listening environments. In this study, we aimed to determine how UCHL affects speech recognition under multisource competing environments and the ability of sound source localization, as well as whether assistance with a bone conduction device (BCD) can confer hearing benefits in such listening tasks. Design: Acquired UCHL was simulated using an earplug combined with an earmuff in 10 listeners (mean age: 29.9 ± 4.77 years) with bilateral normal hearing (NH), and a within-subject repeated-measures design was used to compare hearing results among three listening conditions: NH (both ears open, C1), unilateral plug (UP) (simulated UCHL, C2), and UP + BCD (simulated UCHL aided with a BCD, C3). The speech reception threshold (SRT) of summation, squelch, and head shadow (HS) effects and the mean absolute error of sound source localization were used as markers for binaural hearing abilities. Results: BCD assistance (C3) improved the summation and HS effects in all listeners with simulated UCHL, resulting in a lower (i.e., better) SRT than that observed in C2; however, no significant differences in squelch effects were observed between C2 and C3. Notably, most listeners exhibited more accurate sound source localization in C3 than in C2. Further, BCD assistance mainly improved localization accuracy when the noise stimuli were presented at low intensities and on the hearing-impaired (plugged) side, suggesting that the benefits of BCD for sound localization are not based on the reacquisition of binaural processing. Conclusions: The current results have clinical implications for the promotion of BCDs in patients with UCHL, especially those with acquired UCHL who are unable to undergo surgery.

Three-dimensional grid-free sound source localization method based on deep learning

Sound source localization (SSL) technology is a popular method for identifying the locations of noise sources, which serves as a prerequisite for noise control. Deep learning, as a data-driven tool, shows broad perspectives in the field of SSL with its powerful nonlinear fitting ability. The existing deep learning-based SSL methods only provide a two-dimensional (2D) representation of the sound source location and cannot obtain the specific coordinates of the sound source in three-dimensional (3D) space. Although traditional beamforming methods can be directly generalized to 3D scenes in principle, they suffer from the limitations of insufficient vertical resolution and high computational cost. Therefore, a 3D grid-free SSL method (3DGF) informed by deep learning is suggested in this study to enhance the accuracy and computational efficiency of 3D localization. First, the number of data channels is compressed to respect limited memory resources during the training process. Subsequently, a dense convolutional neural network (DenseNet) model is utilized to obtain the 3D spatial coordinates of the sound source using the processed 3D beamforming map as input. Since the coordinates are continuous and are not constrained by the grid of the beamforming map, the grid-free strategy presents more accurate localization results. Then, the effects of the volume of training data and the compression ratio are analyzed, respectively, in simulation, and the localization performance with different signal-to-noise ratios (SNRs) is also tested. Finally, by comparing 3DGF with DAMAS, both simulation and experimental results demonstrate that 3DGF improves the accuracy and efficacy of 3D localization. Meanwhile, its satisfactory generalization ability and robustness against noise highlight its potential for practical applications.

Sound source localization based on microphone array mounted on unmanned aerial vehicles

Abstract Microphone array installed on unmanned aerial vehicles (UAV) can be used for acoustic detection in scenes, such as detecting noise sources in factories or monitoring corona in power grids. However, using acoustic sensors on UAV has great challenges. The motor rotation of UAV and the non-stationary noise generated by propeller make it difficult for microphone array to obtain useful information for sound source localization. In order to reduce self-noise, an improved least mean square (LMS) adaptive filtering algorithm is proposed. Combining filtering algorithm and multiple signal classification (MUSIC) algorithm to locate the sound source, the noise spectrum characteristics of UAV in outdoor free flight mode are analyzed. The microphone array is used to collect data and realize sound source localization in low Signal-to-Noise Ratio (SNR) environment. The simulation and experimental results show that the algorithm improves the robustness of the sound source localization algorithm in low SNR environment, and realizes the real-time localization of the target sound source during the flight of UAV with microphone array.

Microcontroller-based camera with the sound source localization for automated accident detection

Microcontroller-based camera with the sound source localization for automated accident detection

Eliminating quantization errors in classification-based sound source localization

Sound Source Localization (SSL) involves estimating the Direction of Arrival (DOA) of sound sources. Since the DOA estimation output space is continuous, regression might be more suitable for DOA, offering higher precision. However, in practice, classification often outperforms regression, exhibiting greater robustness. Conversely, classification’s drawback is inherent quantization error. Within the classification paradigm, the DOA output space is discretized into several intervals, each treated as a class. These classes show strong inter-class correlations, being inherently ordered, with higher similarity as intervals grow closer. Nevertheless, this characteristic has not been fully exploited. To address this, we propose Unbiased Label Distribution (ULD) to eliminate quantization error in training targets. Furthermore, we introduce Weighted Adjacent Decoding (WAD) to overcome quantization error during the decoding stage. Finally, we tailor two loss functions for the soft labels: Negative Log Absolute Error (NLAE) and Mean Squared Error without activation (MSE(wo)). Experimental results show our approach surpasses classification quantization limits, achieving state-of-the-art performance. Our code and supplementary material are available at https://github.com/linfeng-feng/ULD.

Benefits of Cochlear Implantation for Older Adults With Asymmetric Hearing Loss.

FDA-approved indications for cochlear implantation include patients with severe-to-profound unilateral hearing loss (UHL) or asymmetric hearing loss (AHL); however, these indications are not covered for Medicare beneficiaries. We assessed the outcomes of cochlear implant (CI) use for older adults with UHL or AHL. Eighteen older adults (≥65 years of age at surgery) with UHL/AHL participated in a prospective, longitudinal investigation evaluating outcomes of CI use. Speech recognition for the affected ear was evaluated with consonant-nucleus-consonant (CNC) words. Spatial hearing was assessed with measures of sound source localization and sentence recognition in noise. The target sentence was presented from the front and the masker was either co-located with the target (SoNo), presented toward the affected ear (SoNci) or contralateral ear (SoNcontra). Perceived benefit was assessed with the Speech, Spatial, and Qualities of Hearing scale (SSQ) and the Tinnitus Handicap Inventory (THI). Participants experienced significant improvements with CI use for CNC words (mean [SD]; preop: 8% [10%], 1 yr: 51% [22%], 5 yr: 50% [19%]), masked sentence recognition (SoNcontra preop: 5% [6%], 1 yr: 22% [15%], 5 yr: 41% [14%]), and localization (preop: 76° [18°], 1 yr: 40° [11°], 5 yr: 41° [14°]),and reported significant improvements in hearing abilities (SSQ Spatial Hearing preop: 3 [1], 1 yr: 6 [2], 5 yr: 6 [2]) and tinnitus severity (THI preop: 16 [18], 1 yr: 4 [14], 5 yr: 6 [12]). Older adults with UHL/AHL experience significant improvements in speech recognition, spatial hearing, and subjective perceptions (e.g., hearing abilities and tinnitus severity) with a CI as compared to pre-operative abilities. IV Laryngoscope, 2024.

Sound source localization in non-line-of-sight environment based on equivalent source method combined with time reversal method

Sound source localization is a critical task in various applications. However, it becomes challenging when the sound sources are not directly visible to the sensors due to obstacles or other environmental factors. This paper proposes a novel approach for locating sound sources in non-line-of-sight environments by combining the equivalent source method (ESM) with the time reversal method (TRM). The ESM is used to compute the frequency responses between the focusing points and the measurement points numerically. These frequency responses are then substituted to the TRM to localize the sound sources. To evaluate the performance of the proposed method, it is tested through numerical simulations and an experiment. The results prove that the ESM can compute the frequency responses accurately, and the combination of TRM and ESM effectively localizes sound sources in non-line-of-sight environments.

High-Resolution Identification of Sound Sources Based on Sparse Bayesian Learning with Grid Adaptive Split Refinement

Sound source identification technology based on a microphone array has many application scenarios. The compressive beamforming method has attracted much attention due to its high accuracy and high-resolution performance. However, for the far-field measurement problem of large microphone arrays, existing methods based on fixed grids have the defect of basis mismatch. Due to the large number of grid points representing potential sound source locations, the identification accuracy of traditional grid adjustment methods also needs to be improved. To solve this problem, this paper proposes a sound source identification method based on adaptive grid splitting and refinement. First, the initial source locations are obtained through a sparse Bayesian learning framework. Then, higher-weight candidate grids are retained, and local regions near them are split and updated. During the iteration process, Green’s function and the source strength obtained in the previous iteration are multiplied to get the sound pressure matrix. The robust principal component analysis model of the Gaussian mixture separates and replaces the sound pressure matrix with a low-rank matrix. The actual sound source locations are gradually approximated through the dynamically adjusted sound pressure low-rank matrix and optimized grid transfer matrix. The performance of the method is verified through numerical simulations. In addition, experiments on a standard aircraft model are conducted in a wind tunnel and speakers are installed on the model, proving that the proposed method can achieve fast, high-precision imaging of low-frequency sound sources in an extensive dynamic range at long distances.

A Study of Improved Two-Stage Dual-Conv Coordinate Attention Model for Sound Event Detection and Localization.

Sound Event Detection and Localization (SELD) is a comprehensive task that aims to solve the subtasks of Sound Event Detection (SED) and Sound Source Localization (SSL) simultaneously. The task of SELD lies in the need to solve both sound recognition and spatial localization problems, and different categories of sound events may overlap in time and space, making it more difficult for the model to distinguish between different events occurring at the same time and to locate the sound source. In this study, the Dual-conv Coordinate Attention Module (DCAM) combines dual convolutional blocks and Coordinate Attention, and based on this, the network architecture based on the two-stage strategy is improved to form the SELD-oriented Two-Stage Dual-conv Coordinate Attention Model (TDCAM) for SELD. TDCAM draws on the concepts of Visual Geometry Group (VGG) networks and Coordinate Attention to effectively capture critical local information by focusing on the coordinate space information of the feature map and dealing with the relationship between the feature map channels to enhance the feature selection capability of the model. To address the limitation of a single-layer Bi-directional Gated Recurrent Unit (Bi-GRU) in the two-stage network in terms of timing processing, we add to the structure of the two-layer Bi-GRU and introduce the data enhancement techniques of the frequency mask and time mask to improve the modeling and generalization ability of the model for timing features. Through experimental validation on the TAU Spatial Sound Events 2019 development dataset, our approach significantly improves the performance of SELD compared to the two-stage network baseline model. Furthermore, the effectiveness of DCAM and the two-layer Bi-GRU structure is confirmed by performing ablation experiments.

Enhancing Sound Source Localization via False Negative Elimination.

Sound source localization aims to localize objects emitting the sound in visual scenes. Recent works obtaining impressive results typically rely on contrastive learning. However, the common practice of randomly sampling negatives in prior arts can lead to the false negative issue, where the sounds semantically similar to visual instance are sampled as negatives and incorrectly pushed away from the visual anchor/query. As a result, this misalignment of audio and visual features could yield inferior performance. To address this issue, we propose a novel audio-visual learning framework which is instantiated with two individual learning schemes: self-supervised predictive learning (SSPL) and semantic-aware contrastive learning (SACL). SSPL explores image-audio positive pairs alone to discover semantically coherent similarities between audio and visual features, while a predictive coding module for feature alignment is introduced to facilitate the positive-only learning. In this regard SSPL acts as a negative-free method to eliminate false negatives. By contrast, SACL is designed to compact visual features and remove false negatives, providing reliable visual anchor and audio negatives for contrast. Different from SSPL, SACL releases the potential of audio-visual contrastive learning, offering an effective alternative to achieve the same goal. Comprehensive experiments demonstrate the superiority of our approach over the state-of-the-arts. Furthermore, we highlight the versatility of the learned representation by extending the approach to audio-visual event classification and object detection tasks. Code and models are available at: https://github.com/zjsong/SACL.

The Design of Sound Source Location and Tracking System Based on TDOA

The Design of Sound Source Location and Tracking System Based on TDOA

Hybrid deconvolution method based on mode composition beamforming for separating sound sources with different motion modes

The presence of sound sources that rotate can cause interference with stationary acoustic imaging, especially if the intensity of the rotating sound source is strong. Vice versa, stationary sound sources can also interfere with rotational acoustic imaging. Current algorithms for separating and locating rotating and stationary sound sources have problems such as weak separation capabilities, complex calculations, and limited array layouts. To address these challenges more effectively, a frequency-domain hybrid deconvolution method based on modal composition beamforming is proposed to separate and localize rotating and stationary sound sources. This algorithm operates within the frequency domain and is not restricted by array layout or the tonal single-frequency nature of sound sources. First, the point spread function and cross-point spread function matrices of the array are derived based on the modal transfer function. Then, a system of linear equations is constructed to separate rotating and stationary sound sources. Finally, the sound source separation and localization problems are solved through Gaussian Seidel and sparse-constrained deconvolution, respectively. Compared with the virtual rotating array method combined with matrix completion, simulation and experimental results demonstrate that this method can accurately locate and separate rotating and stationary sound sources, even when there is a substantial difference in intensity between rotating and stationary sound sources.

The influence of affective voice on sound distance perception.

Affective stimuli in our environment indicate reward or threat and thereby relate to approach and avoidance behavior. Previous findings suggest that affective stimuli may bias visual perception, but it remains unclear whether similar biases exist in the auditory domain. Therefore, we asked whether affective auditory voices (angry vs. neutral) influence sound distance perception. Two VR experiments (data collection 2021-2022) were conducted in which auditory stimuli were presented via loudspeakers located at positions unknown to the participants. In the first experiment (N = 44), participants actively placed a visually presented virtual agent or virtual loudspeaker in an empty room at the perceived sound source location. In the second experiment (N = 32), participants were standing in front of several virtual agents or virtual loudspeakers and had to indicate the sound source by directing their gaze toward the perceived sound location. Results in both preregistered experiments consistently showed that participants estimated the location of angry voice stimuli at greater distances than the location of neutral voice stimuli. We discuss that neither emotional nor motivational biases can account for these results. Instead, distance estimates seem to rely on listeners' representations regarding the relationship between vocal affect and acoustic characteristics. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Fiber-optic bionic microphone based compact sound source localization system with extended directional range.

Traditional sound source localization (SSL) systems based on electret condenser microphone arrays are bulky because their localization accuracy depends on the size of the array. Inspired by the hearing mechanism of the parasitic fly Ormia ochracea, the localization accuracy of miniature bionic SSL devices breaks through the limitations of device size, but their ability to localize low-frequency sound sources over a wide angular range remains a challenge. In this work, a compact low-frequency SSL system with an extended directional range was prepared using two bionic micro-electro-mechanical system diaphragm based fiber-optic microphones, which form a non-coplanar array with a size of Φ44 mm × 13 mm. An algorithm for quantifying the azimuthal angle of a sound source is established for the prepared SSL system. Simulation and experimental results show that the prepared SSL system is capable of determining the propagation direction of acoustic signals with a frequency of less than 1 kHz in the azimuthal range from -90° to 90°, with a linear response in the range from -70° to 70°, and an angular measurement accuracy of the system within the range of ±7°.

Sound source localization based on data and neural network model

Abstract In this study, the integration of sound source localization with neural network techniques is proposed. Firstly, the development history and basic principle of neural networks are introduced. Combined with the characteristics of sound source location, the residual neural network is analyzed, and the residual neural network is used as the neural network model. The sound signal received by the microphone array is computed by PHAT-SCOT and the output is trained as the input feature of ResNet. Finally, the results of indoor sound source location are obtained in the test set. Experimental findings demonstrate that the neural network-based localization algorithm exhibits robust localization accuracy across diverse environmental conditions.