Sound Source Localization Accuracy Research Articles

TSSL: Trusted Sound Source Localization

Despite significant advancements in the accuracy of sound source localization, the confidence and robustness of the results still require improvement. Conducting uncertainty estimation can effectively alleviate this problem, since it provides a measure of confidence in the sound source localization results. In this work, we propose a trusted sound source localization neural network which can generate the accurate localization results and reliable uncertainty estimations. In this approach, the subjective theory was employed for associating the Dirichlet distribution with the predictions obtained from the neural network, treating them as subjective opinions. This allows us to model the overall uncertainty by parameterizing the class probabilities of sound source position using a Dirichlet distribution. Moreover, the reliable evidence supporting for the robust and reliable sound source localization results can also be gathered by the proposed method. In sum, this approach enhances the robustness of the neural network when facing with the out-of-distribution samples. To comprehensively evaluate the proposed method in the aspect of accuracy and robustness, extensive experiments were conducted on both simulated and real-world datasets. The proposed method outperforms other competing methods in the performance of robustness and reliability.

Improved Binaural Hearing Effects and Sound Source Localization in Listeners with Simulated Unilateral Conductive Hearing Loss: Effect of a Bone Conduction Device.

Objectives: It has been proven that patients with unilateral conductive hearing loss (UCHL) may encounter typical problems associated with asymmetric hearing, especially in challenging listening environments. In this study, we aimed to determine how UCHL affects speech recognition under multisource competing environments and the ability of sound source localization, as well as whether assistance with a bone conduction device (BCD) can confer hearing benefits in such listening tasks. Design: Acquired UCHL was simulated using an earplug combined with an earmuff in 10 listeners (mean age: 29.9 ± 4.77 years) with bilateral normal hearing (NH), and a within-subject repeated-measures design was used to compare hearing results among three listening conditions: NH (both ears open, C1), unilateral plug (UP) (simulated UCHL, C2), and UP + BCD (simulated UCHL aided with a BCD, C3). The speech reception threshold (SRT) of summation, squelch, and head shadow (HS) effects and the mean absolute error of sound source localization were used as markers for binaural hearing abilities. Results: BCD assistance (C3) improved the summation and HS effects in all listeners with simulated UCHL, resulting in a lower (i.e., better) SRT than that observed in C2; however, no significant differences in squelch effects were observed between C2 and C3. Notably, most listeners exhibited more accurate sound source localization in C3 than in C2. Further, BCD assistance mainly improved localization accuracy when the noise stimuli were presented at low intensities and on the hearing-impaired (plugged) side, suggesting that the benefits of BCD for sound localization are not based on the reacquisition of binaural processing. Conclusions: The current results have clinical implications for the promotion of BCDs in patients with UCHL, especially those with acquired UCHL who are unable to undergo surgery.

Application of adaptive finite focus beamforming method for localization of low-frequency transformer sound

In this paper, the developed adaptive linearly constrained minimum variance finite focus beamforming method is used to increase the accuracy and resolution of low-frequency transformer sound source localization. In the first step, simulations on virtual arrays are conducted to define the array size and compare the influence of three different beamforming algorithms on sound localization accuracy and resolution. In the second step, measurements of the regulating transformer as a noise source are conducted in the semi-anechoic chamber. A large rectangular microphone array, with 90 microphone positions spaced 0.44 meters apart, is used for that purpose. The developed algorithm is compared with infinite and finite focus beamforming for the localization of total noise and noise at specific noise harmonics.

Deep Learning-Based Dereverberation for Sound Source Localization with Beamforming

In this work, an algorithm that combines deep-learning based dereverberation and beamforming is proposed for sound source localization in the reverberant environment. The contribution is combining deep-learning based dereverberation and beamforming together. Through deep learning, the proposed algorithm can directly achieve dereverberation of the cross-spectral matrix of microphone array measurement signals in the reverberant environment, which can overcome the challenge of requiring multiple measurements like average beamforming. In this way, the proposed algorithm can be applied to the reverberant environment that requires efficient sound source localization. At the same time, the network of deep-learning based dereverberation is trained directly using the cross-spectral matrix of signals collected by microphone arrays, which can overcome the challenge of requiring prior knowledge of reflective surfaces like empirical dereverberation beamforming. In this way, the proposed algorithm can be applied to the reverberant environment composed of walls with unknown sound reflection coefficients. Both specular reflections, diffuse reflections, and background noise are considered in the reverberant environment. The cross-spectral matrix of microphone array measurement signals in the reverberant environment is first fed into the U-net for dereverberation and then combined with beamforming for sound source localization. The results in the test set show that the proposed algorithm can eliminate the influence of reverberation on sound source localization. The applicability limitation results of the proposed algorithm show the proposed algorithm’s strong robustness to unseen sound reflection coefficient, unseen SNR, and expandability to some degree for unseen source location. However, the proposed algorithm cannot provide stable and accurate sound source localization results under unseen room geometry. The proposed algorithm has higher spatial resolution and smaller errors than the average beamforming and the empirical dereverberation beamforming.

How does Layer Normalization improve Batch Normalization in self-supervised sound source localization?

Self-supervised sound source localization is usually challenged by the unexpected large input and incorrect direction of normalization in current solutions. A promising way for this challenge is to avoid feature deformation by incorporating more effective normalization, which is the motivation of this study. Based on the mathematical derivation of Layer Normalization (LN) in scale independence, in this work, a correspondence consolidation method is proposed to reinforce the audio–visual correspondence. By ensembling input feature normalization and LN-based simsiam Predictor, a joint gradient stabilization can be further achieved for more accurate sound source localization. Substantial experiments conducted on SoundNet-Flickr and VGG-Sound Source datasets have verified a superior performance in comparison to the other state-of-the-art works.

A novel position estimation method for wayside pass-by noise sources based on Doppler effect correction

The localization of moving noise sources has garnered substantial attention in recent years, prompting researchers to develop a range of methodologies for positioning sound sources across diverse scenarios. Despite this progress, the intricate interplay of complex spatial sound fields and the Doppler effect during motion presents a formidable challenge in achieving accurate sound source localization. To address these challenges, this paper proposes a novel method for estimating positions of wayside pass-by noise sources based on Doppler Effect Correction (Do-bPE). The proposed method capitalizes on a spatial microphone array to capture signals emanating from moving sound sources. Each channel’s sound signal is transformed into the time–frequency domain using Short Time Fourier Transform (STFT). The Approximate Greatest Common Divisor (AGCD) algorithm is then applied to extract the instantaneous frequency of the moving source from the time–frequency spectrogram. By leveraging nonlinear least squares fitting and the acoustic Doppler model, this instantaneous frequency facilitates the kinematic parameters estimation of Doppler effect, which provides the distances between each microphone within the array and the trajectory of moving sound source. Then, by employing the weighted least squares method to solve the relationship between the sound source and the array microphones, an accurate spatial position for the sound source can be obtained. The method successfully applies the Doppler effect to estimate the source distances, subsequently refining position estimation accuracy through microphone array processing. Simulation and experimental results show that the method’s positioning accuracy and robustness are significantly improved compared to existing approaches.

Shift-variant deconvolution beamforming using Implicit Neural Representation for near-field acoustic image measurement

The conventional beamforming (CBF) output can be expressed as a convolution of the source distribution and the array dependent point spread function (PSF), which is defined as the response of the beamformer to a point source. The resolution and accuracy of sound source localization can be improved by deconvolution of CBF. The shift-invariant PSF assumption is only a good approximation when the source region is small compared to the distance between the array and the source. However, the PSF of the near-field underwater acoustic map measurement is shift-variant, whose mismatch with the beam could lead to performance degradation. In this paper, we proposed an optimized deconvolution method by using the neural network called implicit neural representation (INR) to solve the beam mismatch problem, due to its strong performance in learning and reconstructing spatial scenes. Rather than recompute the PSF for each pixel in the scene, INR predicts the complex-valued weight matrix in two-dimensional space that encapsulates both the spatially varying properties of the PSF and the scatter distribution of the scene. Numerical simulations and experimental results verify the effectiveness of our method. And this work can be broadly applied to arrays with shift-variant PSFs.

Orthogonal matching pursuit algorithm based on weighted cosine similarity for sound source localization

Abstract Compressive sensing overcomes the limitations of the Nyquist criteria and is one of the most widely used compressive sensing reconstruction algorithms. Orthogonal matching pursuit (OMP) algorithm is simple, in terms of hardware implementation, and has high computational efficiency. However, the OMP algorithm exhibits poor identification performance for low-frequency sound sources and results in large localization deviations when the mesh spacing of the focus plane is small. In this study, a novel atom selection criterion based on weighted cosine similarity was proposed to improve the OMP algorithm for sound source localization and characterization. This method replaces the original inner product criterion to measure the correlation between the column vectors of the sensing matrix and the residuals, which addresses the atom selection error caused by the high correlation between atoms. Numerical simulations and experimental results show that the proposed method has a stronger anti-noise interference capability and higher accuracy for sound source identification with fewer sampling points, particularly in low-frequency and low signal-to-noise ratio environments. Compared to other OMP algorithms, the proposed method improves the performance of the OMP algorithm in sound source localization and widens the sound frequency range. This study is valuable for achieving highly accurate sound source localization and reducing measurement costs in practical applications.

Research on structural sound source localization method by neural network

To solve problems related to much calculation to adapt to complex scenes in traditional structural sound source localization, this paper proposes a method based on neural network. The structural sound source at other positions was stimulated by successively striking 36 grid centers on the surface of the plate. The time delay between different accelerometer signals was considered as the input, and the location of the predicted sound source was considered as the output. The influence of the number of test sets and epoch training times on sound source localization accuracy was discussed. These results show that with the increase in the epoch training times, the number of test set decreases, and the number of training set increases, increasing the sound source localization accuracy of backpropagation neural network. However, these error conditions will frequently appear due to the overfitting phenomenon. When the epoch is trained to 50,000 times, and the quantity of the test set is 4, the backpropagation neural network has the best localization accuracy with an order of magnitude of 10−3 in error, and the localization error scope of the plate is between 0.01 and 0.1 m.

The Accuracy of Dynamic Sound Source Localization and Recognition Ability of Individual Head-Related Transfer Functions in Binaural Audio Systems with Head Tracking

The use of audio systems that employ binaural synthesis with head tracking has become increasingly popular, particularly in virtual reality gaming systems. The binaural synthesis process uses the Head-Related Transfer Functions (HRTF) as an input required to assign the directions of arrival to sounds coming from virtual sound sources in the created virtual environments. Generic HRTFs are often used for this purpose to accommodate all potential listeners. The hypothesis of the research is that the use of individual HRTF in binaural synthesis instead of generic HRTF leads to improved accuracy and quality of virtual sound source localization, thus enhancing the user experience. A novel methodology is proposed that involves the use of dynamic virtual sound sources. In the experiments, the test participants were asked to determine the direction of a dynamic virtual sound source in both the horizontal and vertical planes using both generic and individual HRTFs. The gathered data are statistically analyzed, and the accuracy of localization is assessed with respect to the type of HRTF used. The individual HRTFs of the test participants are measured using a novel and efficient method that is accessible to a broad range of users.

Effect of tinnitus on sound localization ability in patients with normal hearing

ObjectivesTo determine whether tinnitus negatively impacts the accuracy of sound source localization in participants with normal hearing. MethodsSeventy-five participants with tinnitus and 74 without tinnitus were enrolled in this study. The accuracy of sound source discrimination on the horizontal plane was compared between the two participant groups. The test equipment consisted of 37 loudspeakers arranged in a 180° arc facing forward with 5° intervals between them. The stimuli were pure tones of 0.25, 0.5, 1, 2, 4, and 8kHz at 50dB SPL. The stimuli were divided into three groups: low frequency (LF: 0.25, 0.5, and 1kHz), 2kHz, and high frequency (HF: 4 and 8kHz) stimuli. ResultsThe Root Mean Square Error (RMSE) score of all the stimuli in the tinnitus group was significantly higher than that in the control group (13.45±3.34 vs. 11.44±2.56, p=4.115, t<0.001). The RMSE scores at LF, 2kHz, and HF were significantly higher in the tinnitus group than those in the control group (LF: 11.66±3.62 vs. 10.04±3.13, t=2.918, p=0.004; 2kHz: 16.63±5.45 vs. 14.43±4.52, t=2.690, p=0.008; HF: 13.42±4.74 vs. 11.14 ±3.68, t=3.292, p=0.001). Thus, the accuracy of sound source discrimination in participants with tinnitus was significantly worse than that in those without tinnitus, despite the stimuli frequency. There was no difference in the ability to localize the sound of the matched frequency and other frequencies (12.86±6.29 vs. 13.87±3.14, t=1.204, p=0.236). Additionally, there was no correlation observed between the loudness of tinnitus and RMSE scores (r=0.096, p=0.434), and the Tinnitus Handicap Inventory (THI) and RMSE scores (r=−0.056, p=0.648). ConclusionsOur present data suggest that tinnitus negatively impacted sound source localization accuracy, even when participants had normal hearing. The matched pitch and loudness and the impact of tinnitus on patients’ daily lives were not related to the sound source localization ability. Level of evidence4.

Clinical Feasibility and Familiarization Effects of Device Delay Mismatch Compensation in Bimodal CI/HA Users.

Subjects utilizing a cochlear implant (CI) in one ear and a hearing aid (HA) on the contralateral ear suffer from mismatches in stimulation timing due to different processing latencies of both devices. This device delay mismatch leads to a temporal mismatch in auditory nerve stimulation. Compensating for this auditory nerve stimulation mismatch by compensating for the device delay mismatch can significantly improve sound source localization accuracy. One CI manufacturer has already implemented the possibility of mismatch compensation in its current fitting software. This study investigated if this fitting parameter can be readily used in clinical settings and determined the effects of familiarization to a compensated device delay mismatch over a period of 3-4 weeks. Sound localization accuracy and speech understanding in noise were measured in eleven bimodal CI/HA users, with and without a compensation of the device delay mismatch. The results showed that sound localization bias improved to 0°, implying that the localization bias towards the CI was eliminated when the device delay mismatch was compensated. The RMS error was improved by 18% with this improvement not reaching statistical significance. The effects were acute and did not further improve after 3 weeks of familiarization. For the speech tests, spatial release from masking did not improve with a compensated mismatch. The results show that this fitting parameter can be readily used by clinicians to improve sound localization ability in bimodal users. Further, our findings suggest that subjects with poor sound localization ability benefit the most from the device delay mismatch compensation.

Simultaneous Execution of Dereverberation, Denoising, and Speaker Separation Using a Neural Beamformer for Adapting Robots to Real Environments

It remains challenging for robots to accurately perform sound source localization and speech recognition in a real environment with reverberation, noise, and the voices of multiple speakers. Accordingly, we propose “U-TasNet-Beam,” a speech extraction method for extracting only the target speaker’s voice from all ambient sounds in a real environment. U-TasNet-Beam is a neural beamformer comprising three elements: a neural network for removing reverberation and noise, a second neural network for separating the voices of multiple speakers, and a minimum variance distortionless response (MVDR) beamformer. Experiments with simulated data and recorded data show that the proposed U-TasNet-Beam can improve the accuracy of sound source localization and speech recognition in robots compared to the conventional methods in a noisy, reverberant, and multi-speaker environment. In addition, we propose the spatial correlation matrix loss (SCM loss) as a loss function for the neural network learning the spatial information of the sound. By using the SCM loss, we can improve the speech extraction performance of the neural beamformer.

A grid-free global optimization algorithm for sound sources localization in three-dimensional reverberant environments

In this work, a new grid-free global optimization algorithm that combined the image source model(ISM) and state transition algorithm(STA) was proposed to realize the sound source localization in reverberant environments. The simulation results show that only first-order mirror sources being kept can ensure localization accuracy. Compared with de-reverberation beamforming, the localization results of the proposed algorithm have no sidelobes and smaller mainlobes. Compared with the differential evolution(DE) algorithm, STA can converge to smaller fitness values, which can achieve more accurate sound source localization in reverberant environments. Meanwhile, the proposed algorithm can be applied to both single-source and multi-source localization in practical scenarios. The experimental results show that the proposed algorithm can locate the sound source accurately in reverberant environments.

Spontaneous head movements support accurate horizontal auditory localization in a virtual visual environment.

This study investigates the relationship between auditory localization accuracy in the horizontal plane and the spontaneous translation and rotation of the head in response to an acoustic stimulus from an invisible sound source. Although a number of studies have suggested that localization ability improves with head movements, most of them measured the perceived source elevation and front-back disambiguation. We investigated the contribution of head movements to auditory localization in the anterior horizontal field in normal hearing subjects. A virtual reality scenario was used to conceal visual cues during the test through a head mounted display. In this condition, we found that an active search of the sound origin using head movements is not strictly necessary, yet sufficient for achieving greater sound source localization accuracy. This result may have important implications in the clinical assessment and training of adults and children affected by hearing and motor impairments.

Research on Underwater Sound Source Localization Based on NLM-EEMD and FCM-Generalized Quadratic Correlation

The accuracy of underwater sound source localization depends on the preprocessing effect of signal denoising and delay estimation performance. In order to meet the needs of high-precision underwater positioning, this article studies a time delay estimation method based on nonlocal mean-enforced mean empirical mode decomposition (NLM-EEMD) autocorrelation denoising preprocessing and fuzzy C-means clustering (FCM)-generalized quadratic for underwater sound source localization technology. The theories such as NLM, EEMD, FCM, and generalized quadratic correlation delay estimation are studied in detail, and the speech signal of the test library in the TIMIT standard library is selected for simulation analysis, which verifies the correctness of the delay estimation method. Experiments were carried out in a larger outdoor pool. The analysis results show that the NLM-EEMD adaptive denoising method effectively reduces noise interference and improves the quality of EEMD decomposition and the FCM-generalized quadratic correlation method increases the correlation between two signals. The time delay estimation accuracy is improved, and the positioning accuracy is 48.66% higher than that of the direct generalized cross-correlation method.

Optimization Algorithm for Delay Estimation Based on Singular Value Decomposition and Improved GCC-PHAT Weighting.

The accuracy of time delay estimation seriously affects the accuracy of sound source localization. In order to improve the accuracy of time delay estimation under the condition of low SNR, a delay estimation optimization algorithm based on singular value decomposition and improved GCC-PHAT weighting (GCC-PHAT- weighting) is proposed. Firstly, the acoustic signal collected by the acoustic sensor array is subjected to singular value decomposition and noise reduction processing to improve the signal-to-noise ratio of the signal; then, the cross-correlation operation is performed, and the cross-correlation function is processed by the GCC-PHAT- weighting method to obtain the cross-power spectrum; finally, the inverse transformation is performed to obtain the generalized correlation time domain function, and the peak detection is performed to obtain the delay difference. The experiment was carried out in a large outdoor pool, and the experimental data were processed to compare the time delay estimation performance of three methods: GCC-PHAT weighting, SVD-GCC-PHAT weighting (meaning: GCC-PHAT weighting based on singular value decomposition) and SVD-GCC-PHAT- weighting (meaning: GCC-PHAT- weighting based on singular value decomposition). The results show that the delay estimation optimization algorithm based on SVD-GCC-PHAT- improves the delay estimation accuracy by at least 37.95% compared with the other two methods. The new optimization algorithm has good delay estimation performance.

TDOA-Based Robust Sound Source Localization With Sparse Regularization in Wireless Acoustic Sensor Networks

Time difference of arrival (TDOA) measurements, which are contaminated by large values of error, known as outliers, would have a significant impact on the accuracy of sound source localization (SSL) in wireless acoustic sensor networks (WASNs). Few techniques are reported in the literature to tackle SSL in WASNs by taking TDOA outliers into consideration. To mitigate the effect of outliers on the accuracy of SSL, we propose outlier-resistant robust sound source localization (RSSL) algorithms based on sparse regularization using an unsynchronized network of microphone arrays. The TDOA errors are divided into two components: a) energy-bounded inliers and b) outliers. Assuming that outliers are sparse in the measurement set, we formulate the RSSL problem as that of minimizing the number of outliers, mathematically, a <inline-formula><tex-math notation="LaTeX">$\ell _{0}$</tex-math></inline-formula> (pseudo)-norm optimization problem with non-convex constraints. Five sub-optimal RSSL solvers are derived, among which the first two solvers are applicable to the scenario involving only outliers while the last three solvers concentrate on the scenario incorporating both outliers and inliers. In common, these solvers exploit a convex approximation technique called Concave Convex Procedure to dispose of the non-convex constraints. Differently, the first solver approximates the original <inline-formula><tex-math notation="LaTeX">$\ell _{0}$</tex-math></inline-formula> (pseudo)-norm cost function with the <inline-formula><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> norm while a concave surrogate function is adopted in the second solver to yield a tighter approximation to the <inline-formula><tex-math notation="LaTeX">$\ell _{0}$</tex-math></inline-formula> (pseudo)-norm. Apart from the application of these two approximation techniques, the third and fourth solvers relax the non-convex <inline-formula><tex-math notation="LaTeX">$\ell _{2}$</tex-math></inline-formula> norm constraint with the <inline-formula><tex-math notation="LaTeX">$\ell _{\infty }$</tex-math></inline-formula> norm. The fifth solver is dedicated to the <inline-formula><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> norm regularization problem with the Lasso formulation, which is equivalent to the M-estimator of Huber&#x2019;s function solved via the iteratively reweighted least squares paradigm. Experimental results validate the effectiveness and robustness of the proposed algorithms.

Phase calibration method for microphone array based on multiple sound sources

In the microphone array, the phase error of each microphone causes a deviation in sound source localization. At present, there is a lack of effective methods for phase error calibration of the entire microphone array. In order to solve this problem, a phase mismatch calculation method based on multiple sound sources is proposed. This method requires collecting data from multiple sound sources in turn, and constructing a nonlinear equation setthrough the signal delay and the geometric relationship between the microphones and the sound source positions. The phase mismatch of each microphone can be solved from the nonlinear equation set. Taking the single frequency signal as an example, the feasibility of the method is verified by experiments in a semi-anechoic chamber. The phase mismatches are compared with the calibration results of exchanging microphone. The difference of the phase error values measured by the two methods is small. The experiment also shows that the accuracy of sound source localization by beamforming is improved. The method is efficient for phase error calibration of arrays with a large number of microphones.

Deep learning-enhanced single point sound source localization for spherical microphone array

In this contribution, we present a high-resolution and accurate sound source localization via a deep learning framework. While the spherical microphone arrays can be utilized to produce omnidirectional beams, it is widely known that the conventional spherical harmonics beamforming (SHB) has a limit in terms of its spatial resolution. To accomplish the sound source localization with high resolution and preciseness, we propose a convolutional neural network (CNN)-based source localization model as a way of a data-driven approach. We first present a novel way to define the source distribution map that can spatially represent the single point source's position and strength. By utilizing paired dataset with spherical harmonics beamforming maps and our proposed high-resolution maps, we develop a fully convolutional neural network based on the encoder-decoder structure for establishing the image-to-image transformation model. Both quantitative and qualitative results are demonstrated to evaluate the powerfulness of the proposed data-driven source localization model.