Sound Localization Research Articles

TSSL: Trusted Sound Source Localization

Despite significant advancements in the accuracy of sound source localization, the confidence and robustness of the results still require improvement. Conducting uncertainty estimation can effectively alleviate this problem, since it provides a measure of confidence in the sound source localization results. In this work, we propose a trusted sound source localization neural network which can generate the accurate localization results and reliable uncertainty estimations. In this approach, the subjective theory was employed for associating the Dirichlet distribution with the predictions obtained from the neural network, treating them as subjective opinions. This allows us to model the overall uncertainty by parameterizing the class probabilities of sound source position using a Dirichlet distribution. Moreover, the reliable evidence supporting for the robust and reliable sound source localization results can also be gathered by the proposed method. In sum, this approach enhances the robustness of the neural network when facing with the out-of-distribution samples. To comprehensively evaluate the proposed method in the aspect of accuracy and robustness, extensive experiments were conducted on both simulated and real-world datasets. The proposed method outperforms other competing methods in the performance of robustness and reliability.

Neural-network-based sound source localization from unmanned aerial vehicles

As a mobile platform, unmanned aerial vehicles (UAV) can carry optics and acoustics sensors recording visual and audio information from ground. Over the past decade, UAVs are widely used in different applications, e.g., search and rescue, pipe leakage detection, etc. Compared with visual information, audio information is challenging to capture from UAVs due to the strong ego-noise generated by the rotating motors, propellers on the UAV as well as the motion of the UAV, which result in a low signal to noise ratio (SNR) of the received acoustics signals. With the contaminated signals, the performance of sound source localization algorithms, e.g., conventional beamforming, is severely degraded. To improve the quality of sound source localization result, different sound source enhancement algorithms have been proposed by researchers. In this paper, simulation experiments were conducted to study the potential of apply convolutional neural networks (CNN) to improve the beamforming result. It was found that with CNN, the estimation of a single point sound source location and source strength can be improved. The model's localization accuracy and source strength estimation accuracy were evaluated respectively.

Moving sound source localization in 3D space based on time delay and energy ratio technique

Moving sound source localization has numerous applications in interdisciplinary realms such as in teleconferencing, in defense, in robotics, and fault localization in vehicles. Localization of moving source is more challenging than localizing stationary sources because the motion disturbs the sound field and manipulates the frequency and amplitude of the acoustic pressure at recording location, which hampers the accuracy of source localization. Existing moving source localization techniques have numerous restrictions such as source trajectory should be known beforehand as a straight line perpendicular to the array geometry, source should be in same plane or parallel to the microphone array plane, and source velocity should be known. Moreover, when the source elevation is high, these techniques give high error. This study proposes the time delay and energy ratio based real-time moving sound source localization technique that is free from above-mentioned restrictions and can track the source's 3D coordinates accurately. Six microphones are used to localize a moving source by processing the time delay and energy ratios between sensor pairs. The root mean square error found for low source speed and high source speed conditions are 0.73 m and 1.8 m respectively.

Validation of sound source localization quality in binaural audio synthesis with head-tracking

Binaural synthesis with head tracking has become a highly important tool in creating an auditory experience for users that is as similar as possible to their perception of sound in the real world. This is crucial in many different applications that rely on accurate representation of real or virtual sound environments, ranging from scientific research to gaming applications. The quality of the binaural synthesis process and the accuracy and naturalness of the produced result depend on numerous factors, ranging from the selection of suitable IMU sensor technology used for head tracking to the choice of methods for implementing binaural synthesis using generic or individual sets of Head-Related Transfer Functions (HRTF). This is especially significant in Virtual Reality (VR) systems with three or six degrees of freedom that fully immerse the users in the virtual world through visual and auditory stimuli. This paper presents the authors' experiences in addressing a variety of challenges that arose during the design of such spatial audio systems and the testing of sound source localization quality. Strong emphasis is put on the testing of sound source localization quality, where the position of the source dynamically changes in relation to the listener.

Designing a practical physical framework for AI-enhanced sound detection and localization in vehicles

This paper presents the practical framework of an AI-implemented system currently in development for the detection and localization of external sound events to the vehicle. We delve into the initial block of the automotive project, analyzing sounds of interest and the noise we encounter, addressing the strategic arrangement of the four microphones on the sides of the vehicle, conceived as the sound input system. This configuration not only optimizes signal capture but also acts as sensitive ears, allowing the system to effectively analyze the acoustic environment. To mitigate wind interference, we have designed a microphone placement system that considers the vehicle's aerodynamics. The strategic location and implementation of aerodynamic protectors significantly minimize the impact of wind on the microphones, ensuring a sharper capture of acoustic signals. Complementing this arrangement, we apply advanced noise elimination processes, further enhancing the quality of the captured signals. Although in its early stages, this project aspires to provide effective solutions for vehicular acoustic perception, marking the beginning of a journey towards innovation in the detection and localization of sounds in mobile environments.

Markov chain-based temporal correlation processing algorithm in reverberant environment by sparse Bayesian learning for acoustic imaging

Identifying sound sources is an important step in noise control, different sound source localization algorithms have been proposed and studied for almost a century. Most of the studies assume an ideal measurement environment, i.e., an anechoic or half-anechoic environment, but there are limited studies discussing localizing sound sources in a reverberation environment. This paper proposes an acoustic imaging method in a reverberation environment based on the sparse Bayesian learning framework. Initially, a sound propagation model was established by accounting for the multipath effects of the sound source in a reverberant environment. Based on the temporal correlation of a Markov chain, the algorithm is further improved in the sparse Bayesian learning framework. By leveraging the temporal correlation the algorithm provides, identifying the true source in a reverberant environment becomes more accurate. Numerical simulation results have shown that the proposed algorithm achieves real sound source localization and eliminates the influence of image sources in contour maps under realistic reverberant environments.

AES67 based microphone array for sound source localization

Within the research project EAV-Infra, a novel microphone array system was developed. The system stands out for its utilization of Ethernet-based signal, control data and power transmission, ensuring a high degree of flexibility in operation. The key feature is its ability to facilitate real-time multichannel audio streaming via AES67, allowing for scalable and distributed recording and computing. With a total of 64 digital miniaturized microphones integrated into its printed circuit board-based design, the system allows for analysis of audio data for beamforming and other sound localization mechanisms. Adherence to audio and networking standards ensures synchronized transmission of audio data over packet-based networks. The system's emphasis on scalability, flexibility, and affordability makes it well-suited for a wide range of applications in fields like acoustic metrology and the entertainment industry, providing a comprehensive solution for high-quality audio processing needs across diverse domains.

Evaluating the relationship between high-frequency hearing loss and front-back localization in the elderly

Sound localization is an essential ability for normal hearing listeners, aiding in the localization of objects and the detection of potential dangers, as well as enabling the extraction of target speech information in a noisy environment. However, high-frequency hearing loss, commonly observed in the elderly population, may lead to a decline in spatial localization ability. The present work evaluated the ability of horizontal localization of 60 elderly individuals in a psychoacoustic experiment and investigated the relationship between localization impairment and hearing loss in different frequency ranges. Results indicate that high-frequency hearing loss above 2 kHz has a minor impact on left-right localization, but significantly impairs front-back localization. The physical and psychoacoustic explanations are provided for the experimental observations.

Development of low-cost MEMS microphone array for sound localization in urban environments

Understanding noise pollution and mitigating its impact on the quality of life in urban environment is a major challenge from a societal and health point of view. In order to monitor and ultimately understand these noisy environments a process of long term acoustic measurements and analysis is required. These urban environments are noisy and obtaining a clear recording of a desired acoustic signal is difficult hence an effective approach is the use of beamforming theory coupled with a microphone array. In this paper a novel approach utilizing compact low-cost microphone arrays equipped with MEMS sensors for continuous sound localization in urban areas is proposed. This approach focuses on leveraging the affordability and scalability of MEMS sensors to deploy distributed acoustic sensor networks across diverse urban locations. The aim is to capture real-time acoustic data, enabling comprehensive monitoring of sound sources and their spatial distribution within the urban environment. This sensor unit can be upgraded with a camera, i.e. acoustic camera, which will provide information on the conditions of the environment where the measurements are performed. This low-cost solution facilitates long-term monitoring, providing valuable insights into the dynamic nature of urban soundscapes.

Evaluating on the spatial localization cues of binaural sound pressure recorded via behind-the-ear microphones in a hearing aid

Spatial hearing ability enables people with normal hearing to localize a sound source and utilizes the cocktail party effect to detect the target speech information in a noise environment. Hearing-impairment causes not only hearing loss but also the degradation of spatial auditory ability. Hearing aid is desired to improve the spatial auditory ability of some hearing-impaired people, in addition to compensating hearing loss. However, the microphones for conventional behind-the-ear (BTE) hearing aid are located behind the pinnae rather than in the entrances of ear canal, which may cause distortion in spatial cues in the recorded binaural signals. The present work comprehensively evaluates the spatial hearing cues recorded by BTE-microphones. The head-related transfer functions (HRTF) for a KEMAR artificial head and eight human subjects, at three BTE positions and the entrances of blocked ear-canal, are measured respectively. Analysis from the HRTF data indicates that BTE microphone recording basically reserves the interaural time difference and interaural level difference as lateral-direction information. It also reserves the dynamic cues, but obvious distortions appear in the spectral cues at high frequencies. However, due to the redundancy among different spatial information, it can be inferred that dynamic cues alone may provide appropriate front-back and vertical-direction information.

A method of estimating degree of influence from wind for improving sound source localization in windy place

In this paper, we propose a method for visualizing the influence of wind on sound propagating in a room where wind is blowing, and rating the degree of influence. Since non-uniform airflow affects sound propagation and gives time-variant characteristics to the transfer function, a method is needed to understand the influence of wind. Therefore we propose a method that measures impulse responses multiple times indoors, determines a similarity matrix between impulse responses, and visualizes it as a heatmap. We evaluate the proposed method in four types of average wind velocity (0.0 m/s, 2.7 m/s, 3.3 m/s, 3.9 m/s). The result of the evaluation showed that the greater the influence of the wind, the higher the value of diagonal elements of the similarity matrix than off-diagonal elements, and the tendency for diagonal elements to be emphasized. We also confirm that the rating based on the Frobenius norm of the similarity matrix takes a lower value as the influence of the wind is greater.

Effect of background noise and diffraction on localization of sound sources using distributed acoustic measurement equipment

This study is focused on the development of a system designed to pinpoint the positions of sound sources within a plane through the analysis of cross-correlated signals obtained from distributed microphones. Our investigation delved into assessing the impact of physical barriers and background noise on the accuracy of the proposed method for localizing sound sources. Prior research has demonstrated that measurements conducted within an anechoic chamber yield estimations accurate to within ±20 mm. When the signal-to-noise ratio of the target sound source is reduced by the background noise, the estimation is performed at the point of background noise generation. Furthermore, the introduction of physical barriers between the sound source and the microphones resulted in sound diffraction, thereby leading to estimation errors attributable to the varying lengths of diffraction paths. These findings underscore the importance of considering environmental factors in sound source localization endeavors. The estimation at Mic, where the sound-receiving level drops below -10 dB due to physical barriers, confirmed that the background noise source point is estimated as well as the background noise estimation error.

Sound Source Localization at the Rolling Tire

A vehicle's tire rolling on pavement causes complex noise generating mechanisms which are a dominant sub-source of vehicle traffic noise. For the quantification of noise levels due to these tire-pavement interactions, standardized acoustic measurement methods exist, such as the Close-Proximity (CPX) method. These methods require microphones mounted near a rolling tire. However, they do not provide insight in the distribution and amplitudes of present sound sources. In this contribution, we aim at identifying the dominant sound sources of the rolling tire at discrete frequencies. This is accomplished by an inverse method combining microphone array measurements and Finite Element (FE) simulations. The microphone measurements were recorded on Austrian highways with a large measurement trailer built for research purposes. With the considered inverse method, not only the amplitudes of the dominant sound sources but also their phases can be identified. With these identified sources and a validated FE model of the measurement trailer, the sound pressure field within the trailer may be computed. The thereby calculated sound pressure is compared to measurements at the CPX microphone positions and other locations within the trailer.

On improving sound source localization in the wind tunnel

When traveling in an open-jet wind tunnel, the path of an acoustic wave is affected by the flow causing a shift of source positions in acoustical maps of phased arrays outside the flow. In this paper, we start by comparing several well-known approaches to correct travel times between microphones and assumed sources, used, for example, by beamforming algorithms in such an environment. The methods under consideration include the original 1D-Amiet/Bahr formulation, a standard 2.5D-planar approach that assumes the constancy of angular frequency and wave-number-vector components along a plane, and finally, a 3D-ray-tracing method. Common to the former two of these methods is the assumption that the boundary layer of the flow, the so-called shear layer, is infinitely thin and refracts the acoustical ray, whereas, in principle, the latter algorithm allows for an arbitrary (but sufficiently smooth) vector field modeling the flow. Going through these methods, we discuss travel-time results relative to each other and in terms of differences in source localization using beamforming maps. In particular, we model the flow of an existing wind tunnel and discuss results of real-world measurements comparing the 2.5D-planar approach with the 3D-ray-tracing method.

Assessment of planar array accuracy for 3D sound source reconstruction with Bayesian Focusing

As planar microphone arrays were initially designed for bidimensional sound source localisation, their use for three-dimensional acoustic imaging consistently suffers from a poor resolution along the normal axis. Such a statement is already well documented for beamforming approaches and their declensions. Bearing this in mind, this paper delves into the possibilities offered by inverse methods and especially iterative Bayesian Focusing to tackle this issue and deliver accurate source positions in three dimensions based on planar array measurements only. To this end, an experimental set up is designed: omnidirectional sources are mounted in a wooden mock-up featuring faces at different distances from an 81 MEMs planar array. Based on this measurements, a benchmark study of sound source reconstruction on a 3D mesh of the mock-up is conducted and compared to classical 2D acoustic maps set at different depths. The latter process is declined for various source correlation values and at extended frequency ranges. By doing so, the current paper delivers an overview of the ability to discriminate acoustic sources along the normal axis of a planar array, on a simple and well controlled test case, with a view to provide guidelines for further applications of 3D acoustic imaging on complex geometries.

Study of speech intelligibility in anechoic and reverberant environments using Slovak language

Binaural aspects play a significant role in speech intelligibility, with listeners demonstrating better performance in scenarios where the signal and noise sources are spatially separated, leveraging interaural time and level differences for improved sound segregation and localization. Thus, the question is: How do binaural aspects influence speech intelligibility in different spatial configurations of signal and noise? In this paper, speech intelligibility is examined in an anechoic and reverberant environment, specifically focusing on three configurations of signal and noise: (1) signal and noise both coming from the front at a distance of 1-meter, (2) signal from the front and noise from the right side (90 degrees) at a 1-meter distance, and (3) signal from the front and noise from the back (180 degrees) at a 1-meter distance. The study employs two different noises-speech-weighted noise and multi-talker noise-during listening tests conducted in the Slovak language. Furthermore, the study compares the outcomes with a similar experiment conducted a few years ago in the Dutch language, providing valuable insights into language-specific variations in speech perception under different acoustic environments.

Mimicking Human Hearing with Topologically Optimized Structure

The remarkable auditory capabilities of human ears have inspired the development of bio-inspired acoustic sensing technologies. In this study, a topology optimization problem is formulated to replicate the hearing profile of the human ear and is solved using a gradient-based optimization method. To ensure manufacturability and mesh independence, a density filter is employed to eliminate gray scale transitions between solid and void regions. The objective function is defined to minimize the error between the human's Head-Related Transfer Function (HRTF) of a reference structure and the designed domain in specific frequencies and directions. Numerical simulations demonstrate that the optimized structure exhibits similar acoustic characteristics and spatial directivity to that of the ideal human ear at the designated frequencies. The proposed structure and optimization method show great potential for applications such as 3D audio rendering, hearing- aid design, and sound source localization.

How sighted people experience the sense of volume? A phenomenal study

This study aims to provide a phenomenological description of the sense of volume in sighted people. Especially developed by non-sighted people, the sense of volume is an ability to perceive solids and voids by analysing sound reflection. Our recent works have brought it closer to the human echolocation's ability, but also to a form of spatial hearing based on the analysis of variations in the ambient sound field. In many studies, blindfolded participants were able to detect a surface positioned in front of them. However, there are no studies reporting on the experience of the sense of volume in sighted people under conditions of ecological perception. Consequently, we carried out commented walks, combined with a micro-phenomenological interview method in an urban environment. By cross-referencing participants' perceptions (wall detection at one metre, typologies' discrimination of) with feelings they share (acoustic feelings combined with synesthetic tactile feelings), we demonstrated the ability of sighted people to rapidly unveil a functional sense of volume. Several researchers have hypothesized that sighted people may use a form of automated passive echolocation. In the light of our results, we wonder whether the sense of volume might be a non-conscious factor in an urban atmosphere.

Loss of C1q alters the auditory brainstem response.

Neural circuits in the auditory brainstem compute interaural time and intensity differences used to determine the locations of sound sources. These circuits display features that are specialized for these functions. The projection from the ventral cochlear nucleus (VCN) to the medial nucleus of the trapezoid (MNTB) body travels along highly myelinated fibers and terminates in the calyx of Held. This monoinnervating synapse emerges during development as multiple inputs are eliminated. We previously demonstrated that elimination of microglia with a colony stimulating factor-1 inhibitor results in impaired synaptic pruning so that multiple calyceal terminals reside on principal cells of MNTB. This inhibitor also resulted in impaired auditory brainstem responses (ABRs), with elevated thresholds and increased peak latencies. Loss of the microglial fractalkine receptor, CX3CR1, decreased peak latencies in the ABR. The mechanisms underlying these effects are not known. One prominent microglial signaling pathway involved in synaptic pruning and plasticity during development and aging is the C1q-initiated compliment cascade. Here we investigated the classical complement pathway initiator, C1q, in auditory brainstem maturation. We found that C1q expression is detected in the MNTB by the first postnatal week. C1q levels increased with age and were detected within microglia and surrounding the soma of MNTB principal neurons. Loss of C1q did not affect microglia-dependent calyceal pruning. Excitatory and inhibitory synaptic markers in the MNTB and LSO were not altered with C1q deletion. ABRs showed that C1q KO mice had normal hearing thresholds but shortened peak latencies. Altogether this study uncovers the developmental time frame of C1q expression in the sound localization pathway and shows a subtle functional consequence of C1q knockdown.

The effect of right versus left long-term single-sided deafness on sound source localization

The effect of right versus left long-term single-sided deafness on sound source localization