Sound Source Localization System Research Articles

Three-dimensional sound source localization system based on fiber optic sensor array with an adaptive algorithm

Sound Source Localization (SSL) based on microphone arrays nowadays is widely used in the fields of machinery rattle localization, weapon navigation, and robot hearing. However, there is also a need for SSL in some special environments, such as long-distance gas pipelines where there is a risk of leakage. In these environments, arrays built with ordinary microphones can be a safety hazard, which can be compensated by the External Fabry-Perot interferometer (EFPI) fiber-optic sensor's advantages of being uncharged, inexpensive, and highly sensitive. To expand the application areas of microphone array-based SSL techniques, this paper using four EFPI fiber-optic sensors to constructs a 1 m × 1 m × 1 m sensor array, which is used for real-time localization of sound sources based on three-wavelength adaptive intensity demodulation methods and Time Difference Of Arrival (TDOA) localization techniques. An improved method is proposed for the three-wavelength adaptive intensity demodulation method to avoid the occurrence of signal flips. Finally, formulas are provided for this sensor array to quickly calculate the sound source position based on the arrival delay. Several experiments show that the system can achieve sound source localization over a wide range with an average error of less than 3 cm.

A Survey of Sound Source Localization and Detection Methods and Their Applications.

This study is a survey of sound source localization and detection methods. The study provides a detailed classification of the methods used in the fields of science mentioned above. It classifies sound source localization systems based on criteria found in the literature. Moreover, an analysis of classic methods based on the propagation model and methods based on machine learning and deep learning techniques has been carried out. Attention has been paid to providing the most detailed information on the possibility of using physical phenomena, mathematical relationships, and artificial intelligence to determine sound source localization. Additionally, the article underscores the significance of these methods within both military and civil contexts. The study culminates with a discussion of forthcoming trends in the realms of acoustic detection and localization. The primary objective of this research is to serve as a valuable resource for selecting the most suitable approach within this domain.

Closed-loop sound source localization in neuromorphic systems

Sound source localization (SSL) is used in various applications such as industrial noise-control, speech detection in mobile phones, speech enhancement in hearing aids and many more. Newest video conferencing setups use SSL. The position of a speaker is detected from the difference in the audio waves received by a microphone array. After detection the camera focuses onto the location of the speaker. The human brain is also able to detect the location of a speaker from auditory signals. It uses, among other cues, the difference in amplitude and arrival time of the sound wave at the two ears, called interaural level and time difference. However, the substrate and computational primitives of our brain are different from classical digital computing. Due to its low power consumption of around 20 W and its performance in real time the human brain has become a great source of inspiration for emerging technologies. One of these technologies is neuromorphic hardware which implements the fundamental principles of brain computing identified until today using complementary metal-oxide-semiconductor technologies and new devices. In this work we propose the first neuromorphic closed-loop robotic system that uses the interaural time difference for SSL in real time. Our system can successfully locate sound sources such as human speech. In a closed-loop experiment, the robotic platform turned immediately into the direction of the sound source with a turning velocity linearly proportional to the angle difference between sound source and binaural microphones. After this initial turn, the robotic platform remains at the direction of the sound source. Even though the system only uses very few resources of the available hardware, consumes around 1 W, and was only tuned by hand, meaning it does not contain any learning at all, it already reaches performances comparable to other neuromorphic approaches. The SSL system presented in this article brings us one step closer towards neuromorphic event-based systems for robotics and embodied computing.

A Sound Source Localization Method Based on Frequency Divider and Time Difference of Arrival

In recent years, sound source localization, as a passive localization technique with higher safety and convenience compared with other localization techniques such as active emission of electromagnetic waves, has received more and more attention in academia. This paper researches and improves the far-field sound source localization algorithm based on the generalized cross-correlation method (GCC) Time Difference of Arrival (TDOA) estimation algorithm and completes the design and implementation of the microphone array sound source localization system. This paper adds a frequency divider to the traditional generalized correlation time delay estimation algorithm for pre-processing, sampling, and localization of sound source acoustic waves and adopts a low-cost microphone array deployment scheme as far as possible to improve the flexibility and practicality of the localization system; at the same time, the “Minimum Sphere Method” is used at the back end of the algorithm to classify the localization coordinates at different frequencies and, finally, output reasonable sound source coordinates. In the back-end of the algorithm, the “Minimum Sphere Method” is used to classify the localization coordinates at different frequencies and, finally, output the reasonable sound source coordinates. The experimental results show that the sound source localization system designed in this paper has good performance in terms of localization accuracy and cost-effectiveness and overcomes the failure of the generalized mutual correlation algorithm in the original application of high noise environment and multi-source environment localization.

Sound source localization model of cube microphone array

AbstractThis paper conducts in‐depth research on the sound source localization technology of the microphone array and designs a sound source localization system based on the cube microphone array. Firstly, the sound source localization model is established using the cube microphone array combined with the spherical near‐field acoustic holography. Secondly, the numerical and sound source localization simulations are carried out using the spherical wave. Finally, the simulation and experiment of sound source location for sound at 100, 1000, and 2000 Hz are carried out using the model. Both simulation and experimental results show that when the sound source frequency is 100 and 1000 Hz, the location of the sound source can be accurately located by using the sound source localization model of a cube microphone array, and the sound field reconstruction error is low. When the sound source frequency is 2000 Hz, the location of the sound source cannot be located, and the sound field reconstruction error is very high, which will cause the misjudgment of the sound source location.

Stochastic Computing Design and Implementation of a Sound Source Localization System

Stochastic computing (SC) is an alternative computing paradigm that processes data in the form of uniform bit-streams. SC is fault-tolerant and can compute on small, efficient circuits. However, SC is primarily used in scientific research, and its practical implementations for end-users are rare. Digital sound source localization (SSL) is a useful signal processing technique that locates speakers using multiple microphones. SC has not been integrated into SSL in practice or theory. In this work, for the first time to the best of our knowledge, we implement an SSL algorithm in the stochastic domain and develop a functional SC-based sound source localizer. The practical part of this work shows that the proposed stochastic circuit does not depend on conventional analog-to-digital conversion and can process data in the form of pulse-width-modulated (PWM) signals. The proposed SC design consumes up to 39% less area than the conventional binary design. It can also consume less power depending on the computational accuracy, for example, 6% less power consumption for 3-bit inputs. We propose a new cross-correlation (CC) design based on the state-of-the-art Sobol bit-streams for further area and power saving. The proposed design utilizes a <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MUX</monospace> unit for bit-stream generation. It saves the area footprint up to 64% and the power consumption up to 82% compared to the counter-based SC design of CC, which relies on a comparator for bit-stream generation. The presented stochastic circuits, are not limited to SSL and are readily applicable to other practical applications such as radar ranging, wireless location, sonar direction finding, beamforming, and sensor calibration. The project’s source code is made available for public access.

Application of omnidirectional sound-source localization system for high portability and three-dimensional sound field recording

Application of omnidirectional sound-source localization system for high portability and three-dimensional sound field recording

Gaussian Process Regression for Single-Channel Sound Source Localization System Based on Homomorphic Deconvolution

To extract the phase information from multiple receivers, the conventional sound source localization system involves substantial complexity in software and hardware. Along with the algorithm complexity, the dedicated communication channel and individual analog-to-digital conversions prevent an increase in the system's capability due to feasibility. The previous study suggested and verified the single-channel sound source localization system, which aggregates the receivers on the single analog network for the single digital converter. This paper proposes the improved algorithm for the single-channel sound source localization system based on the Gaussian process regression with the novel feature extraction method. The proposed system consists of three computational stages: homomorphic deconvolution, feature extraction, and Gaussian process regression in cascade. The individual stages represent time delay extraction, data arrangement, and machine prediction, respectively. The optimal receiver configuration for the three-receiver structure is derived from the novel similarity matrix analysis based on the time delay pattern diversity. The simulations and experiments present precise predictions with proper model order and ensemble average length. The nonparametric method, with the rational quadratic kernel, shows consistent performance on trained angles. The Steiglitz-McBride model with the exponential kernel delivers the best predictions for trained and untrained angles with low bias and low variance in statistics.

A sound source localization method based on improved second correlation time delay estimation

The sound source localization (SSL) system based on the microphone array has important applications in audio and video conference, security monitoring and intelligent cockpit. However, the SSL method based on time difference of arrival is susceptible to ambient noise. Therefore, an improved second correlation delay estimation algorithm is proposed in this paper. The pure source signal is obtained by wavelet denoising, and then the time delay is calculated by the second correlation time delay estimation algorithm with the weighting functions of the smoothed coherence transform and the Roth processor. The position of the sound target is calculated from the time delay. Aiming at the moving target, an extended Kalman filter is introduced to track the moving trajectory of the sound source. The static and moving SSL simulations are conducted and the results of the proposed algorithm are compared with those of the single-weighted quadratic correlation (SQC) algorithm and the high-power quadratic correlation algorithm. The static sound source positioning errors of the proposed algorithm under −10 dB SNR are respectively 3.97 m and 5.86 m smaller than those of the HQC algorithm and the SQC algorithm. The moving SSL trajectory based on the proposed algorithm is still closest to the real track under −10 dB SNR. This indicates that the proposed algorithm has high precision and strong robustness for sound source location in the low signal-to-noise ratio (SNR) environment. In the experiment, the proposed algorithm can accurately calculate the direction of arrival (DOA) of static sound source and stably track DOA of moving sound source. This is consistent with the simulation results, which further verifies the effectiveness and practicability of the algorithm. This novel algorithm with high time delay estimation accuracy is of great significance for SSL in low SNR environment.

Exploiting spatial diversity for increasing the robustness of sound source localization systems against reverberation

Acoustic reverberation is one of the most relevant factors that hampers the localization of a sound source inside a room. To date, several approaches have been proposed to deal with it, but have not always been evaluated under realistic conditions. This paper proposes exploiting spatial diversity as an alternative approach to achieve robustness against reverberation. The theoretical arguments supporting this approach are first presented and later confirmed by means of simulation results and real measurements. Simulations are run for reverberation times up to 2 s, thus providing results with a wider range of validity than in other previous research works. It is concluded that the use of systems consisting of several, sufficiently separated, small arrays leads to the best results in reverberant environments. Some recommendations are given regarding the choice of the array sizes, the separation among them, and the way to combine SRP-PHAT maps obtained from diverse arrays.

Towards a Consensus on an ICF-Based Classification System for Horizontal Sound-Source Localization.

The study aimed to develop a consensus classification system for the reporting of sound localization testing results, especially in the field of cochlear implantation. Against the background of an overview of the wide variations present in localization testing procedures and reporting metrics, a novel classification system was proposed to report localization errors according to the widely accepted International Classification of Functioning, Disability and Health (ICF) framework. The obtained HEARRING_LOC_ICF scale includes the ICF graded scale: 0 (no impairment), 1 (mild impairment), 2 (moderate impairment), 3 (severe impairment), and 4 (complete impairment). Improvement of comparability of localization results across institutes, localization testing setups, and listeners was demonstrated by applying the classification system retrospectively to data obtained from cohorts of normal-hearing and cochlear implant listeners at our institutes. The application of our classification system will help to facilitate multi-center studies, as well as allowing better meta-analyses of data, resulting in improved evidence-based practice in the field.

AI-based sound source localization system with higher accuracy

This paper focuses on the research of sound source detection technology. Combining the artificial intelligence (AI) and 6th generation (6G) communications, it is a hot topic to build a centralized microphone array to locate the sound source in a particular scene. However, degrees of vagueness still exist concerning the correlations and influences between the array establishment, localization implementation and its performance. To this end, we propose a sound source localization system (SSLS) with a high accuracy, ultra reliability and low latency communication. Meanwhile, an AI-based back propagation (BP) neural network enables the SSLS to predict the sound source position (SSP). SSLS is mainly composed of a data acquisition terminal and a data processing terminal. After relying on the Visual Studio and adopting the C/C#, a reliable SSP measurement is provided by the SSLS. When SSLS is running, sound source signals (SSSs) are collected through the array to obtain the time difference of arrival (TDOA) values in real time. Different from existing methods, this SSLS changes the SSS into a pulse signal by detecting the signal with a fixed energy intensity. At the same time, an intensity-based trigger interruption is designed to complete the TDOA acquisition. In this way, besides a reduction in TDOA calculation, the signal recognition is also improved. Further, while using TDOAs to locate the sound source, localization steps with a BP network are presented. Considering extreme sound source angles, a multi-plane based data fusion method is proposed. With the setting of different angles, especially extreme angles, we assess both quantitative and qualitative results of the proposed method, demonstrating that our method can achieve high-accuracy localization results regardless of angle degrees and the number of planes. Finally, comparison experiments confirm that the proposed SSLS significantly outperforms existing methods.

FRL: Fast and Reconfigurable Accelerator for Distributed Sound Source Localization

Sound source localization (SSL) has been widely applied in industrial and civil fields. And with the development of wearable devices and the Internet of Things (IoT), it is attractive to deploy the SSL system onto embedded and portable devices. However, the software-based SSL system causes excessive response delay and is often affected by environmental noise. To overcome this obstacle, we propose the fast and precise localization (FPL) algorithm for distributed SSL systems. It combines the benefits of both time difference of arrival (TDOA) and steered response power (SRP) methods, and thus it is able to localize sound sources fast and precisely. To further improve the localization speed, we propose the fast and reconfigurable localization (FRL) accelerator, which is an algorithm-hardware co-designed SSL accelerator. It adopts multiple distributed localization nodes for higher localization precision and higher robustness to environmental interference, and it can be configured into either the fast or precise mode to adapt to various environments. Experimental evaluations show that our proposed FPL algorithm can achieve high localization speed and precision, and the field-programmable gate array (FPGA)-based FRL accelerator outperforms the software implementation by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$48.6\times $ </tex-math></inline-formula> and outperforms the prior FPGA-based SSL accelerators by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$20\times \sim 838.2\times $ </tex-math></inline-formula> .

Providing a simple and efficient algorithm for passive sound localization

The sound source localization (SSL) system is one of the main approaches for detecting targets. This kind of localization method is carried out in a passive way, without the emission of any signals, and cannot be detected easily. This paper investigates the two-dimensional SSL of stationary targets using only three receivers. Given that, flow-induced noise of the bluff-bodies in viscous fluids is one of the most important noise generation sources, firstly, this noise is simulated numerically. Due to the distance between the receivers in an array, the sound reaches the receiver with time delay. High accuracy SSL needs highly accurate time delays measurement. In the following, time delays are obtained by calculating the cross-correlation in a time domain of sound signals arriving at the receiver array, and next, the bearing angle of the sound source is estimated. Later, using a new, simple (easy installation), fast (real-time), and accurate pure geometric algorithm, the source candidate locations are obtained. Then, using an elimination algorithm, the ambiguity of the noise source location is removed, and source imaginary locations are removed. Finally, this process is validated by a physical algorithm. Numerical results verify the effectiveness of this geometric algorithm.

A Review on Sound Source Localization Systems

A Review on Sound Source Localization Systems

Two-dimensional planar sound source localization based on microphone array

In recent years, people’s awareness of wildlife protection has been increasing, and the demand for wildlife observation and recording has increased. However, the existing sound source localization system needs to continuously collect sound signals and convert the time-domain signals of the sound into frequency-domain signals. These shortcomings will increase the power consumption of the device, adversely affect the battery life, and the sound source localization result is easily interfered by the echo of environmental noise and target sound waves. In this study, the algorithm and hardware are improved on the basis of the existing sound source positioning system, which can improve the positioning accuracy and reduce the power consumption and cost of the whole machine. The adaptive circuit is used to filter and adjust the sound signal in real time to eliminate the influence of the difference between the individual microphones in the microphone array, which makes the sound source localization system more robust to the environmental noise and the echo of the target sound wave, and more suitable for the forest scene of wildlife protection. Through the research of this sound source localization system, we hope to provide some new ideas for producing economical and practical sound source localization equipment and lay a foundation for the intelligent development of wildlife protection in the future.

Evaluating ultrasonic vocalizations as predictors of social behavior during group interaction

During social interaction, animals integrate sensory information—including auditory cues—to influence future behavioral decisions. For example, animals respond to distinct vocalizations with unique actions (Seyfarth et al., 1980). Mice produce ultrasonic vocalizations (USVs) associated with discrete behaviors (Sangiamo et al., 2020); however, it is unclear if USVs inform subsequent actions. To investigate the predictive power of USVs on behavior, while mice freely interacted (n = 11 groups, 2 males and 2 females per group), we continuously recorded video and audio data using a sound source localization system with an 8-channel microphone array to accurately assign vocalizations to individuals. Using machine learning-based approaches to categorize vocalizations based on acoustic properties and extract behaviors in which two mice play unique roles (actor or recipient), we found a significant interaction between particular social behaviors and preceding vocalizations. A permutation test indicated that, depending on the behavioral role of the vocalizer, specific vocalizations preceded dominant (courtship or aggressive) or submissive (avoidant) behaviors at above-chance levels. Additionally, though classifiers trained to distinguish behavioral role based on preceding vocal emission failed for some behaviors, they were successful for particular dominant behaviors. These results suggest a potentially predictive relationship between acoustic communication and subsequent behaviors.

Fundamental study on sound source localization inside a structure using a deep neural network and computer-aided engineering

This study proposes a sound source localization (SSL) method applicable to sources inside structures such as mechanical equipment or buildings. Presently, an SSL system employing a microphone array based on the time difference of arrival estimation can be used to localize a sound source in the same acoustic space as the microphone. However, conventional SSL methods cannot be adopted when the sound source is located inside a structure. Achieving SSL is more difficult in the case of an indirect sound than in that of a direct sound, because the correlation between the observed signals becomes stronger owing to the effect of coupling between the acoustics and structure. To solve this problem, an SSL method employing a deep neural network and computer-aided engineering, which is applicable to the structure’s interiors, is proposed. The proposed method’s effectiveness and feasibility are examined via numerical and actual experimentation. The proposed method can estimate the position of the sound source inside the structure based on the spectrum measured by an accelerometer on the surface of the structure. The results of the numerical experiment indicate a test accuracy of 93.20%, whereas the actual experiment yielded an accuracy of 61.53%. The learning and validation curves show that the accuracy of the actual experiment is lower owing to the occurrence of overlearning, which results from the small amount of data applied. To overcome this issue, data augmentation was used; consequently, the accuracy was improved to 99.82%.

Design and Implementation of the Voice Command Recognition and the Sound Source Localization System for Human–Robot Interaction

SUMMARYHuman–robot interaction (HRI) is becoming more important nowadays. In this paper, a low-cost communication system for HRI is designed and implemented on the Scout robot and a robotic face. A hidden Markov model-based voice command detection system is proposed and a non-native database has been collected by Persian speakers, which contains 10 desired English commands. The experimental results confirm that the proposed system is capable to recognize the voice commands, and properly performs the task or expresses the right answer. Comparing the system with a trained system on the Julius native database shows a better true detection (about 10%).

Single-Channel Multiple-Receiver Sound Source Localization System with Homomorphic Deconvolution and Linear Regression.

The conventional sound source localization systems require the significant complexity because of multiple synchronized analog-to-digital conversion channels as well as the scalable algorithms. This paper proposes a single-channel sound localization system for transport with multiple receivers. The individual receivers are connected by the single analog microphone network which provides the superimposed signal over simple connectivity based on asynchronized analog circuit. The proposed system consists of two computational stages as homomorphic deconvolution and machine learning stage. A previous study has verified the performance of time-of-flight estimation by utilizing the non-parametric and parametric homomorphic deconvolution algorithms. This paper employs the linear regression with supervised learning for angle-of-arrival prediction. Among the circular configurations of receiver positions, the optimal location is selected for three-receiver structure based on the extensive simulations. The non-parametric method presents the consistent performance and Yule–Walker parametric algorithm indicates the least accuracy. The Steiglitz–McBride parametric algorithm delivers the best predictions with reduced model order as well as other parameter values. The experiments in the anechoic chamber demonstrate the accurate predictions in proper ensemble length and model order.