3D Sound Localization Research Articles

A Platform for Evaluation of Synthetic Reflected Sounds on 3D Sound Localization

A Platform for Evaluation of Synthetic Reflected Sounds on 3D Sound Localization

Effect of Perceptual Training with Sound-Guided and Kinesthetic Feedback on Human 3D Sound Localization Capabilities.

In this paper, we experimentally investigate how the 3D sound localization capabilities of the blind can improve through perceptual training. To this end, we develop a novel perceptual training method with sound-guided feedback and kinesthetic assistance to evaluate its effectiveness compared to conventional training methods. In perceptual training, we exclude visual perception by blindfolding the subjects to apply the proposed method to the visually impaired. Subjects used a specially designed pointing stick to generate a sound at the tip, indicating localization error and tip position. The proposed perceptual training aims to evaluate the training effect on 3D sound localization, including variations in azimuth, elevation, and distance. The six days of training based on six subjects resulted in the following outcomes: (1) In general, accuracy in full 3D sound localization can be improved based on training. (2) Training based on relative error feedback is more effective than absolute error feedback. (3) Subjects tend to underestimate distance when the sound source is near, less than 1000 mm, or larger than 15° to the left, and overestimate the elevation when the sound source is near or in the center, and within ±15° in azimuth estimations.

Spontaneous head-movements improve sound localization in aging adults with hearing loss.

Moving the head while a sound is playing improves its localization in human listeners, in children and adults, with or without hearing problems. It remains to be ascertained if this benefit can also extend to aging adults with hearing-loss, a population in which spatial hearing difficulties are often documented and intervention solutions are scant. Here we examined performance of elderly adults (61–82 years old) with symmetrical or asymmetrical age-related hearing-loss, while they localized sounds with their head fixed or free to move. Using motion-tracking in combination with free-field sound delivery in visual virtual reality, we tested participants in two auditory spatial tasks: front-back discrimination and 3D sound localization in front space. Front-back discrimination was easier for participants with symmetrical compared to asymmetrical hearing-loss, yet both groups reduced their front-back errors when head-movements were allowed. In 3D sound localization, free head-movements reduced errors in the horizontal dimension and in a composite measure that computed errors in 3D space. Errors in 3D space improved for participants with asymmetrical hearing-impairment when the head was free to move. These preliminary findings extend to aging adults with hearing-loss the literature on the advantage of head-movements on sound localization, and suggest that the disparity of auditory cues at the two ears can modulate this benefit. These results point to the possibility of taking advantage of self-regulation strategies and active behavior when promoting spatial hearing skills.

Benefits of active listening during 3D sound localization

In everyday life, sound localization entails more than just the extraction and processing of auditory cues. When determining sound position in three dimensions, the brain also considers the available visual information (e.g., visual cues to sound position) and resolves perceptual ambiguities through active listening behavior (e.g., spontaneous head movements while listening). Here, we examined to what extent spontaneous head movements improve sound localization in 3D—azimuth, elevation, and depth—by comparing static vs. active listening postures. To this aim, we developed a novel approach to sound localization based on sounds delivered in the environment, brought into alignment thanks to a VR system. Our system proved effective for the delivery of sounds at predetermined and repeatable positions in 3D space, without imposing a physically constrained posture, and with minimal training. In addition, it allowed measuring participant behavior (hand, head and eye position) in real time. We report that active listening improved 3D sound localization, primarily by ameliorating accuracy and variability of responses in azimuth and elevation. The more participants made spontaneous head movements, the better was their 3D sound localization performance. Thus, we provide proof of concept of a novel approach to the study of spatial hearing, with potentials for clinical and industrial applications.

The Effects of Laboratory Environment Type on Intermittent Sound Localization

Virtual reality (VR) and augmented reality (AR) are gaining commercial popularity. 3D sound guidelines for AR and VR are derived from psychoacoustic experiments performed in contrived, sterile laboratory settings. Often, these settings are expensive, inaccessible, and unattainable for researchers. The feasibility of conducting psychoacoustic experiments outside the laboratory remains unclear. To investigate, we explore 3D sound localization experiments in-lab (IL) and out-of-the lab (OL). The IL study condition was conducted as a traditional psychoacoustic experiment in a soundproof booth. The OL condition occurred in a quiet environment of the participants' choosing, using commercial-grade headphones. Localization performance did not vary significantly for OL participants compared to the IL participants, with larger variation observed in the IL condition. Participants needed significantly more time to complete the experiment IL than OL. The results suggest that conducting headphone-based psychoacoustic experiments outside the laboratory is feasible if completion time is negligible.

Intensive Training of Spatial Hearing Promotes Auditory Abilities of Bilateral Cochlear Implant Adults: A Pilot Study.

The aim of this study was to evaluate the feasibility of a virtual reality-based spatial hearing training protocol in bilateral cochlear implant (CI) users and to provide pilot data on the impact of this training on different qualities of hearing. Twelve bilateral CI adults aged between 19 and 69 followed an intensive 10-week rehabilitation program comprised eight virtual reality training sessions (two per week) interspersed with several evaluation sessions (2 weeks before training started, after four and eight training sessions, and 1 month after the end of training). During each 45-minute training session, participants localized a sound source whose position varied in azimuth and/or in elevation. At the start of each trial, CI users received no information about sound location, but after each response, feedback was given to enable error correction. Participants were divided into two groups: a multisensory feedback group (audiovisual spatial cue) and an unisensory group (visual spatial cue) who only received feedback in a wholly intact sensory modality. Training benefits were measured at each evaluation point using three tests: 3D sound localization in virtual reality, the French Matrix test, and the Speech, Spatial and other Qualities of Hearing questionnaire. The training was well accepted and all participants attended the whole rehabilitation program. Four training sessions spread across 2 weeks were insufficient to induce significant performance changes, whereas performance on all three tests improved after eight training sessions. Front-back confusions decreased from 32% to 14.1% ( p = 0.017); speech recognition threshold score from 1.5 dB to -0.7 dB signal-to-noise ratio ( p = 0.029) and eight CI users successfully achieved a negative signal-to-noise ratio. One month after the end of structured training, these performance improvements were still present, and quality of life was significantly improved for both self-reports of sound localization (from 5.3 to 6.7, p = 0.015) and speech understanding (from 5.2 to 5.9, p = 0.048). This pilot study shows the feasibility and potential clinical relevance of this type of intervention involving a sensorial immersive environment and could pave the way for more systematic rehabilitation programs after cochlear implantation.

An Analytical Method for 3-D Sound Source Localization Based on a Five-Element Microphone Array

An analytical method for 3D sound source localization based on a five-element microphone array is proposed in this work. With four time-difference-of-arrival (TDOA) values, the proposed method realizes the sound source localization by analytical solution. A new five-element microphone array configuration (which is characterized in that four microphones are in a same plane and at four vertices of a concave quadrilateral respectively, while the fifth microphone is out of the plane) is analyzed and studied. It is proved that with the proposed configuration, the five-element microphone array can obtain a unique and analytical solution. To investigate the effectiveness and the localization performance of the proposed method, a practical five-element microphone array is developed and 3D sound source localization experiments are carried out. The TDOA values are obtained by the generalized cross correlation (GCC) method based on the phase transform (PHAT). The experimental results show that the proposed method is an effective near-field localization method. The localization performance of the practical five-element microphone array is satisfactory. Compared with the conventional methods, the proposed method realizes the 3D localization by analytical solution and has the advantages of simple construction and lower computational complexity.

Multiarray Eigenbeam-ESPRIT for 3D Sound Source Localization With Multiple Spherical Microphone Arrays

A 3D sound source localization (SSL) technique for multiple spherical arrays is proposed. Spherical arrays have been popularly used for direction of arrival (DoA) estimation, which serves as important prior knowledge for source separation, dereverberation, and binaural rendering for VR applications. In this work, a parametric 3D SSL technique is proposed that can detect multiple sources using multiple spherical array recordings. From the enriched information by using multiple arrays, we show that the parametric estimation of 3D positions is possible without scanning candidate positions in 3D space. To mitigate the problem of the ambiguous association between DoAs estimated from different arrays, a total covariance matrix including both auto- and cross-covariance matrices between arrays is utilized such that DoAs from arrays are automatically paired to uniquely localize 3D source positions. The joint parameter estimation is based on the generalized joint Schur decomposition of multiple matrices combined with a geometric projection robustly guiding the convergence of the proposed iteration algorithm. Simulations conducted in anechoic and reverberant room conditions reveal that the proposed technique can accurately determine multiple source positions in 3D space without ambiguities.

Novel Position Estimation using Differential Timing Information for Asynchronous LTE/NR Networks

Positioning techniques have been a common objective since the early development of wireless networks. However, current positioning methods in cellular networks, for instance, are still primarily focused on the use of the Global Navigation Satellite System (GNSS), which has several limitations, like high power drainage and failure in indoor scenarios. This study introduces a novel approach employing standard LTE signaling in order to provide high accuracy positioning estimation. The proposed technique is designed in analogy to the human sound localization system, eliminating the need of having information from three spatially diverse Base Stations (BSs). This is inspired by the perfect human 3D sound localization with two ears. A field study is carried out in a dense urban city to verify the accuracy of the proposed technique, with more than 20 thousand measurement samples collected. The achieved positioning accuracy is meeting the latest Federal Communications Commission (FCC) requirements in the planner dimension.

3D Multiple Sound Source Localization by Proposed Cuboids Nested Microphone Array in Combination with Adaptive Wavelet-Based Subband GEVD

Sound source localization is one of the applicable areas in speech signal processing. The main challenge appears when the aim is a simultaneous multiple sound source localization from overlapped speech signals with an unknown number of speakers. Therefore, a method able to estimate the number of speakers, along with the speaker’s location, and with high accuracy is required in real-time conditions. The spatial aliasing is an undesirable effect of the use of microphone arrays, which decreases the accuracy of localization algorithms in noisy and reverberant conditions. In this article, a cuboids nested microphone array (CuNMA) is first proposed for eliminating the spatial aliasing. The CuNMA is designed to receive the speech signal of all speakers in different directions. In addition, the inter-microphone distance is adjusted for considering enough microphone pairs for each subarray, which prepares appropriate information for 3D sound source localization. Subsequently, a speech spectral estimation method is considered for evaluating the speech spectrum components. The suitable spectrum components are selected and the undesirable components are denied in the localization process. The speech information is different in frequency bands. Therefore, the adaptive wavelet transform is used for subband processing in the proposed algorithm. The generalized eigenvalue decomposition (GEVD) method is implemented in sub-bands on all nested microphone pairs, and the probability density function (PDF) is calculated for estimating the direction of arrival (DOA) in different sub-bands and continuing frames. The proper PDFs are selected by thresholding on the standard deviation (SD) of the estimated DOAs and the rest are eliminated. This process is repeated on time frames to extract the best DOAs. Finally, K-means clustering and silhouette criteria are considered for DOAs classification in order to estimate the number of clusters (speakers) and the related DOAs. All DOAs in each cluster are intersected for estimating the position of the 3D speakers. The closest point to all DOA planes is selected as a speaker position. The proposed method is compared with a hierarchical grid (HiGRID), perpendicular cross-spectra fusion (PCSF), time-frequency wise spatial spectrum clustering (TF-wise SSC), and spectral source model-deep neural network (SSM-DNN) algorithms based on the accuracy and computational complexity of real and simulated data in noisy and reverberant conditions. The results show the superiority of the proposed method in comparison with other previous works.

3D sound source localization with fiber optic sensor array based on genetic algorithm

Abstract Sound source localization (SLL), which is an area of multi-point acoustic detection, has always been an important research topic since it has been employed in many applications. It has been notable to use fiber optic sensors in the SSL due to the advantages compared to traditional sensors. In this paper, a fiber optic sensor array is obtained with sensor tips formed with polymer-based diaphragm material, which is low cost and easy to manufacture. The SSL process has been experimentally verified in the measurement medium with a volume of 1 m3. Both constant frequency sound signals and speech signals were used in experimental measurements. The signals received from the sensors were processed by using the genetic algorithm, and the average distance to the source was calculated in 3D with an error of 0.0248 m. In the measurements, results were obtained at very close distances to the sound source, such as 0.0032 m, 0.0046 m, 0.0071 m.

3D sound source localization based on coherence-adjusted monopole dictionary and modified convex clustering

In this paper, a sound source localization method for simultaneously estimating both direction-of-arrival (DOA) and distance from the microphone array is proposed. For estimating distance, the off-grid problem must be overcome because the range of distance to be considered is quite broad and even not bounded. The proposed method estimates the positions based on a modified version of the convex clustering method combined with the sparse coefficients estimation. A method for constructing a suitable monopole dictionary based on the coherence is also proposed so that the convex clustering based method can appropriately estimate distance of the sound sources. Numerical and measurement experiments were performed to investigate the performance of the proposed method.

3D Moving Sound Source Localization via Conventional Microphones

In this paper, we present an approach to sound localization for moving sound sources. We use four conventional microphones and a multi-channel sound recording device for capturing sound signals. A database with different types of moving sound source scenarios is created and one of the scenarios is analysed in this paper. This scenario involves walking of a person and creating sounds simultaneously at certain positions in a room, and our aim is to determine the location of the person as three dimensions. We use time difference of arrival for 3D sound source localization. The time delays between sounds channels are obtained by excitation source information based time-delay estimation algorithm. In addition, signal processing methods are used for increasing the success of moving sound source localization. Savitzky-Golay and threshold filters are utilized to pre-filter the recorded sounds and removing the background noise signals, respectively. The non-linear equation of the sound localization is solved by using the modified Levenberg–Marquardt Algorithm (LMA) on the Time Difference of Arrival (TDOA) process. The results for the 3D localization of the moving sound source are obtained by measuring the differences between the real and estimated positions of the person.DOI: http://dx.doi.org/10.5755/j01.eie.23.4.18724

Sound source localization based on sparse estimation and convex clustering

Sound source localization techniques in array signal processing have been interested for many years. A lot of direction-of-arrival (DOA) estimation methods have been proposed. Most of them use plane waves as model. On the other hand, 3D sound source localization techniques are important for many applications. They are required to estimate the DOA and the distance of sound sources. In the case of the techniques targeted to estimate the distances of sound sources, monopoles must be used for the model. The positions of monopoles are candidates for the sound source positions. However, it is difficult to handle a large number of monopoles placed at various distances. Moreover, it is difficult to decide the number of sound sources to be estimated from the candidates. The proposed method uses sparse estimation with a monopole dictionary in frequency domain, as is well known that the effectiveness of sparse representation for DOA estimation has been in late years. After the sparse estimation, the proposed method ...

Comparison of Different Egocentric Pointing Methods for 3D Sound Localization Experiments

This study evaluates several methods for reporting the perceived location of real sound sources. It is well known that the method used for collecting judgments in auditory-localization experiments has a strong influence on the accuracy of a subject’s response. Previous works on auditory-localization tasks revealed that egocentric pointing methods (which are based on a body-centered coordinate system) allow for more accurate judgments than verbal reporting or exocentric pointing techniques (which are based on a 2D or 3D reporting device). Three different egocentric methods are compared: the most commonly applied “manual pointing” and “head pointing” methods, and the “proximal pointing” method, which forces the participants to indicate the apparent direction by pointing in the proximal region of the head with a marker held at the fingertips. The two first methods involve a rotation of the body of the participant, whereas the third method only involves movements of the arm(s) and hand(s) with a fixed head. Sound stimuli were presented randomly over 24 loudspeakers that were uniformly distributed on the upper hemisphere around the subject. The merits of the different methods are compared and discussed with regard to localization errors and to practical considerations. Although they show similar trends, each of the different methods affects the pointing accuracy in a specific way. The proximal pointing method, for example, is more accurate for sources located at high elevation angles. However, at rear locations close to the median plane an increased bias appears due to difficulties in performing the motor task to reach these positions. The proximal pointing method shows faster response times, which may be advantageous when planning 3D sound localization experiments.

Real-time Sound Source Localization on an Embedded GPU Using a Spherical Microphone Array

Abstract Spherical microphone arrays are becoming increasingly important in acoustic signal processing systems for their applications in sound field analysis, beamforming, spatial audio, etc. The positioning of target and interfering sound sources is a crucial step in many of the above applications. Therefore, 3D sound source localization is a highly relevant topic in the acoustic signal processing field. However, spherical microphone arrays are usually composed of many microphones and running signal processing localization methods in real time is an important issue. Some works have already shown the potential of Graphic Processing Units (GPUs) for developing high-end real-time signal processing systems. New embedded systems with integrated GPU accelerators providing low power consumption are becoming increasingly relevant. These novel systems play a very important role in the new era of smartphones and tablets, opening further possibilities to the design of high-performance compact processing systems. This paper presents a 3D source localization system using a spherical microphone array fully implemented on an embedded GPU. The real-time capabilities of these platforms are analyzed, providing also a performance analysis of the localization system under different acoustic conditions.

An investigation of matching symmetry in the human pinnae with possible implications for 3D ear recognition and sound localization.

The human external ears, or pinnae, have an intriguing shape and, like most parts of the human external body, bilateral symmetry is observed between left and right. It is a well-known part of our auditory sensory system and mediates the spatial localization of incoming sounds in 3D from monaural cues due to its shape-specific filtering as well as binaural cues due to the paired bilateral locations of the left and right ears. Another less broadly appreciated aspect of the human pinna shape is its uniqueness from one individual to another, which is on the level of what is seen in fingerprints and facial features. This makes pinnae very useful in human identification, which is of great interest in biometrics and forensics. Anatomically, the type of symmetry observed is known as matching symmetry, with structures present as separate mirror copies on both sides of the body, and in this work we report the first such investigation of the human pinna in 3D. Within the framework of geometric morphometrics, we started by partitioning ear shape, represented in a spatially dense way, into patterns of symmetry and asymmetry, following a two-factor anova design. Matching symmetry was measured in all substructures of the pinna anatomy. However, substructures that 'stick out' such as the helix, tragus, and lobule also contained a fair degree of asymmetry. In contrast, substructures such as the conchae, antitragus, and antihelix expressed relatively stronger degrees of symmetric variation in relation to their levels of asymmetry. Insights gained from this study were injected into an accompanying identification setup exploiting matching symmetry where improved performance is demonstrated. Finally, possible implications of the results in the context of ear recognition as well as sound localization are discussed.

Embedded System Implementation of Sound Localization in Proximal Region

A sound localization method in the proximal region is proposed, which is based on a low-cost 3D sound localization algorithm with the use of head-related transfer functions (HRTFs). The auditory parallax model is applied to the current algorithm so that more accurate HRTFs can be used for sound localization in the proximal region. In addition, head-shadowing effects based on rigid-sphere model are reproduced in the proximal region by means of a second-order IIR filter. A subjective listening test demonstrates the effectiveness of the proposed method. Embedded system implementation of the proposed method is also described claiming that the proposed method improves sound effects in the proximal region only with 5.1% increase of memory capacity and 8.3% of computational costs.

ROBUST SOUND LOCALIZATION USING SMALL NUMBER OF MICROPHONES

In this paper, robust sound localization is proposed for systems with spatially distributed microphones fewer than 4, the minimum required criteria for 3D sound localization in the literature, in an effort to address the robustness issues of microphone failures. It is shown that, for three- and two-microphone systems, two and four samples of the Interaural Time Difference (ITD) measurements are required respectively. In a one-microphone system, five samples of the Interaural Intensity Difference (IID) measurement should be used instead. These investigations are important because if one microphone fails, the remaining system can still function to locate the sound source without significant impact on the localization system. In addition, our investigation also shows that a 3-microphone system can estimate the azimuth and elevation simultaneously, which usually requires 4 microphones in the literature. Simulation and experimental results are presented to illustrate the performance of a three-microphone system. Results show that the system can locate the sound source with satisfactory accuracy.

Embedded 3D sound movement system based on feature extraction of head-related transfer function

VLSI implementation of a sound movement system is described, which is dedicated to mobile applications. This system is based on low cost 3D sound localization method, where feature extraction of head-related transfer function is attempted in order to realize the sound movement at all positions in 3D space by means of efficient coefficient storing scheme. The proposed system can be implemented by only 30,000 gates, which is small enough to mobile applications. Moreover, a prototype system is constructed by using an FPGA board and subjective tests are performed to evaluate the practicability of the proposed 3D sound movement system.