Sound Localization Experiments Research Articles

Mechanical Fault Sound Source Localization Estimation in a Multisource Strong Reverberation Environment

Aiming at the sound source localization of mechanical faults in a strong reverberation scenario with multiple sound sources, this paper investigates a mechanical fault source localization method using the U-net deep convolutional neural network. The method utilizes the SRP-PHAT algorithm to calculate the response power spectra of the collected multichannel fault signals. Through the utilization of the U-net neural network, the response power spectra containing spurious peaks are transformed into “clean” estimated source distribution maps. By employing interpolation search, the estimated source distribution maps are processed to obtain location estimations for multiple fault sources. To validate the effectiveness of the proposed method, this paper constructs an experimental dataset using mechanical fault data from electromechanical equipment relays and conducts sound source localization experiments. The experimental results show that the U-net network under 0.2 s/0.5 s/0.7 s reverberation time can effectively eliminate spurious peak interference in the response power spectrum. As the signal-to-noise ratio decreases, it can still distinguish the sound sources with a distance of 0.2 m. In the context of multifault source localization, the method is capable of simultaneously locating the positions of four fault sources, with an average localization error of less than 0.02 m. The method in this paper effectively eliminates spurious peaks in the response power spectra under conditions of multisource strong reverberation. It accurately locates multiple mechanical fault sources, thereby significantly enhancing the efficiency of mechanical fault detection.

Dynamic spectral cues do not affect human sound localization during small head movements.

Natural listening involves a constant deployment of small head movement. Spatial listening is facilitated by head movements, especially when resolving front-back confusions, an otherwise common issue during sound localization under head-still conditions. The present study investigated which acoustic cues are utilized by human listeners to localize sounds using small head movements (below ±10° around the center). Seven normal-hearing subjects participated in a sound localization experiment in a virtual reality environment. Four acoustic cue stimulus conditions were presented (full spectrum, flattened spectrum, frozen spectrum, free-field) under three movement conditions (no movement, head rotations over the yaw axis and over the pitch axis). Localization performance was assessed using three metrics: lateral and polar precision error and front-back confusion rate. Analysis through mixed-effects models showed that even small yaw rotations provide a remarkable decrease in front-back confusion rate, whereas pitch rotations did not show much of an effect. Furthermore, MSS cues improved localization performance even in the presence of dITD cues. However, performance was similar between stimuli with and without dMSS cues. This indicates that human listeners utilize the MSS cues before the head moves, but do not rely on dMSS cues to localize sounds when utilizing small head movements.

The Effects of Laboratory Environment Type on Intermittent Sound Localization

Virtual reality (VR) and augmented reality (AR) are gaining commercial popularity. 3D sound guidelines for AR and VR are derived from psychoacoustic experiments performed in contrived, sterile laboratory settings. Often, these settings are expensive, inaccessible, and unattainable for researchers. The feasibility of conducting psychoacoustic experiments outside the laboratory remains unclear. To investigate, we explore 3D sound localization experiments in-lab (IL) and out-of-the lab (OL). The IL study condition was conducted as a traditional psychoacoustic experiment in a soundproof booth. The OL condition occurred in a quiet environment of the participants' choosing, using commercial-grade headphones. Localization performance did not vary significantly for OL participants compared to the IL participants, with larger variation observed in the IL condition. Participants needed significantly more time to complete the experiment IL than OL. The results suggest that conducting headphone-based psychoacoustic experiments outside the laboratory is feasible if completion time is negligible.

Sound localization in web-based 3D environments

Sound delivery is a key aspect of immersivity in virtual and augmented reality (VR/AR), with studies hinting at a correlation between users’ ability to locate sounds around them and the ‘feeling of being there’. This is particularly true for WebVR, a method of delivering immersive experiences through a local web browser that has recently captured attention in multiple industries. In WebVR, audio is the main spatial cue. Designers need to select the correct number of sound sources so that users perceive the location of incoming sound correctly. Information on how users localize sound is essential. Sound localization experiments, so far, have been run only in empty spaces or closed rooms, without clear indications for designers in WebVR. Thus, in this study, we investigate sound localization directly through WebVR. To do so, we designed a traditional empty room for training and a city-like virtual environment for testing purposes. In our paper, we also discuss key design parameters, differences in perception for vertical and horizontal directions, the impact of training, and the role of changing virtual environments. In addition, we introduce and test a new sound cue along with the traditional pink noise sound to measure and explore the impact of different sound cues in different environments. The results demonstrate the potential of exploring sound localization using WebVR, and our study will support the development of virtual experiences in human-computer interaction that may be able to reach a large number of participants using a local web browser.

The Ventriloquist Effect is not Consistently Affected by Stimulus Realism<xref ref-type="fn" rid="jpi0144fn1"><sup>†</sup></xref>

Despite more than 60 years of research, it has remained uncertain if and how realism affects the ventriloquist effect. Here, a sound localization experiment was run using spatially disparate audio-visual stimuli. The visual stimuli were presented using virtual reality, allowing for easy manipulation of the degree of realism of the stimuli. Starting from stimuli commonly used in ventriloquist experiments, i.e., a light flash and noise burst, a new factor was added or changed in each condition to investigate the effect of movement and realism without confounding the effects of an increased temporal correlation of the audio-visual stimuli. First, a distractor task was introduced to ensure that participants fixated their eye gaze during the experiment. Next, movement was added to the visual stimuli while maintaining a similar temporal correlation between the stimuli. Finally, by changing the stimuli from the flash and noise stimuli to the visuals of a bouncing ball that made a matching impact sound, the effect of realism was assessed. No evidence for an effect of realism and movement of the stimuli was found, suggesting that, in simple scenarios, the ventriloquist effect might not be affected by stimulus realism.

An Analytical Method for 3-D Sound Source Localization Based on a Five-Element Microphone Array

An analytical method for 3D sound source localization based on a five-element microphone array is proposed in this work. With four time-difference-of-arrival (TDOA) values, the proposed method realizes the sound source localization by analytical solution. A new five-element microphone array configuration (which is characterized in that four microphones are in a same plane and at four vertices of a concave quadrilateral respectively, while the fifth microphone is out of the plane) is analyzed and studied. It is proved that with the proposed configuration, the five-element microphone array can obtain a unique and analytical solution. To investigate the effectiveness and the localization performance of the proposed method, a practical five-element microphone array is developed and 3D sound source localization experiments are carried out. The TDOA values are obtained by the generalized cross correlation (GCC) method based on the phase transform (PHAT). The experimental results show that the proposed method is an effective near-field localization method. The localization performance of the practical five-element microphone array is satisfactory. Compared with the conventional methods, the proposed method realizes the 3D localization by analytical solution and has the advantages of simple construction and lower computational complexity.

The Ventriloquist Effect is not Consistently Affected by Stimulus Realism †

Despite more than 60 years of research, it has remained uncertain if and how realism affects the ventriloquist effect. Here, a sound localization experiment was run using spatially disparate audio-visual stimuli. The visual stimuli were presented using virtual reality, allowing for easy manipulation of the degree of realism of the stimuli. Starting from stimuli commonly used in ventriloquist experiments, i.e., a light flash and noise burst, a new factor was added or changed in each condition to investigate the effect of movement and realism without confounding the effects of an increased temporal correlation of the audio-visual stimuli. First, a distractor task was introduced to ensure that participants fixated their eye gaze during the experiment. Next, movement was added to the visual stimuli while maintaining a similar temporal correlation between the stimuli. Finally, by changing the stimuli from the flash and noise stimuli to the visuals of a bouncing ball that made a matching impact sound, the effect of realism was assessed. No evidence for an effect of realism and movement of the stimuli was found, suggesting that, in simple scenarios, the ventriloquist effect might not be affected by stimulus realism.

The Ventriloquist Effect is not Consistently Affected by Stimulus Realism

Despite more than 60 years of research, it has remained uncertain if and how realism affects the ventriloquist effect. Here, a sound localization experiment was run using spatially disparate audio-visual stimuli. The visual stimuli were presented using virtual reality, allowing for easy manipulation of the degree of realism of the stimuli. Starting from stimuli commonly used in ventriloquist experiments, i.e., a light flash and noise burst, a new factor was added or changed in each condition to investigate the effect of movement and realism without confounding the effects of an increased temporal correlation of the audio-visual stimuli. First, a distractor task was introduced to ensure that participants fixated their eye gaze during the experiment. Next, movement was added to the visual stimuli while maintaining a similar temporal correlation between the stimuli. Finally, by changing the stimuli from the flash and noise stimuli to the visuals of a bouncing ball that made a matching impact sound, the effect of realism was assessed. No evidence for an effect of realism and movement of the stimuli was found, suggesting that, in simple scenarios, the ventriloquist effect might not be affected by stimulus realism.

Spatial Hearing by Bilateral Cochlear Implant Users With Temporal Fine-Structure Processing.

Several studies have demonstrated the advantages of the bilateral vs. unilateral cochlear implantation in listeners with bilateral severe to profound hearing loss. However, it remains unclear to what extent bilaterally implanted listeners have access to binaural cues, e.g., accurate processing of interaural timing differences (ITDs) for low-frequency sounds (<1.5 kHz) and interaural level differences (ILDs) for high frequencies (>3 kHz). We tested 25 adult listeners, bilaterally implanted with MED-EL cochlear implant (CI) devices, with and without fine-structure (FS) temporal processing as encoding strategy in the low-frequency channels. In order to assess whether the ability to process binaural cues was affected by fine-structure processing, we performed psychophysical ILD and ITD sensitivity measurements and free-field sound localization experiments. We compared the results of the bilaterally implanted listeners with different numbers of FS channels. All CI listeners demonstrated good sensitivity to ILDs, but relatively poor to ITD cues. Although there was a large variability in performance, some bilateral CI users showed remarkably good localization skills. The FS coding strategy for bilateral CI hearing did not improve fine-structure ITD processing for spatial hearing on a group level. However, some CI listeners were able to exploit weakly informative temporal cues to improve their low-frequency spatial perception.

Chance-level hit rates in closed-set, forced-choice audiometry and a novel utility for the significance test-based detection of malingering.

The primary aim of this study was to extend existing theory on the relationship between chance-level performance and the number of alternatives and trials in closed-set, forced-choice speech audiometry and sound localization methods. When calculating chance performance for closed-set, forced-choice experiments with multiple trials, the binomial distribution should be preferred over the simple 1/a probability, as the latter is appropriate only for single trial experiments. The historical use of constant hit rates for determining chance performance has been based upon the assumption that random hits are distributed evenly across multiple trials. For any closed-set, forced-choice task with 2 to 10 alternatives and 2 to 100 trials, we calculated the probability of obtaining any given hit rate due to random guessing alone according to the binomial distribution. Hit rates with probabilities p > 0.05 were interpreted as being likely to occur due to random chance alone, whereas hit rates with probabilities of p ≤ 0.05 were interpreted as being unlikely to occur due to chance alone. For sound localization experiments with speakers at fixed positions, the expected probability of a random hit was also calculated using the binomial distribution. The expected angular root mean square (rms) error in sound localization resulting from the random selection of sound sources was investigated using Monte Carlo simulations. A new aspect in the interpretation of test results was identified for situations in which the observed number of hits is much lower than would be expected due to chance alone. For test methods incorporating a relatively low number of alternatives and a sufficiently high, yet clinically feasible, number of trials, both upper and lower thresholds for chance-level performance could be identified. This lower threshold represents the lowest hit rate which can be expected through random chance alone. Extending interpretation of results to include this lower threshold affords the ability to not only identify performance significantly superior to that of chance, but also that significantly poorer than chance and thereby represents a simple method for the objective detection of malingering.

Multiplexing fiber-optic Fabry-Perot acoustic sensors using self-calibrating wavelength shifting interferometry.

Flexible and stable demodulation techniques of large-scale fiber-optic Fabry-Perot (FP) acoustic sensors are highly desirable for accelerating their industrial applications. In this paper, we report a novel self-calibrating wavelength shifting interferometry (WSI) technique that enables simultaneous multi-point acoustic detection using diaphragm based fiber-optic FP acoustic sensors. A widely tunable modulated grating Y-branch (MG-Y) laser (1527∼1567 nm) performs high-speed wavelength switching, introducing phase-shifts in the wavelength domain for real-time phase retrieval. The proposed self-calibrating WSI is easily extended for multiplexing FP acoustic sensors by calibrating the corresponding phase-shift step of each sensor probe. Based on a modified Hariharan 5-step phase shifting algorithm, the phase-shift step for each channel can be calibrated in real-time, making the system robust in applications involving large environmental perturbations. An all-optical multi-point acoustic detection system based on WSI is proposed and experimentally demonstrated for the first time. Sound source localization experiments show that the multi-point acoustic detection system works stably and the positioning accuracy is about 2.42 cm.

Sound source localization with varying amount of visual information in virtual reality.

To achieve accurate spatial auditory perception, subjects typically require personal head-related transfer functions (HRTFs) and the freedom for head movements. Loudspeaker-based virtual sound environments allow for realism without individualized measurements. To study audio-visual perception in realistic environments, the combination of spatially tracked head mounted displays (HMDs), also known as virtual reality glasses, and virtual sound environments may be valuable. However, HMDs were recently shown to affect the subjects’ HRTFs and thus might influence sound localization performance. Furthermore, due to limitations of the reproduction of visual information on the HMD, audio-visual perception might be influenced. Here, a sound localization experiment was conducted both with and without an HMD and with a varying amount of visual information provided to the subjects. Furthermore, interaural time and level difference errors (ITDs and ILDs) as well as spectral perturbations induced by the HMD were analyzed and compared to the perceptual localization data. The results showed a reduction of the localization accuracy when the subjects were wearing an HMD and when they were blindfolded. The HMD-induced error in azimuth localization was found to be larger in the left than in the right hemisphere. When visual information of the limited set of source locations was provided, the localization error induced by the HMD was found to be negligible. Presenting visual information of hand-location and room dimensions showed better sound localization performance compared to the condition with no visual information. The addition of possible source locations further improved the localization accuracy. Also adding pointing feedback in form of a virtual laser pointer improved the accuracy of elevation perception but not of azimuth perception.

Perceived azimuth-based creditability and self-reported confidence for sound localization experiments using crowdsourcing

Perceived azimuth-based creditability and self-reported confidence for sound localization experiments using crowdsourcing

Directional hearing and sound source localization by fishes.

Directional hearing may enable fishes to seek out prey, avoid predators, find mates, and detect important spatial cues. Early sound localization experiments gave negative results, and it was thought unlikely that fishes utilized the same direction-finding mechanisms as terrestrial vertebrates. However, fishes swim towards underwater sound sources, and some can discriminate between sounds from different directions and distances. The otolith organs of the inner ear detect the particle motion components of sound, acting as vector detectors through the presence of sensory hair cells with differing orientation. However, many questions remain on inner ear functioning. There are problems in understanding the actual mechanisms involved in determining sound direction and distance. Moreover, very little is still known about the ability of fishes to locate sound sources in three-dimensional space. Do fishes swim directly towards a source, or instead "sample" sound levels while moving towards the source? To what extent do fishes utilize other senses and especially vision in locating the source? Further behavioral studies of free-swimming fishes are required to provide better understanding of how fishes might actually locate sound sources. In addition, more experiments are required on the auditory mechanism that fishes may utilize.

Introduction to sound localization experiment

Introduction to sound localization experiment

Localization Experiments with Reporting by Head Orientation: Statistical Framework and Case Study

This research focuses on sound localization experiments in which subjects report the position of an active sound source by turning toward it. A statistical framework for the analysis of the data is presented together with a case study from a large-scale listening experiment. The statistical framework is based on a model that is robust to the presence of front/back confusions and random errors. Closed-form natural estimators are derived, and one-sample and two-sample statistical tests are described. The framework is used to analyze the data of an auralized experiment undertaken by nearly nine hundred subjects. The objective was to explore localization performance in the horizontal plane in an informal setting and with little training, which are conditions that are similar to those typically encountered in consumer applications of binaural audio. Results show that responses had a rightward bias and that speech was harder to localize than percussion sounds, which are results consistent with the literature. Results also show that it was harder to localize sound in a simulated room with a high ceiling despite having a higher direct-to-reverberant ratio than other simulated rooms.

Improving the Audio Game–Playing Performances of People with Visual Impairments through Multimodal Training

Introduction As the number of people with visual impairments (that is, those who are blind or have low vision) is continuously increasing, rehabilitation and engineering researchers have identified the need to design sensory-substitution devices that would offer assistance and guidance to these people for performing navigational tasks. Auditory and haptic cues have been shown to be an effective approach towards creating a rich spatial representation of the environment, so they are considered for inclusion in the development of assistive tools that would enable people with visual impairments to acquire knowledge of the surrounding space in a way close to the visually based perception of sighted individuals. However, achieving efficiency through a sensory substitution device requires extensive training for visually impaired users to learn how to process the artificial auditory cues and convert them into spatial information. Methods Considering all the potential advantages game-based learning can provide, we propose a new method for training sound localization and virtual navigational skills of visually impaired people in a 3D audio game with hierarchical levels of difficulty. The training procedure is focused on a multimodal (auditory and haptic) learning approach in which the subjects have been asked to listen to 3D sounds while simultaneously perceiving a series of vibrations on a haptic headband that corresponds to the direction of the sound source in space. Results The results we obtained in a sound-localization experiment with 10 visually impaired people showed that the proposed training strategy resulted in significant improvements in auditory performance and navigation skills of the subjects, thus ensuring behavioral gains in the spatial perception of the environment.

複数フレームのCSP係数を用いたTDOA推定による複数人物定位·追跡

In this paper, multi-speaker localization methods using voice signal are proposed. Sound source localization systems consisting of many microphones have been widely researched. In our laboratory, we have researched sound source localization of persons based on TDOA (Time Difference of Arrival) between distributed microphones for the purpose of efficient use of air conditioner and lighting in the office . Localization systems based on sound such as footsteps or voice are superior to those based on image data in terms of privacy-consciousness and utility in the dark. As sound source localization methods, beamforming, subspace methods and TDOA-based methods are widely used. CSP (Cross-power Spectrum Phase) analysis is one of the most famous method to estimate TDOA. We have researched sound source localization methods using CSP analysis because it spends less calculation cost than beamforming or subspace methods. However, CSP analysis doesn't work well by itself in the real office environment because of sound lasting for a long time, reverberations, low SNR, and so on. The main cause of difficulty in sound source localization in such a case is the “weak correlation” between different sound signals. In this paper, it is shown that the bad effect of the “weak correlation” can be improved by our proposed methods, Wiener-filter-based CSP analysis and H-CSP (Histogram-CSP) analysis. H-CSP analysis is a statistics data analysis method for CSP coefficients which uses the histogram of CSP coefficients. In this research, the effectiveness of proposed methods are shown by sound source localization experiments in the case sound sources are stable and the case sound sources are moving.

Sound source localization method in an environment with flow based on Amiet–IMACS

A sound source localization method is proposed to localize and analyze the sound source in an environment with airflow. It combines the improved mapping of acoustic correlated sources (IMACS) method and Amiet's method, and is called Amiet–IMACS. It can localize uncorrelated and correlated sound sources with airflow. To implement this approach, Amiet's method is used to correct the sound propagation path in 3D, which improves the accuracy of the array manifold matrix and decreases the position error of the localized source. Then, the mapping of acoustic correlated sources (MACS) method, which is as a high-resolution sound source localization algorithm, is improved by self-adjusting the constraint parameter at each irritation process to increase convergence speed. A sound source localization experiment using a pair of loud speakers in an anechoic wind tunnel under different flow speeds is conducted. The experiment exhibits the advantage of Amiet–IMACS in localizing a more accurate sound source position compared with implementing IMACS alone in an environment with flow. Moreover, the aerodynamic noise produced by a NASA EPPLER 862 STRUT airfoil model in airflow with a velocity of 80m/s is localized using the proposed method, which further proves its effectiveness in a flow environment. Finally, the relationship between the source position of this airfoil model and its frequency, along with its generation mechanism, is determined and interpreted.

A Novel Dual-Channel Matching Method Based on Time Reversal and its Performance for Sound Source Localization in Enclosed Space

For sound source localization in enclosed space, the background noise and reverberation have significant influences on traditional localization algorithms and the array, and these algorithms are usually complex. A sound source localization method called dual-channel matching is proposed in this paper. The method is based on the theory of time reversal. It has strong anti-reverberation capability and the ability to overcome multipath effects. A series of sound source localization experiments were carried out in an underground parking lot, where the real-time performances of the method under different dimensions of sound source devices, SNR, and reverberation were evaluated. The parameters regarding the microphone, obstacle’s influence, different heights between sampling and localization processes, and localization of slow-moving sound sources were also evaluated. The results and their analysis show that (1) high spatial resolution can be obtained while the sound devices satisfy the requirement of the radiated acoustic energy, and that the smaller the size, the smaller is the error range which corresponds to higher spatial resolution; (2) when the SNR is greater than 2 dB, spatial resolution in common room is 10 cm; (3) spatial reverberation has little effects on localization resolution; (4) the method has strong tolerance for environment; and (5) when the sound source moves slowly, this method also has a good localization effect.