Front-back Confusion Research Articles

Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localization of Multiple Sources in Reverberant Environments

This paper presents a novel machine-hearing system that exploits deep neural networks (DNNs) and head movements for robust binaural localisation of multiple sources in reverberant environments. DNNs are used to learn the relationship between the source azimuth and binaural cues, consisting of the complete cross-correlation function (CCF) and interaural level differences (ILDs). In contrast to many previous binaural hearing systems, the proposed approach is not restricted to localisation of sound sources in the frontal hemifield. Due to the similarity of binaural cues in the frontal and rear hemifields, front-back confusions often occur. To address this, a head movement strategy is incorporated in the localisation model to help reduce the front-back errors. The proposed DNN system is compared to a Gaussian mixture model (GMM) based system that employs interaural time differences (ITDs) and ILDs as localisation features. Our experiments show that the DNN is able to exploit information in the CCF that is not available in the ITD cue, which together with head movements substantially improves localisation accuracies under challenging acoustic scenarios in which multiple talkers and room reverberation are present.

Impact of microphone position on sound localization in cochlear implant users

The delivery of directional cues of a hearing device microphone are highly dependent on the position of the microphones. The aim of this work was the evaluation of different microphone positions with regard to the transmission of interaural time and level differences as well as the spectral characteristics and its impact on the localization abilities. Head-related transfer functions of 30 subjects were measured with three different omnidirectional microphones at different positions: in the pinna (ITP), behind the ear (BTE), at the entrance of the ear canal (EEC). Sound localization abilities of 12 bilateral CI users was assessed for the microphone positions ITP and BTE. Only the microphone positions in the ear (ITP, EEC) could sample the spectral cues of the pinna. However, the positioning of the microphone inside of the pinna did not significantly improve sound localization abilities compared to BTE microphones. For sound incidence from rear significantly less front-back confusions were achieved with the microphone inside of the pinna. The microphone position in the pinna showed only a slight improvement in sound localization compared with BTE microphones in CI users. A precondition for better sound localization abilities is the improvement of the delivery of temporal and spectral fine structure cues in CI systems.

Congruent auditory display and confusion in sound localization: Case of elderly drivers

Elderly people may suffer from age-related hearing loss. They often report difficulties to perceive and localize sound sources in noisy environment. How this can be a driving safety issue? This study investigates the effect of hearing impairment, on the driver behavior to localize external sound sources such as emergency vehicles sirens. Subjective tests show that localization confusion appears to be a common problem. Thus, we focus on how to assist the driver, taking into consideration the age-related hearing impairment, to better localize emergency vehicle siren. The hypothesis was based on the stimulus-response compatibility using an effective congruent auditory display, for attentional guidance toward the direction of arrival (DOA) of the external alarm. The proposed approach aims to reduce front-back confusion and enhance the sound localization accuracy of the driver, which is very important for elderly driver, subject to older age cognitive decline. A localization test was performed in lab and in vivo on a test track, where a group of drivers was asked to identify the DOA of an emergency vehicle siren, with and without a dedicated embedded system set up to assist them on sound localization task.

Vertical sound source localization when listeners and sounds rotate: The Wallach vertical illusion

In addition to testing his prediction that listeners would use changes in binaural cues relative to listeners’ self-induced head movements to disambiguate front-back confusions, Wallach (1939) also tested his calculations that the relative rate at which these binaural cues change could be used by listeners to determine the elevation of the sound source. Wallach was able to induce the illusion that a sound source rotating along the azimuth plane was perceived as though it were above the listener. We sought to replicate and expand upon Wallach’s study. We rotated listeners in a specialized chair at constant velocity. We presented filtered Gaussian noises at bandwidths of one-tenth octave, two octaves, and broadband, using center frequencies of 500 Hz and 4 kHz from a ring of 24 azimuthal loudspeakers located at pinna height. Sounds could also be presented from loudspeakers elevated relative to pinna level. The relative rates of sound and listener rotation around the azimuth plane were varied according to the relationship established by Wallach (1939) and listeners made judgments about the perceived elevation and rotation of the sounds. [Partially supported by a grant from the National Institute on Deafness and Other Communication Disorders, NIDCD.]

Sound localization models as evaluation tools for tactical communication and protective systems.

Tactical Communication and Protective Systems (TCAPS) are hearing protection devices that sufficiently protect the listener's ears from hazardous sounds and preserve speech intelligibility. However, previous studies demonstrated that TCAPS still deteriorate the listener's situational awareness, in particular, the ability to locate sound sources. On the horizontal plane, this is mainly explained by the degradation of the acoustical cues normally preventing the listener from making front-back confusions. As part of TCAPS development and assessment, a method predicting the TCAPS-induced degradation of the sound localization capability based on electroacoustic measurements would be more suitable than time-consuming behavioral experiments. In this context, the present paper investigates two methods based on Head-Related Transfer Functions (HRTFs): a template-matching model and a three-layer neural network. They are optimized to fit human sound source identification performance in open ear condition. The methods are applied to HRTFs measured with six TCAPS, providing identification probabilities. They are compared with the results of a behavioral experiment, conducted with the same protectors, and which ranks the TCAPS by type. The neural network predicts realistic performances with earplugs, but overestimates errors with earmuffs. The template-matching model predicts human performance well, except for two particular TCAPS.

Individualization of head-related transfer functions by the principle component analysis based on anthropometric measurements

A binaural reproduction either uses individual or generic head-related functions (HRTFs). The assessment of individual spatial high-resolution HRTF datasets is very time-consuming and requires a special measurement setup. On the other hand, a generic HRTF dataset from a dummy head can lead to localization errors such as front-back confusions and in-head localizations. In the present work, the generation of individualized HRTF datasets using the principle component analysis and multiple regression analysis from a generic HRTF dataset is discussed. The individualized HRTFs reduce the localization errors by taking individual anthropometric dimensions of the head and pinna into account. The method is applied to 48 individual HRTF datasets to determine the principal components and their weights. The anthropometric dimensions are obtained from the 3D magnetic resonance imaging scans of the 48 subjects. The focus of the work is on the number of necessary principle components with respect to the measurement grid and redundancy of anthropometric dimensions.

Effect of individualization of interaural time/level differences with non-individualized head-related transfer functions on sound localization

Head-related transfer functions (HRTFs) are known to include comprehensive auditory cues for sound source positions and show large inter-subject variation. Therefore, individualization of HRTFs is important for highly accurate sound localization systems such as virtual auditory displays. In the present study, we assume that the interaural time deffierence (ITD) and the interaural level difference (ILD) can be estimated from the listener’s anthropometric parameters. The individualization of the HRTFs is achieved by replacing the ITDs and ILDs of the non-individualized HRTFs with the listener’s ones. In this report, the effect of the individualization was evaluated by listening experiment. The non-individual HRTFs were obtained by interpolating magnitude response of the listener’s own HRTF and the flat ones. The ratios of the interpolation corresponded to the degrees of individuality. Then, the ITDs and ILDs were added to individualize HRTFs. From the results, the effect of the degree of individuality was insignificant, but the front-back confusions were significantly different between HRTFs on ipsilateral side and contralateral side. The results suggested that the individualization of HRTFs with ITDs/ILDs might be influenced by the spectrum of non-individual HRTFs.

Modification of Osseointegrated Device Parameters to Improve Speech in Noise and Localization Ability: Clinical Recommendations.

To determine how best to modify osseointegrated (OI) devices or environmental settings to maximize hearing performance. Prospective cohort study. Tertiary referral center. Fourteen adults with single-sided deafness (SSD) with a minimum of 6 months OI usage and nine bilaterally normal hearing controls : Speech in noise (SIN) and localization ability were assessed in a multi-speaker array (R-Space) with patients repeating sentences embedded in competing noise and verbally indicating the source speaker, respectively. SIN and localization were assessed with multiple OI microphone settings-fixed-directional, omnidirectional, and adaptive-as well as an unaided (OI off) condition. Participants completed the Abbreviated Profile of Hearing Aid Benefit questionnaire. Localization performance remains compromised for OI users with a high number of front-back confusions, but rapid learning using the fixed-directional microphone setting improved localization of sounds on the device side despite poorer localization of sounds on the normal-hearing side. SIN performance is greatly enhanced with speech presented to the contra hearing ear rather than the OI device side. Subjective report of hearing ability is highly predictive of objective SIN measures. Clinicians should consider implementing a fixed-directional microphone setting for improved localization for sounds behind the OI device, but inform patients of the trade-off in performance on the normal-hearing side. For better hearing in noise, clinicians should counsel OI recipients to orient the speech signal to their normal hearing ear rather than their OI device. The background noise subscale of the abbreviated profile of hearing aid benefit (APHAB) provides a meaningful metric by which to assess SIN performance of OI device users.

Influence of the Quality of Consumer Headphones in the Perception of Spatial Audio

High quality headphones can generate a realistic sound immersion reproducing binaural recordings. However, most people commonly use consumer headphones of inferior quality, as the ones provided with smartphones or music players. Factors, such as weak frequency response, distortion and the sensitivity disparity between the left and right transducers could be some of the degrading factors. In this work, we are studying how these factors affect spatial perception. To this purpose, a series or perceptual tests have been carried out with a virtual headphone listening test methodology. The first experiment focuses on the analysis of how the disparity of sensitivity between the two transducers affects the final result. The second test studies the influence of the frequency response relating quality and spatial impression. The third test analyzes the effects of distortion using a Volterra kernels scheme for the simulation of the distortion using convolutions. Finally, the fourth tries to relate the quality of the frequency response with the accuracy on azimuth localization. The conclusions of the experiments are: the disparity between both transducers can affect the localization of the source; the perception of quality and spatial impression has a high correlation; the distortion produced by the range of headphones tested at a fixed level does not affect the perception of binaural sound; and that some frequency bands have an important role in the front-back confusions.

Interactions between front-back confusion and visual capture in summing localization

Ambiguity in binaural timing and level information often causes front-back confusions in sound localization. This experiment investigated the extent to which front-back confusions are modulated by concurrent visual stimuli. 15-ms duration noise stimuli were presented over two loudspeakers positioned at ±45o in front or behind a listener with a delay between them of −1—1 ms in steps of 0.5 ms, thus evoking summing localization. These sound stimuli were presented with or without light-flash stimuli at three different frontal locations. In comparison to audio only trials, when visual stimuli were also presented front-back confusion decreased for sound stimuli presented in front and increased for rear-presented auditory stimuli. This effect was greater for phantom sound sources than single speaker controls. These results suggest that visual cues influence the perceived location of a sound source when it is outside the field of vision. Further, the phantom sound sources presented using “summing localization” may offer less robust localization cues than physical sound sources, resulting in greater front-back confusion and stronger visual capture. Acoustic analyses of binaural and spectral cues are conducted to identify potential causes of the differences in FBC rates between single speaker and phantom sound sources.

Transaural experiments and a revised duplex theory for the localization of low-frequency tones.

The roles of interaural time difference (ITD) and interaural level difference (ILD) were studied in free-field source localization experiments for sine tones of low frequency (250-750 Hz). Experiments combined real-source trials with virtual trials created through transaural synthesis based on real-time ear canal measurements. Experiments showed the following: (1) The naturally occurring ILD is physically large enough to exert an influence on sound localization well below 1000 Hz. (2) An ILD having the same sign as the ITD modestly enhances the perceived azimuth of tones for all values of the ITD, and it eliminates left-right confusions that otherwise occur when the interaural phase difference (IPD) passes 180°. (3) Increasing the ILD to large, implausible values can decrease the perceived laterality while also increasing front-back confusions. (4) Tone localization is more directly related to the ITD than to the IPD. (5) An ILD having a sign opposite to the ITD promotes a slipped-cycle ITD, sometimes with dramatic effects on localization. Because the role of the ITD itself is altered by the ILD, the duplex processing of ITD and ILD reflects more than mere trading; the effect of the ITD can be reversed in sign.

Multimodal Perceptual Training for Improving Spatial Auditory Performance in Blind and Sighted Listeners

Abstract The use of individualised Head Related Transfer Functions (HRTF) is a fundamental prerequisite for obtaining an accurate rendering of 3D spatialised sounds in virtual auditory environments. The HRTFs are transfer functions that define the acoustical basis of auditory perception of a sound source in space and are frequently used in virtual auditory displays to simulate free-field listening conditions. However, they depend on the anatomical characteristics of the human body and significantly vary among individuals, so that the use of the same dataset of HRTFs for all the users of a designed system will not offer the same level of auditory performance. This paper presents an alternative approach to the use on non-individualised HRTFs that is based on a procedural learning, training, and adaptation to altered auditory cues.We tested the sound localisation performance of nine sighted and visually impaired people, before and after a series of perceptual (auditory, visual, and haptic) feedback based training sessions. The results demonstrated that our subjects significantly improved their spatial hearing under altered listening conditions (such as the presentation of 3D binaural sounds synthesised from non-individualized HRTFs), the improvement being reflected into a higher localisation accuracy and a lower rate of front-back confusion errors.

Scalable Multiband Binaural Renderer for MPEG-H 3D Audio

To provide immersive 3D multimedia service, MPEG has launched MPEG-H, ISO/IEC 23008, “High Efficiency Coding and Media Delivery in Heterogeneous Environments.” As part of the audio, MPEG-H 3D Audio has been standardized based on a multichannel loudspeaker configuration (e.g., 22.2). Binaural rendering is a key application of 3D audio; however, previous studies focus on binaural rendering with low complexity such as IIR filter design for HRTF or pre-/post-processing to solve in-head localization or front-back confusion. In this paper, a new binaural rendering algorithm is proposed to support the large number of input channel signals and provide high-quality in terms of timbre, parts of this algorithm were adopted into the MPEG-H 3D Audio. The proposed algorithm truncates binaural room impulse response at mixing time, the transition point from the early-reflections to the late reverberation part. Each part is processed independently by variable order filtering in frequency domain (VOFF) and parametric late reverberation filtering (PLF), respectively. Further, a QMF domain tapped delay line (QTDL) is proposed to reduce complexity in the high-frequency band, based on human auditory perception and codec characteristics. In the proposed algorithm, a scalability scheme is adopted to cover a wide range of applications by adjusting the threshold of mixing time. Experimental results show that the proposed algorithm is able to provide the audio quality of a binaural rendered signal using full-length binaural room impulse responses. A scalability test also shows that the proposed scalability scheme smoothly compromises between audio quality and computational complexity.

Biomimetic direction of arrival estimation for resolving front-back confusions in hearing aids.

Sound sources at the same angle in front or behind a two-microphone array (e.g., bilateral hearing aids) produce the same time delay and two estimates for the direction of arrival: A front-back confusion. The auditory system can resolve this issue using head movements. To resolve front-back confusion for hearing-aid algorithms, head movement was measured using an inertial sensor. Successive time-delay estimates between the microphones are shifted clockwise and counterclockwise by the head movement between estimates and aggregated in two histograms. The histogram with the largest peak after multiple estimates predicted the correct hemifield for the source, eliminating the front-back confusions.

Front-back confusions when sources and listeners rotate

Tonal stimuli presented from directly in-front or directly in-back of a stationary listener are often miss-located in that sounds presented from in-front are occasionally perceived as being in-back of the listener; and often sounds presented in-back of a listener are perceived as being in-front. These types of front-back/back-front confusions in the azimuth plane were investigated as a function of the type of stimuli (e.g., tones, speech, and wideband noise) used and as a function of the sound or listener rotating around an azimuthal 24-loudspeaker array. Front-back confusions/back-front confusions depended on the type of stimulus; and on the number of loudspeaker positions the sound rotated through and the rate of sound or listener rotation. For example, if the sound rotated through two loudspeakers from in-front to in-back, there were a large number of confusions; if the sound rotated through each of the 24 loudspeakers on the circular array there were almost no confusions. [Research supported by an AFOSR grant.]

Quadri-stability of a spatially ambiguous auditory illusion.

In addition to vision, audition plays an important role in sound localization in our world. One way we estimate the motion of an auditory object moving towards or away from us is from changes in volume intensity. However, the human auditory system has unequally distributed spatial resolution, including difficulty distinguishing sounds in front vs. behind the listener. Here, we introduce a novel quadri-stable illusion, the Transverse-and-Bounce Auditory Illusion, which combines front-back confusion with changes in volume levels of a nonspatial sound to create ambiguous percepts of an object approaching and withdrawing from the listener. The sound can be perceived as traveling transversely from front to back or back to front, or “bouncing” to remain exclusively in front of or behind the observer. Here we demonstrate how human listeners experience this illusory phenomenon by comparing ambiguous and unambiguous stimuli for each of the four possible motion percepts. When asked to rate their confidence in perceiving each sound’s motion, participants reported equal confidence for the illusory and unambiguous stimuli. Participants perceived all four illusory motion percepts, and could not distinguish the illusion from the unambiguous stimuli. These results show that this illusion is effectively quadri-stable. In a second experiment, the illusory stimulus was looped continuously in headphones while participants identified its perceived path of motion to test properties of perceptual switching, locking, and biases. Participants were biased towards perceiving transverse compared to bouncing paths, and they became perceptually locked into alternating between front-to-back and back-to-front percepts, perhaps reflecting how auditory objects commonly move in the real world. This multi-stable auditory illusion opens opportunities for studying the perceptual, cognitive, and neural representation of objects in motion, as well as exploring multimodal perceptual awareness.

Eight-choice sound localization by manatees: performance abilities and head related transfer functions.

Two experiments investigated the ability and means by which two male Florida manatees (Trichechus manatus latirostris) may determine the direction of a sound source. An eight-choice discrimination paradigm was used to determine the subjects' sound localization abilities of five signal conditions covering a range of frequencies, durations, and levels. Subjects performed above the 12.5% chance level for all broadband frequencies and were able to localize sounds over a large level range. Errors were typically located to either side of the signal source location when presented in the front 180° but were more dispersed when presented from locations behind the subject. Front-to-back confusions were few and accuracy was greater when signals originated from the front 180°. Head-related transfer functions were measured to determine if frequencies were filtered by the manatee body to create frequency-specific interaural level differences (ILDs). ILDs were found for all frequencies as a function of source location, although they were largest with frequencies above 18 kHz and when signals originated to either side of the subjects. Larger ILDs were found when the signals originated behind the subjects. A shadowing-effect produced by the body may explain the relatively low occurrence of front-back confusions in the localization study.

Personalization of head-related transfer functions in the median plane based on the anthropometry of the listener's pinnae.

A listener's own head-related transfer functions (HRTFs) are required for accurate three-dimensional sound image control. The HRTFs of other listeners often cause front-back confusion and errors in the perception of vertical angles. However, measuring the HRTFs of all listeners for all directions of a sound source is impractical because the measurement requires a special apparatus and a lot of time. The present study proposes a method for estimating the appropriate HRTFs for an individual listener. The proposed method estimates the frequencies of the two lowest spectral notches (N1 and N2), which play an important role in vertical localization, in the HRTF of an individual listener by anthropometry of the listener's pinnae. The best-matching HRTFs, of which N1 and N2 are the closest to the estimates, are then selected from an HRTF database. In order to examine the validity of the proposed method, localization tests in the upper median plane were performed using four subjects. The results revealed that the best-matching HRTFs provided approximately the same performance as the listener's own HRTFs for the target directions of the front and rear for all four subjects. For the upper target directions, however, the performance of the localization for some of the subjects decreased.

Impact of hearing protection devices on sound localization performance

Hearing Protection Devices (HPDs) can protect the ear against loud potentially damaging sounds while allowing lower-level sounds such as speech to be perceived. However, the impact of these devices on the ability to localize sound sources is not well known. To address this question, we propose two different methods: one behavioral and one dealing with acoustical measurements. For the behavioral method, sound localization performance was measured with, and without, HPDs on 20 listeners. Five HPDs, including both passive (non-linear attenuation) and three active (talk-through) systems were evaluated. The results showed a significant increase in localization errors, especially front-back and up-down confusions relative to the “naked ear” test condition for all of the systems tested, especially for the talk-through headphone system. For the acoustic measurement method, Head-Related Transfer Functions (HRTFs) were measured on an artificial head both without, and with the HPDs in place. The effects of the HPDs on the spectral cues for the localization of different sound sources in the horizontal plane were analyzed. Alterations of the Interaural Spectral Difference (ISD) cues were identified, which could explain the observed increase in front-back confusions caused by the talk-through headphone protectors.

The role of interaural level differences in the localization of low-frequency sine tones

Five human listeners reported the azimuthal locations of low-frequency sine tones presented in free field, either by a 180-degree loudspeaker array or by virtual reality. The virtual sources were synthesized using cross-talk cancellation based on signals continuously monitored in the listener's ear canals. The experiment tested the duplex model of sound localization, especially the role of interaural level differences (fixed ILD = 0, 6, or 12 dB) for frequencies of 750 Hz or less. Trials with real sources, baseline virtual sources, and virtual sources with manipulated ILDs were always combined in all experiments. Experiments showed that the interaural time differences (ITD) dominated most fixed opposing ILDs, as previously reported, only when interaural phase differences (IPD) were less than 90 degrees. When IPDs exceeded 90 degrees, the ITD lost its influence, and localization was dominated by ILDs. Localization judgments never followed IPDs across a 180-degree boundary, except when a zeroed ILD caused judgments to become chaotic. Within the 90-degree IPD limit, judgments for fixed ILD appeared to follow the ITD better than the IPD. Abnormal interaural conditions (e.g., ITDs and ILDs of opposite sign) led to a notable increase in front-back confusions. [Work supported by AFOSR grant FA9550-11-1-0101.]