Auditory Space Perception Research Articles

The power of vision: calibration of auditory space after sight restoration from congenital cataracts.

Early visual deprivation typically results in spatial impairments in other sensory modalities. It has been suggested that, since vision provides the most accurate spatial information, it is used for calibrating space in the other senses. Here we investigated whether sight restoration after prolonged early onset visual impairment can lead to the development of more accurate auditory space perception. We tested participants who were surgically treated for congenital dense bilateral cataracts several years after birth. In Experiment 1 we assessed participants' ability to understand spatial relationships among sounds, by asking them to spatially bisect three consecutive, laterally separated sounds. Participants performed better after surgery than participants tested before. However, they still performed worse than sighted controls. In Experiment 2, we demonstrated that single sound localization in the two-dimensional frontal plane improves quickly after surgery, approaching performance levels of sighted controls. Such recovery seems to be mediated by visual acuity, as participants gaining higher post-surgical visual acuity performed better in both experiments. These findings provide strong support for the hypothesis that vision calibrates auditory space perception. Importantly, this also demonstrates that this process can occur even when vision is restored after years of visual deprivation.

Effects of Visually Induced Self-Motion on Sound Localization Accuracy

The deterioration of sound localization accuracy during a listener’s head/body rotation is independent of the listener’s rotation velocity. However, whether this deterioration occurs only during physical movement in a real environment remains unclear. In this study, we addressed this question by subjecting physically stationary listeners to visually induced self-motion, i.e., vection. Two conditions—one with a visually induced perception of self-motion (vection) and the other without vection (control)—were adopted. Under both conditions, a short noise burst (30 ms) was presented via a loudspeaker in a circular array placed horizontally in front of a listener. The listeners were asked to determine whether the acoustic stimulus was localized relative to their subjective midline. The results showed that in terms of detection thresholds based on the subjective midline, the sound localization accuracy was lower under the vection condition than under the control condition. This indicates that sound localization can be compromised under visually induced self-motion perception. These findings support the idea that self-motion information is crucial for auditory space perception and can potentially enable the design of dynamic binaural displays requiring fewer computational resources.

Auditory Space Perception in the Blind: Horizontal Sound Localization in Acoustically Simple and Complex Situations.

Even though it is widely believed that cross-modal neuroplasticity in blindness results in enhanced auditory spatial abilities, a growing number of studies also indicate disadvantages of blind persons for specific, more demanding audiospatial tasks. Here, the effects of blindness on horizontal sound localization were compared for simple localization of target sounds presented in isolation and target localization in an acoustically complex ( cocktail-party) situation with multiple distractor sounds. While performances of blind and sighted subjects were similar in the cocktail-party setting, a significant disadvantage of blind subjects was found for simple localization of sound sources presented in the center of the frontal space, with the mean absolute error being more than twice that of controls for targets in the median plane. The latter result could be due to the lack of visual calibration of auditory space, which may, in sighted persons, be most effective in the region of maximum visual acuity. On the other hand, the normal performance of blind persons in cocktail-party localization could be explained by the fact that this task, in addition to genuine spatial analysis, critically involved spectrotemporal analysis of the auditory scene to separate different sources for which superior performance is known from previous research.

The Role of Visual Experience in Auditory Space Perception around the Legs

It is widely accepted that vision plays a key role in the development of spatial skills of the other senses. Recent works have shown that blindness is often associated with auditory spatial deficits. The majority of previous studies have focused on understanding the representation of the upper frontal body space where vision and actions have a central role in mapping the space, however less research has investigated the back space and the space around the legs. Here we investigate space perception around the legs and the role of previous visual experience, by studying sighted and blind participants in an audio localization task (front-back discrimination). Participants judged if a sound was delivered in their frontal or back space. The results showed that blindfolded sighted participants were more accurate than blind participants in the frontal space. However, both groups were similarly accurate when auditory information was delivered in the back space. Blind individuals performed the task with similar accuracy for sounds delivered in the frontal and back space, while sighted people performed better in the frontal space. These results suggest that visual experience influences auditory spatial representations around the legs. Moreover, these results suggest that hearing and vision play different roles in different spaces.

A novel concept for dynamic adjustment of auditory space

Traditionally, the auditory system is thought to serve reliable sound localization. Stimulus-history driven feedback circuits in the early binaural pathway, however, contradict this canonical concept and raise questions about their functional significance. Here we show that stimulus-history dependent changes in absolute space perception are poorly captured by the traditional labeled-line and hemispheric-difference models of auditory space coding. We therefore developed a new decoding model incorporating recent electrophysiological findings in which sound location is initially computed in both brain hemispheres independently and combined to yield a hemispherically balanced code. This model closely captures the observed absolute localization errors caused by stimulus history, and furthermore predicts a selective dilation and compression of perceptional space. These model predictions are confirmed by improvement and degradation of spatial resolution in human listeners. Thus, dynamic perception of auditory space facilitates focal sound source segregation at the expense of absolute sound localization, questioning existing concepts of spatial hearing.

Influence of Visual Prism Adaptation on Auditory Space Representation.

Prisms shifting the visual input sideways produce a mismatch between the visual versus felt position of one’s hand. Prism adaptation eliminates this mismatch, realigning hand proprioception with visual input. Whether this realignment concerns exclusively the visuo-(hand)motor system or it generalizes to acoustic inputs is controversial. We here show that there is indeed a slight influence of visual adaptation on the perceived direction of acoustic sources. However, this shift in perceived auditory direction can be fully explained by a subconscious head rotation during prism exposure and by changes in arm proprioception. Hence, prism adaptation does only indirectly generalize to auditory space perception.

Sensitivity to an Illusion of Sound Location in Human Auditory Cortex.

Human listeners place greater weight on the beginning of a sound compared to the middle or end when determining sound location, creating an auditory illusion known as the Franssen effect. Here, we exploited that effect to test whether human auditory cortex (AC) represents the physical vs. perceived spatial features of a sound. We used functional magnetic resonance imaging (fMRI) to measure AC responses to sounds that varied in perceived location due to interaural level differences (ILD) applied to sound onsets or to the full sound duration. Analysis of hemodynamic responses in AC revealed sensitivity to ILD in both full-cue (veridical) and onset-only (illusory) lateralized stimuli. Classification analysis revealed regional differences in the sensitivity to onset-only ILDs, where better classification was observed in posterior compared to primary AC. That is, restricting the ILD to sound onset—which alters the physical but not the perceptual nature of the spatial cue—did not eliminate cortical sensitivity to that cue. These results suggest that perceptual representations of auditory space emerge or are refined in higher-order AC regions, supporting the stable perception of auditory space in noisy or reverberant environments and forming the basis of illusions such as the Franssen effect.

Hearing Scenes: A Neuromagnetic Signature of Auditory Source and Reverberant Space Separation.

Perceiving the geometry of surrounding space is a multisensory process, crucial to contextualizing object perception and guiding navigation behavior. Humans can make judgments about surrounding spaces from reverberation cues, caused by sounds reflecting off multiple interior surfaces. However, it remains unclear how the brain represents reverberant spaces separately from sound sources. Here, we report separable neural signatures of auditory space and source perception during magnetoencephalography (MEG) recording as subjects listened to brief sounds convolved with monaural room impulse responses (RIRs). The decoding signature of sound sources began at 57 ms after stimulus onset and peaked at 130 ms, while space decoding started at 138 ms and peaked at 386 ms. Importantly, these neuromagnetic responses were readily dissociable in form and time: while sound source decoding exhibited an early and transient response, the neural signature of space was sustained and independent of the original source that produced it. The reverberant space response was robust to variations in sound source, and vice versa, indicating a generalized response not tied to specific source-space combinations. These results provide the first neuromagnetic evidence for robust, dissociable auditory source and reverberant space representations in the human brain and reveal the temporal dynamics of how auditory scene analysis extracts percepts from complex naturalistic auditory signals.

Relative head angle and eye position drive systematic distortions in the perception of auditory space

The minimum audible movement angle (MAMA) increases as a function of azimuth. This study examined a potential consequence of this: if this change in resolution across azimuth is not perceptually compensated for it would suggest that a sound rotating at a constant angular velocity around the head would not appear to do so but would appear to move faster at the front than at the side. We therefore examined whether the azimuth of two signals affects their point of subjective similarity for motion. We found that equal movement from two different azimuths is not judged as being equivalent: signals centered at 90° must move at least twice as much as ones centered at 0° to be judged as moving the same amount. Signals centered at 45° were also perceived in a proportionally distorted manner. Furthermore, in some listeners, this expansion/contraction of auditory space appears to shift systematically with eye position. These interactions suggest that the MAMA may not be simply a measure of acuity, but also reflects an underlying head- and eye-driven nonlinear perceptual organization of space, one we term the “equivalent arc,” in which an absolute number of degrees does not correspond to a consistent perceptual unit across azimuth.

Auditory space perception during self-motion

Spatial information inputted to the auditory periphery dramatically changes with a listener’s body movements relative to the sound source. Nevertheless, listeners can perceive a stable auditory environment and react appropriately to the sound source. This suggests that the spatial information is reinterpreted in the brain by being integrated with information regarding movement, while it is well known that the motion itself sometimes negatively affects sound localization acuity. We have focused on how people perceive auditory space during movement. A linear-motor-driven chair and a spinning chair were installed in an anechoic room to provide linear and rotatory motion, respectively. The results revealed that the perceived sound position during linear self-motion was displaced compared with the physical sound position. This displacement was observed regardless of the self-motion direction and the inputted sensory information which induces self-motion. Such degradation of sound localization acuity was also o...

Effect of visual and vestibular information on auditory space perception

We investigated how auditory space would be represented during linear self-motion and visually-induced self-motion (vection) (Teramoto et al., 2012, 2014). The previous studies indicated that the subjective coronal plane (SCP) was displaced during forward self-motion, while during backward motion in the case of vection. The present study investigated how auditory space perception would be altered when both visual and vestibular information were presented. A random-dot pattern simulating linear self-motion was used. At the beginning of the trial, the random-dot pattern started to move at a velocity of ±0.1 m/s. When an observer perceived vection, the velocity was changed at an acceleration of ±0.2, 0, and 0.4 m/s2. At the same time, the observer was moved forward at an acceleration of 0.2 m/s2 by using a linear-motor-driven chair. A short noise-burst was presented from a loudspeaker when the observer moved 2 m. The observers indicated the direction in which the sound was perceived relative to their coronal plane. The results showed that the direction of the SCP shift was different only when the acceleration of both visual and vestibular information was identical. These results suggest that auditory space distortion effect occurs closely related to the integration of vestibular and visual information.

Six Degrees of Auditory Spatial Separation.

The location of a sound is derived computationally from acoustical cues rather than being inherent in the topography of the input signal, as in vision. Since Lord Rayleigh, the descriptions of that representation have swung between "labeled line" and "opponent process" models. Employing a simple variant of a two-point separation judgment using concurrent speech sounds, we found that spatial discrimination thresholds changed nonmonotonically as a function of the overall separation. Rather than increasing with separation, spatial discrimination thresholds first declined as two-point separation increased before reaching a turning point and increasing thereafter with further separation. This "dipper" function, with a minimum at 6 ° of separation, was seen for regions around the midline as well as for more lateral regions (30 and 45°). The discrimination thresholds for the binaural localization cues were linear over the same range, so these cannot explain the shape of these functions. These data and a simple computational model indicate that the perception of auditory space involves a local code or multichannel mapping emerging subsequent to the binaural cue coding.

Editorial: How, and Why, Does Spatial-Hearing Ability Differ among Listeners? What is the Role of Learning and Multisensory Interactions?

Large individual differences are relatively common in human perception. Spatial hearing is not an exception; for instance, two listeners can perceive the same auditory target to be at very different spatial locations. Such variability cannot be considered as mere experimental noise but as true data that we have to use for explaining the mechanisms underlying the perception of auditory space. The 22 papers of this research topic explore individual differences in almost every aspect of auditory space perception.

Effect of Active Self-Motion on Auditory Space Perception

Effect of Active Self-Motion on Auditory Space Perception

Chapter 27 - Deafness in cochlear and auditory nerve disorders

Sensorineural hearing loss is the most common type of hearing impairment worldwide. It arises as a consequence of damage to the cochlea or auditory nerve, and several structures are often affected simultaneously. There are many causes, including genetic mutations affecting the structures of the inner ear, and environmental insults such as noise, ototoxic substances, and hypoxia. The prevalence increases dramatically with age. Clinical diagnosis is most commonly accomplished by measuring detection thresholds and comparing these to normative values to determine the degree of hearing loss. In addition to causing insensitivity to weak sounds, sensorineural hearing loss has a number of adverse perceptual consequences, including loudness recruitment, poor perception of pitch and auditory space, and difficulty understanding speech, particularly in the presence of background noise. The condition is usually incurable; treatment focuses on restoring the audibility of sounds made inaudible by hearing loss using either hearing aids or cochlear implants.

Distortion of auditory space during visually induced self-motion in depth.

Perception of self-motion is based on the integration of multiple sensory inputs, in particular from the vestibular and visual systems. Our previous study demonstrated that vestibular linear acceleration information distorted auditory space perception (Teramoto et al., 2012). However, it is unclear whether this phenomenon is contingent on vestibular signals or whether it can be caused by inputs from other sensory modalities involved in self-motion perception. Here, we investigated whether visual linear self-motion information can also alter auditory space perception. Large-field visual motion was presented to induce self-motion perception with constant accelerations (Experiment 1) and a constant velocity (Experiment 2) either in a forward or backward direction. During participants' experience of self-motion, a short noise burst was delivered from one of the loudspeakers aligned parallel to the motion direction along a wall to the left of the listener. Participants indicated from which direction the sound was presented, forward or backward, relative to their coronal (i.e., frontal) plane. Results showed that the sound position aligned with the subjective coronal plane (SCP) was significantly displaced in the direction of self-motion, especially in the backward self-motion condition as compared with a no motion condition. These results suggest that self-motion information, irrespective of its origin, is crucial for auditory space perception.

Neural Correlates of Realistic and Unrealistic Auditory Space Perception

Binaural recordings can simulate externalized auditory space perception over headphones. However, if the orientation of the recorder's head and the orientation of the listener's head are incongruent, the simulated auditory space is not realistic. For example, if a person lying flat on a bed listens to an environmental sound that was recorded by microphones inserted in ears of a person who was in an upright position, the sound simulates an auditory space rotated 90 degrees to the real-world horizontal axis. Our question is whether brain activation patterns are different between the unrealistic auditory space (ie, the orientation of the listener's head and the orientation of the recorder's head are incongruent) and the realistic auditory space (ie, the orientations are congruent). River sounds that were binaurally recorded either in a supine position or in an upright body position were served as auditory stimuli. During fMRI experiments, participants listen to the stimuli and pressed one of two buttons indi...

Tactile recalibration of auditory spatial representations

In the well-known spatial ventriloquism effect, auditory stimuli are mislocalized towards the location of synchronous but spatially disparate visual stimuli. Recent studies have demonstrated a similar influence of tactile stimuli on auditory localization, which predominantly operates in an external coordinate system. Here, we investigated whether this audio-tactile ventriloquist illusion leads to comparable aftereffects in the perception of auditory space as have been observed previously for audiovisual stimulation. Participants performed a relative sound localization task in which they had to judge whether a brief sound was perceived at the same or a different location as a preceding tactile stimulus ("Experiment 1") or to the left or right of a preceding visual stimulus ("Experiment 2"). Sound localization ability was measured before and after exposure to synchronous audio-tactile stimuli with a constant spatial disparity. After audio-tactile adaptation, unimodal sound localization was shifted in the direction of the tactile stimuli during the preceding adaptation phase in both tasks. This finding provides evidence for the existence of an audio-tactile ventriloquism aftereffect and suggests that auditory space (rather than specific audio-tactile connections) can be rapidly recalibrated to compensate for audio-tactile spatial disparities.

Perceptual load influences auditory space perception in the ventriloquist aftereffect

A period of exposure to trains of simultaneous but spatially offset auditory and visual stimuli can induce a temporary shift in the perception of sound location. This phenomenon, known as the ‘ventriloquist aftereffect’, reflects a realignment of auditory and visual spatial representations such that they approach perceptual alignment despite their physical spatial discordance. Such dynamic changes to sensory representations are likely to underlie the brain’s ability to accommodate inter-sensory discordance produced by sensory errors (particularly in sound localization) and variability in sensory transduction. It is currently unknown, however, whether these plastic changes induced by adaptation to spatially disparate inputs occurs automatically or whether they are dependent on selectively attending to the visual or auditory stimuli. Here, we demonstrate that robust auditory spatial aftereffects can be induced even in the presence of a competing visual stimulus. Importantly, we found that when attention is directed to the competing stimuli, the pattern of aftereffects is altered. These results indicate that attention can modulate the ventriloquist aftereffect.

Advancing age alters the influence of eye position on sound localization

Vision and audition provide spatial information about the environment to guide natural behavior. Because the eyes move in the head while the ears remain head-fixed, input conveying eye position in the head is required to maintain audiovisual congruence. Human perception of auditory space was previously shown to shift with changes in eye position, regardless of the target's frequency content and spatial cues underlying horizontal and vertical localization. In this study, we examined whether this interaction is altered by advancing age. Head-restrained young (18-44 yo), middle-aged (45-64 yo), and elderly (65-81 yo) human subjects localized noise bursts under conditions of transient and sustained ocular deflection. All three age groups demonstrated a time-dependent shift of auditory space in the direction of eye position. Moreover, this adaptation showed a clear decline with advancing age, but only for peripheral auditory space (beyond ±10° from midline). Alternatively, adaptation in the periphery may occur, but is more sluggish than in the central field and therefore not fully observed in this experiment. The age-dependent effect cannot be readily explained by senescent peripheral hearing loss, suggesting a change in central processing of auditory space in relation to the control of gaze.