Auditory Spatial Cues Research Articles

Generalisation to novel exemplars of learned shape categories based on visual and auditory spatial cues does not benefit from multisensory information.

Although the integration of information across multiple senses can enhance object representations in memory, how multisensory information affects the formation of categories is uncertain. In particular, it is unclear to what extent categories formed from multisensory information benefit object recognition over unisensory inputs. Two experiments investigated the categorisation of novel auditory and visual objects, with categories defined by spatial similarity, and tested generalisation to novel exemplars. Participants learned to categorise exemplars based on visual-only (geometric shape), auditory-only (spatially defined soundscape) or audio-visual spatial cues. Categorisation to learned as well as novel exemplars was then tested under the same sensory learning conditions. For all learning modalities, categorisation generalised to novel exemplars. However, there was no evidence of enhanced categorisation performance for learned multisensory exemplars. At best, bimodal performance approximated that of the most accurate unimodal condition, although this was observed only for a subset of exemplars within a category. These findings provide insight into the perceptual processes involved in the formation of categories and have relevance for understanding the sensory nature of object representations underpinning these categories.

Dual Representation of the Auditory Space.

Auditory spatial cues contribute to two distinct functions, of which one leads to explicit localization of sound sources and the other provides a location-linked representation of sound objects. Behavioral and imaging studies demonstrated right-hemispheric dominance for explicit sound localization. An early clinical case study documented the dissociation between the explicit sound localizations, which was heavily impaired, and fully preserved use of spatial cues for sound object segregation. The latter involves location-linked encoding of sound objects. We review here evidence pertaining to brain regions involved in location-linked representation of sound objects. Auditory evoked potential (AEP) and functional magnetic resonance imaging (fMRI) studies investigated this aspect by comparing encoding of individual sound objects, which changed their locations or remained stationary. Systematic search identified 1 AEP and 12 fMRI studies. Together with studies of anatomical correlates of impaired of spatial-cue-based sound object segregation after focal brain lesions, the present evidence indicates that the location-linked representation of sound objects involves strongly the left hemisphere and to a lesser degree the right hemisphere. Location-linked encoding of sound objects is present in several early-stage auditory areas and in the specialized temporal voice area. In these regions, emotional valence benefits from location-linked encoding as well.

The addition of a spatial auditory cue improves spatial updating in a virtual reality navigation task.

Auditory cues are integrated with vision and body-based self-motion cues for motion perception, balance, and gait, though limited research has evaluated their effectiveness for navigation. Here, we tested whether an auditory cue co-localized with a visual target could improve spatial updating in a virtual reality homing task. Participants navigated a triangular homing task with and without an easily localizable spatial audio signal co-located with the home location. The main outcome was unsigned angular error, defined as the absolute value of the difference between the participant's turning response and the correct response towards the home location. Angular error was significantly reduced in the presence of spatial sound compared to a head-fixed identical auditory signal. Participants' angular error was 22.79° in the presence of spatial audio and 30.09° in its absence. Those with the worst performance in the absence of spatial sound demonstrated the greatest improvement with the added sound cue. These results suggest that auditory cues may benefit navigation, particularly for those who demonstrated the highest level of spatial updating error in the absence of spatial sound.

The neural representation of an auditory spatial cue in the primate cortex

Humans make use of small differences in the timing of sounds at the two ears-interaural time differences (ITDs)-to locate their sources. Despite extensive investigation, however, the neural representation of ITDs in the human brain is contentious, particularly the range of ITDs explicitly represented by dedicated neural detectors. Here, using magneto- and electro-encephalography (MEG and EEG), we demonstrate evidence of a sparse neural representation of ITDs in the human cortex. The magnitude of cortical activity to sounds presented via insert earphones oscillated as a function of increasing ITD-within and beyond auditory cortical regions-and listeners rated the perceptual quality of these sounds according to the same oscillating pattern. This pattern was accurately described by a population of model neurons with preferred ITDs constrained to the narrow, sound-frequency-dependent range evident in other mammalian species. When scaled for head size, the distribution of ITD detectors in the human cortex is remarkably like that recorded invivo from the cortex of rhesus monkeys, another large primate that uses ITDs for source localization. The data solve a long-standing issue concerning the neural representation of ITDs in humans and suggest a representation that scales for head size and sound frequency in an optimal manner.

Perceptual consequences of reverberant environments on spatial unmasking

Spatial hearing provides access to auditory spatial cues that promote speech perception in noisy listening situations. However, reverberation degrades auditory spatial cues and limits listeners’ ability to utilize these cues for segregating target speech from competing babble. Hence, spatial unmasking—an intelligibility benefit from a spatial separation between a target and masker—is reduced in reverberant environments as compared to free field. To understand the perceptual consequences of poorer spatial unmasking in reverberation, we assessed three aspects of functional spatial hearing in virtual reverberant environments: perceived auditory source width, auditory spatial acuity, and spatial unmasking. Three auditory environments were simulated and auralized using ODEON to vary interaural coherence (IC): (1) a control anechoic environment, (2) a classroom designed to meet classroom acoustics standards (IC = 0.58), and (3) a classroom of the same size with more severe reverberation (IC = 0.37). Individually measured head-related transfer functions were used to binaurally reproduce the auralized signals over headphones. We hypothesize that interaural decorrelation, the result of increasing reverberation, will broaden the perceived auditory source width with a cascading effect of reduced auditory spatial acuity and subsequently poorer spatial unmasking. Preliminary data from normal-hearing adults will be presented. [Work supported by NIH R21DC020532.]

Auditory Spatial Bisection of Blind and Normally Sighted Individuals in Free Field and Virtual Acoustics.

Sound localization is an important ability in everyday life. This study investigates the influence of vision and presentation mode on auditory spatial bisection performance. Subjects were asked to identify the smaller perceived distance between three consecutive stimuli that were either presented via loudspeakers (free field) or via headphones after convolution with generic head-related impulse responses (binaural reproduction). Thirteen azimuthal sound incidence angles on a circular arc segment of ±24° at a radius of 3 m were included in three regions of space (front, rear, and laterally left). Twenty normally sighted (measured both sighted and blindfolded) and eight blind persons participated. Results showed no significant differences with respect to visual condition, but strong effects of sound direction and presentation mode. Psychometric functions were steepest in frontal space and indicated median spatial bisection thresholds of 11°-14°. Thresholds increased significantly in rear (11°-17°) and laterally left (20°-28°) space in free field. Individual pinna and torso cues, as available only in free field presentation, improved the performance of all participants compared to binaural reproduction. Especially in rear space, auditory spatial bisection thresholds were three to four times higher (i.e., poorer) using binaural reproduction than in free field. The results underline the importance of individual auditory spatial cues for spatial bisection, irrespective of access to vision, which indicates that vision may not be strictly necessary to calibrate allocentric spatial hearing.

Enhancing learning outcomes through multisensory integration: A fMRI study of audio-visual training in virtual reality

The integration of information from different sensory modalities is a fundamental process that enhances perception and performance in real and virtual environments (VR). Understanding these mechanisms, especially during learning tasks that exploit novel multisensory cue combinations provides opportunities for the development of new rehabilitative interventions.This study aimed to investigate how functional brain changes support behavioural performance improvements during an audio-visual (AV) learning task. Twenty healthy participants underwent a 30 min daily VR training for four weeks. The task was an AV adaptation of a ‘scanning training’ paradigm that is commonly used in hemianopia rehabilitation. Functional magnetic resonance imaging (fMRI) and performance data were collected at baseline, after two and four weeks of training, and four weeks post-training.We show that behavioural performance, operationalised as mean reaction time reduction in VR, significantly improves. In separate tests in a controlled laboratory environment, we showed that the behavioural performance gains in the VR training environment transferred to a significant mean RT reduction for the trained AV voluntary task on a computer screen. Enhancements were observed in both the visual-only and AV conditions, with the latter demonstrating a faster response time supported by the presence of audio cues. The behavioural learning effect also transfers to two additional tasks that were tested: a visual search task and an involuntary visual task.Our fMRI results reveal an increase in functional activation (BOLD signal) in multisensory brain regions involved in early-stage AV processing: the thalamus, the caudal inferior parietal lobe and cerebellum. These functional changes were only observed for the trained, multisensory, task and not for unimodal visual stimulation. Functional activation changes in the thalamus were significantly correlated to behavioural performance improvements.This study demonstrates that incorporating spatial auditory cues to voluntary visual training in VR leads to augmented brain activation changes in multisensory integration, resulting in measurable performance gains across tasks. The findings highlight the potential of VR-based multisensory training as an effective method for enhancing cognitive function and as a potentially valuable tool in rehabilitative programmes.

Conserved features of eye movement related eardrum oscillations (EMREOs) across humans and monkeys.

Auditory and visual information involve different coordinate systems, with auditory spatial cues anchored to the head and visual spatial cues anchored to the eyes. Information about eye movements is therefore critical for reconciling visual and auditory spatial signals. The recent discovery of eye movement-related eardrum oscillations (EMREOs) suggests that this process could begin as early as the auditory periphery. How this reconciliation might happen remains poorly understood. Because humans and monkeys both have mobile eyes and therefore both must perform this shift of reference frames, comparison of the EMREO across species can provide insights to shared and therefore important parameters of the signal. Here we show that rhesus monkeys, like humans, have a consistent, significant EMREO signal that carries parametric information about eye displacement as well as onset times of eye movements. The dependence of the EMREO on the horizontal displacement of the eye is its most consistent feature, and is shared across behavioural tasks, subjects and species. Differences chiefly involve the waveform frequency (higher in monkeys than in humans) and patterns of individual variation (more prominent in monkeys than in humans), and the waveform of the EMREO when factors due to horizontal and vertical eye displacements were controlled for. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Commentary on the article "Auditory spatial cueing reduces neglect after right-hemispheric stroke: A proof of concept study" by.

Commentary on the article "Auditory spatial cueing reduces neglect after right-hemispheric stroke: A proof of concept study" by.

Response to the Letter by Schenke etal. on "Auditory spatial cueing reduces neglect after righthemispheric stroke: A proof of concept study" by Kaufmann etal., 2022.

Response to the Letter by Schenke etal. on "Auditory spatial cueing reduces neglect after righthemispheric stroke: A proof of concept study" by Kaufmann etal., 2022.

Spatial release from masking in anechoic and reverberant environments

Listening with both ears provides children with access to binaural and monaural cues that are helpful for understanding speech in competing babbles. Specifically, when the target and masker are spatially separated, children can gain an intelligibility benefit which is known as spatial release from masking (SRM). Recent work [Peng et al., 2021 JASA] suggested that school-age children demonstrated immature SRM using binaural cues that are distorted by reverberation. In this follow-up study, we further investigate the effect of reverberant distortion on individual auditory spatial cues, namely binaural and monaural head shadow cues. We compare SRM between adults and school-age children with typical hearing using the novel measure of minimum angle of separation (MAS) between target and masker, for which individual achieves a 20% intelligibility gain, in both virtually simulated anechoic and reverberant environments. MAS was measured in both binaural and monaural hearing conditions, as well asymmetric versus asymmetric masker displacement to probe access to various auditory cues of interest. Preliminary results show that adults demonstrate similar SRM in anechoic and in reverberation of T60 = ∼0.6 s, but better (smaller MAS) when additional spatial cues become available. Reverberation effect on children’s SRM will be discussed. [Work supported by NIH-NIDCD.]

Reaching to Sounds Improves Spatial Hearing in Bilateral Cochlear Implant Users.

We assessed if spatial hearing training improves sound localization in bilateral cochlear implant (BCI) users and whether its benefits can generalize to untrained sound localization tasks. In 20 BCI users, we assessed the effects of two training procedures (spatial versus nonspatial control training) on two different tasks performed before and after training (head-pointing to sound and audiovisual attention orienting). In the spatial training, participants identified sound position by reaching toward the sound sources with their hand. In the nonspatial training, comparable reaching movements served to identify sound amplitude modulations. A crossover randomized design allowed comparison of training procedures within the same participants. Spontaneous head movements while listening to the sounds were allowed and tracked to correlate them with localization performance. During spatial training, BCI users reduced their sound localization errors in azimuth and adapted their spontaneous head movements as a function of sound eccentricity. These effects generalized to the head-pointing sound localization task, as revealed by greater reduction of sound localization error in azimuth and more accurate first head-orienting response, as compared to the control nonspatial training. BCI users benefited from auditory spatial cues for orienting visual attention, but the spatial training did not enhance this multisensory attention ability. Sound localization in BCI users improves with spatial reaching-to-sound training, with benefits to a nontrained sound localization task. These findings pave the way to novel rehabilitation procedures in clinical contexts.

Exploring Sonification Mapping Strategies for Spatial Auditory Guidance in Immersive Virtual Environments

Spatial auditory cues are important for many tasks in immersive virtual environments, especially guidance tasks. However, due to the limited fidelity of spatial sounds rendered by generic Head-Related Transfer Functions (HRTFs), sound localization usually has a limited accuracy, especially in elevation, which can potentially impact the effectiveness of auditory guidance. To address this issue, we explored whether integrating sonification with spatial audio can enhance the perceptions of auditory guidance cues so user performance in auditory guidance tasks can be improved. Specifically, we investigated the effects of sonification mapping strategy using a controlled experiment that compared four elevation sonification mapping strategies: absolute elevation mapping, unsigned relative elevation mapping, signed relative elevation mapping, and binary relative elevation mapping. In addition, we examined whether azimuth sonification mapping can further benefit the perception of spatial sounds. The results demonstrate that spatial auditory cues can be effectively enhanced by integrating elevation and azimuth sonification, where the accuracy and speed of guidance tasks can be significantly improved. In particular, the overall results suggest that binary relative elevation mapping is generally the most effective strategy among four elevation sonification mapping strategies, which indicates that auditory cues with clear directional information are key to efficient auditory guidance.

School-age children show poor use of spatial cues in reverberation for speech-in-speech perception

Understanding speech is particularly difficult for children when there is competing speech in the background. When the target and masker talkers are spatially separated, as compared to co-located, the access to corresponding auditory spatial cues can provide release from masking, resulting in an intelligibility gain for speech-in-speech perception. When tested in free-field environments, previous work showed that children demonstrate adult-like spatial release from masking (SRM) by 9–10 years of age. However, in indoor environments where most critical communications occur such as classrooms, reverberation distorts the critical auditory spatial cues that lead to reduced SRM among adults. Little is known about how children process distorted auditory spatial cues for SRM to aid speech-in-speech perception. In this work, we measure SRM in children in virtual reverberant environments that mimic typical learning spaces. We show free-field measurements overestimate SRM maturation in realistic indoor environments. Children show a more protracted development of SRM in reverberation, likely due to immaturities in using distorted auditory spatial cues.

No sound is more distracting than the one you're trying not to hear: delayed costs of mental control of task-irrelevant neutral and emotional sounds

BackgroundSuppressing intrusive thoughts can result in a post-suppression rebound effect where the same thoughts become hyperaccessible. The current study aimed to investigate if similar so-called rebound effects could be observed when people attempted to mentally suppress awareness of nonsensical auditory stimuli. Based on previous research on thought suppression and mental control in other domains, we hypothesized that attempting to suppress awareness of a task-irrelevant sound while under cognitive load would impact evaluations of the sound on affective dimensions and loudness, and result in increased general vigilance, as evidenced by faster responding on subsequent tasks.MethodsWe performed two experiments where participants in a suppression condition were initially instructed to mentally suppress awareness of a sound while performing a mentally challenging task. Participants in a control condition performed the same task without receiving any instructions regarding the sound. In Experiment 1, the sound was affectively neutral, while in Experiment 2 participants were presented with an inherently aversive (tinnitus-like) sound. After this initial phase, participants performed tasks measuring vigilance and attention, and were also asked to give subjective ratings of the sounds on a number of affective dimensions and loudness.ResultsIn Experiment 1, participants in the suppression condition showed faster response times on both a visual search task and an auditory spatial cueing task, as compared to participants in the control condition. Contrary to our predictions, participants in the suppression condition did not rate the distractor sound as louder than participants in the control condition, and there were no differences on affective dimensions. In Experiment 2, results revealed that participants in the suppression condition made more errors on a visual search task, specifically on trials where the previously suppressed sound was presented. In contrast to results from Experiment 1, participants in the suppression condition also rated the targeted sound as louder.ConclusionsThe findings provide preliminary support for a post-suppression rebound effect in the auditory domain and further suggest that this effect may be moderated by the emotional properties of the auditory stimulus.

Effects of Spatial Speech Presentation on Listener Response Strategy for Talker-Identification.

This study investigates effects of spatial auditory cues on human listeners' response strategy for identifying two alternately active talkers (“turn-taking” listening scenario). Previous research has demonstrated subjective benefits of audio spatialization with regard to speech intelligibility and talker-identification effort. So far, the deliberate activation of specific perceptual and cognitive processes by listeners to optimize their task performance remained largely unexamined. Spoken sentences selected as stimuli were either clean or degraded due to background noise or bandpass filtering. Stimuli were presented via three horizontally positioned loudspeakers: In a non-spatial mode, both talkers were presented through a central loudspeaker; in a spatial mode, each talker was presented through the central or a talker-specific lateral loudspeaker. Participants identified talkers via speeded keypresses and afterwards provided subjective ratings (speech quality, speech intelligibility, voice similarity, talker-identification effort). In the spatial mode, presentations at lateral loudspeaker locations entailed quicker behavioral responses, which were significantly slower in comparison to a talker-localization task. Under clean speech, response times globally increased in the spatial vs. non-spatial mode (across all locations); these “response time switch costs,” presumably being caused by repeated switching of spatial auditory attention between different locations, diminished under degraded speech. No significant effects of spatialization on subjective ratings were found. The results suggested that when listeners could utilize task-relevant auditory cues about talker location, they continued to rely on voice recognition instead of localization of talker sound sources as primary response strategy. Besides, the presence of speech degradations may have led to increased cognitive control, which in turn compensated for incurring response time switch costs.

Auditory spatial cueing reduces neglect after right-hemispheric stroke: A proof of concept study

Spatial neglect after right-hemispheric stroke, characterized by the failure to attend or respond to the contralesional space, is a strong negative outcome predictor. Neglect is a supramodal syndrome affecting not only the visual but also the auditory modality. Preliminary studies used this audio-visual cross-modal effect to show short-lasting effects on attention towards the neglected space. The aim of the present study was to introduce a new technique of auditory stimulation combining the unspecific effect of music (i.e., patients choose their preferred music) with the effects of auditory spatial cueing (i.e., the music is presented dynamically as moving from right to left). The effect of this new auditory stimulation technique was investigated in two proof-of-concept experiments using repeated-measures, cross-over designs including 21 patients with visual neglect after a first right-hemispheric stroke. In Experiment I (n = 9), neglect patients showed a significantly larger improvement in Letter Cancellation after listening to preferred music with than without auditory spatial cueing. After granting the feasibility of this new auditory stimulation technique, we investigated the long-term aftereffects in Experiment II (n = 12). Herefore, we used video-oculography during Free Visual Exploration, a sensitive and reliable tool to assess spatial attention over time. Listening to music with auditory spatialcueing – as compared to music without auditory spatialcueing – significantly improved neglect severity in terms of visual exploration behaviour for up to 3h. A voxel-based-lesion-symptom mapping analysis over all patients revealed that the response variability in listening to music with auditory spatial cueing is determined by the integrity of the right inferior parietal lobule, the second branch of the superior longitudinal fascicle, and parieto-parietal callosal fibres. Our study shows that listening to music with auditory spatial cueing significantly reduces neglect severity and has the potential to be used as an add-on in the neurorehabilitation of neglect.

Sound-source localization when listeners and/or sound sources move

In the dynamic everyday world, sound sources and/or listeners move when animals attempt to localize sound sources. Determining sound-source locations in these dynamic scenarios requires several computations by the brain. The auditory-spatial cues (Interaural Time Differences, ITDs; Interaural Level Differences, ITDS; and spectral cues associated with the head-related transfer function, HRTF) indicate the location of sound sources relative to the head. When the source and/or listener move, these cues change and by knowing the position of the head, the brain could determine which moved and where it moved based on the auditory-spatial cue changes. If interaural cues (ITDs or ILDs) are used for sound-source localization, cone-of-confusion errors occur, and head motion can resolve front-back, cone of-confusion errors. The elevation of a sound source, not effected by cone-of-confusion errors, might be able to be determined when both the source and listener move. This presentation will describe work in the Spatial Hearing Lab at ASU related to how sound source and listener motion affect sound source-location judgements in these scenarios. [Research supported by grants from NIDCD, WAY: R01-DC015214 and MTP: F32-DC016808.]

Binaural Modelling and Spatial Auditory Cue Analysis of 3D-Printed Ears.

In this work, a binaural model resembling the human auditory system was built using a pair of three-dimensional (3D)-printed ears to localize a sound source in both vertical and horizontal directions. An analysis on the proposed model was firstly conducted to study the correlations between the spatial auditory cues and the 3D polar coordinate of the source. Apart from the estimation techniques via interaural and spectral cues, the property from the combined direct and reverberant energy decay curve is also introduced as part of the localization strategy. The preliminary analysis reveals that the latter provides a much more accurate distance estimation when compared to approximations via sound pressure level approach, but is alone not sufficient to disambiguate the front-rear confusions. For vertical localization, it is also shown that the elevation angle can be robustly encoded through the spectral notches. By analysing the strengths and shortcomings of each estimation method, a new algorithm is formulated to localize the sound source which is also further improved by cross-correlating the interaural and spectral cues. The proposed technique has been validated via a series of experiments where the sound source was randomly placed at 30 different locations in an outdoor environment up to a distance of 19 m. Based on the experimental and numerical evaluations, the localization performance has been significantly improved with an average error of 0.5 m from the distance estimation and a considerable reduction of total ambiguous points to 3.3%.

The Role of Interaural Differences, Head Shadow, and Binaural Redundancy in Binaural Intelligibility Benefits Among School-Aged Children.

In complex listening environments, children can benefit from auditory spatial cues to understand speech in noise. When a spatial separation is introduced between the target and masker and/or listening with two ears versus one ear, children can gain intelligibility benefits with access to one or more auditory cues for unmasking: monaural head shadow, binaural redundancy, and interaural differences. This study systematically quantified the contribution of individual auditory cues in providing binaural speech intelligibility benefits for children with normal hearing between 6 and 15 years old. In virtual auditory space, target speech was presented from + 90° azimuth (i.e., listener's right), and two-talker babble maskers were either co-located (+ 90° azimuth) or separated by 180° (–90° azimuth, listener's left). Testing was conducted over headphones in monaural (i.e., right ear) or binaural (i.e., both ears) conditions. Results showed continuous improvement of speech reception threshold (SRT) between 6 and 15 years old and immature performance at 15 years of age for both SRTs and intelligibility benefits from more than one auditory cue. With early maturation of head shadow, the prolonged maturation of unmasking was likely driven by children's poorer ability to gain full benefits from interaural difference cues. In addition, children demonstrated a trade-off between the benefits from head shadow versus interaural differences, suggesting an important aspect of individual differences in accessing auditory cues for binaural intelligibility benefits during development.