Representation Of Auditory Space Research Articles

Visually-guided auditory adaptation and reference frame of the ventriloquism after effect

Ventriloquism aftereffect (VA) is observed as a shift in the perceived locations of auditory stimuli, induced by repeated presentation of audiovisual signals with incongruent locations of auditory and visual components. Since the two modalities use a different reference frame (RF), audition is head-centered (HC) while vision is eye-centered (EC), the representations have to be aligned. A previous study examining RF of VA found inconsistent results: the RF was a mixture of HC and EC for VA induced in the center of the audiovisual field, while it was predominantly HC for VA induced in the periphery [Lin et al., JASA 121, 3095, 2007]. In addition, the study found an adaptation in the auditory space representation even for congruent AV stimuli in the periphery. Here, a computational model examines the origins of these effects. The model assumes that multiple stages of processing interact: (1) the stage of auditory spatial representation (HC), (2) the stage of saccadic eye responses (EC), and (3) some stage at which the representation is mixed (HC + EC). Observed results are most consistent with a suggestion that the neural representation underlying spatial auditory plasticity incorporates both HC and EC auditory information, possibly at different processing stages. [Work supported by VEGA-1/1011/16.]

Distinct Correlation Structure Supporting a Rate-Code for Sound Localization in the Owl's Auditory Forebrain.

While a topographic map of auditory space exists in the vertebrate midbrain, it is absent in the forebrain. Yet, both brain regions are implicated in sound localization. The heterogeneous spatial tuning of adjacent sites in the forebrain compared to the midbrain reflects different underlying circuitries, which is expected to affect the correlation structure, i.e., signal (similarity of tuning) and noise (trial-by-trial variability) correlations. Recent studies have drawn attention to the impact of response correlations on the information readout from a neural population. We thus analyzed the correlation structure in midbrain and forebrain regions of the barn owl’s auditory system. Tetrodes were used to record in the midbrain and two forebrain regions, Field L and the downstream auditory arcopallium (AAr), in anesthetized owls. Nearby neurons in the midbrain showed high signal and noise correlations (RNCs), consistent with shared inputs. As previously reported, Field L was arranged in random clusters of similarly tuned neurons. Interestingly, AAr neurons displayed homogeneous monotonic azimuth tuning, while response variability of nearby neurons was significantly less correlated than the midbrain. Using a decoding approach, we demonstrate that low RNC in AAr restricts the potentially detrimental effect it can have on information, assuming a rate code proposed for mammalian sound localization. This study harnesses the power of correlation structure analysis to investigate the coding of auditory space. Our findings demonstrate distinct correlation structures in the auditory midbrain and forebrain, which would be beneficial for a rate-code framework for sound localization in the nontopographic forebrain representation of auditory space.

From “ear” to there: a review of biorobotic models of auditory processing in lizards

The peripheral auditory system of lizards has been extensively studied, because of its remarkable directionality. In this paper, we review the research that has been performed on this system using a biorobotic approach. The various robotic implementations developed to date, both wheeled and legged, of the auditory model exhibit strong phonotactic performance for two types of steering mechanisms-a simple threshold decision model and Braitenberg sensorimotor cross-couplings. The Braitenberg approach removed the need for a decision model, but produced relatively inefficient robot trajectories. Introducing various asymmetries in the auditory model reduced the efficiency of the robot trajectories, but successful phonotaxis was maintained. Relatively loud noise distractors degraded the trajectory efficiency and above-threshold noise resulted in unsuccessful phonotaxis. Machine learning techniques were applied to successfully compensate for asymmetries as well as noise distractors. Such techniques were also successfully used to construct a representation of auditory space, which is crucial for sound localisation while remaining stationary as opposed to phonotaxis-based localisation. The peripheral auditory model was furthermore found to adhere to an auditory scaling law governing the variation in frequency response with respect to physical ear separation. Overall, the research to date paves the way towards investigating the more fundamental topic of auditory metres versus auditory maps, and the existing robotic implementations can act as tools to compare the two approaches.

Sound-source enumeration by hearing-impaired adults

To help measure the veracity with which the auditory world is heard by hearing-impaired listeners, we studied the ability to count the number of discrete sources that are present. In two experiments, stimuli were newly selected 5-s samples of concatenated sentences from 1 to 7 locations in simulated rooms of varying reverberation, presented from a circular loudspeaker array. In the first experiment, listeners responded with the number of locations heard after each presentation. In the second experiment, listeners heard two presentations, an interval with n sources (one talker per location) and an interval with n + 1 sources; listeners responded which interval had more sources. Results corroborate recent work showing the maximum number of identifiable sources is roughly four. Asymmetry in hearing impairment reduced the ability to enumerate locations. Various conditions of presentation were tested: headphone and aided results were not markedly different from unaided results, and reverberation had only a modest inflating effect on the perceived number of sources. The ramifications for complex listening as well as the potential relation to cortical representations of auditory space are discussed. [Work supported by the MRC (U135097131) and the Chief Scientist Office (Scotland).]

Biases in Visual, Auditory, and Audiovisual Perception of Space.

Localization of objects and events in the environment is critical for survival, as many perceptual and motor tasks rely on estimation of spatial location. Therefore, it seems reasonable to assume that spatial localizations should generally be accurate. Curiously, some previous studies have reported biases in visual and auditory localizations, but these studies have used small sample sizes and the results have been mixed. Therefore, it is not clear (1) if the reported biases in localization responses are real (or due to outliers, sampling bias, or other factors), and (2) whether these putative biases reflect a bias in sensory representations of space or a priori expectations (which may be due to the experimental setup, instructions, or distribution of stimuli). Here, to address these questions, a dataset of unprecedented size (obtained from 384 observers) was analyzed to examine presence, direction, and magnitude of sensory biases, and quantitative computational modeling was used to probe the underlying mechanism(s) driving these effects. Data revealed that, on average, observers were biased towards the center when localizing visual stimuli, and biased towards the periphery when localizing auditory stimuli. Moreover, quantitative analysis using a Bayesian Causal Inference framework suggests that while pre-existing spatial biases for central locations exert some influence, biases in the sensory representations of both visual and auditory space are necessary to fully explain the behavioral data. How are these opposing visual and auditory biases reconciled in conditions in which both auditory and visual stimuli are produced by a single event? Potentially, the bias in one modality could dominate, or the biases could interact/cancel out. The data revealed that when integration occurred in these conditions, the visual bias dominated, but the magnitude of this bias was reduced compared to unisensory conditions. Therefore, multisensory integration not only improves the precision of perceptual estimates, but also the accuracy.

Encoding audio motion: spatial impairment in early blind individuals.

The consequence of blindness on auditory spatial localization has been an interesting issue of research in the last decade providing mixed results. Enhanced auditory spatial skills in individuals with visual impairment have been reported by multiple studies, while some aspects of spatial hearing seem to be impaired in the absence of vision. In this study, the ability to encode the trajectory of a 2-dimensional sound motion, reproducing the complete movement, and reaching the correct end-point sound position, is evaluated in 12 early blind (EB) individuals, 8 late blind (LB) individuals, and 20 age-matched sighted blindfolded controls. EB individuals correctly determine the direction of the sound motion on the horizontal axis, but show a clear deficit in encoding the sound motion in the lower side of the plane. On the contrary, LB individuals and blindfolded controls perform much better with no deficit in the lower side of the plane. In fact the mean localization error resulted 271 ± 10 mm for EB individuals, 65 ± 4 mm for LB individuals, and 68 ± 2 mm for sighted blindfolded controls. These results support the hypothesis that (i) it exists a trade-off between the development of enhanced perceptual abilities and role of vision in the sound localization abilities of EB individuals, and (ii) the visual information is fundamental in calibrating some aspects of the representation of auditory space in the brain.

Change in the coding of interaural time difference along the tonotopic axis of the chicken nucleus laminaris.

Interaural time differences (ITDs) are an important cue for the localization of sounds in azimuthal space. Both birds and mammals have specialized, tonotopically organized nuclei in the brain stem for the processing of ITD: medial superior olive in mammals and nucleus laminaris (NL) in birds. The specific way in which ITDs are derived was long assumed to conform to a delay-line model in which arrays of systematically arranged cells create a representation of auditory space with different cells responding maximally to specific ITDs. This model was supported by data from barn owl NL taken from regions above 3 kHz and from chicken above 1 kHz. However, data from mammals often do not show defining features of the Jeffress model such as a systematic topographic representation of best ITDs or the presence of axonal delay lines, and an alternative has been proposed in which neurons are not topographically arranged with respect to ITD and coding occurs through the assessment of the overall response of two large neuron populations, one in each hemisphere. Modeling studies have suggested that the presence of different coding systems could be related to the animal’s head size and frequency range rather than their phylogenetic group. Testing this hypothesis requires data from across the tonotopic range of both birds and mammals. The aim of this study was to obtain in vivo recordings from neurons in the low-frequency range (<1000 Hz) of chicken NL. Our data argues for the presence of a modified Jeffress system that uses the slopes of ITD-selective response functions instead of their peaks to topographically represent ITD at mid- to high frequencies. At low frequencies, below several 100 Hz, the data did not support any current model of ITD coding. This is different to what was previously shown in the barn owl and suggests that constraints in optimal ITD processing may be associated with the particular demands on sound localization determined by the animal’s ecological niche in the same way as other perceptual systems such as field of best vision.

Response actions influence the categorization of directions in auditory space.

Spatial region concepts such as “front,” “back,” “left,” and “right” reflect our typical interaction with space, and the corresponding surrounding regions have different statuses in memory. We examined the representation of spatial directions in the auditory space, specifically in how far natural response actions, such as orientation movements toward a sound source, would affect the categorization of egocentric auditory space. While standing in the middle of a circle with 16 loudspeakers, participants were presented acoustic stimuli coming from the loudspeakers in randomized order, and verbally described their directions by using the concept labels “front,” “back,” “left,” “right,” “front-right,” “front-left,” “back-right,” and “back-left.” Response actions varied in three blocked conditions: (1) facing front, (2) turning the head and upper body to face the stimulus, and (3) turning the head and upper body plus pointing with the hand and outstretched arm toward the stimulus. In addition to a protocol of the verbal utterances, motion capture and video recording generated a detailed corpus for subsequent analysis of the participants’ behavior. Chi-square tests revealed an effect of response condition for directions within the left and right sides. We conclude that movement-based response actions influence the representation of auditory space, especially within the sides’ regions. Moreover, the representation of auditory space favors the front and the back regions in terms of resolution, which is possibly related to the physiological characteristics of the human auditory system, as well as to the ecological requirements of action control in the different regions.

Integrated processing of spatial cues in human auditory cortex

Human sound source localization relies on acoustical cues, most importantly, the interaural differences in time and level (ITD and ILD). For reaching a unified representation of auditory space the auditory nervous system needs to combine the information provided by these two cues. In search for such a unified representation, we conducted a magnetoencephalography (MEG) experiment that took advantage of the location-specific adaptation of the auditory cortical N1 response. In general, the attenuation caused by a preceding adaptor sound to the response elicited by a probe depends on their spatial arrangement: if the two sounds coincide, adaptation is stronger than when the locations differ. Here, we presented adaptor–probe pairs that contained different localization cues, for instance, adaptors with ITD and probes with ILD. We found that the adaptation of the N1 amplitude was location-specific across localization cues. This result can be explained by the existence of auditory cortical neurons that are sensitive to sound source location independent on which cue, ITD or ILD, provides the location information. Such neurons would form a cue-independent, unified representation of auditory space in human auditory cortex.

Responses of neurons in the marmoset primary auditory cortex to interaural level differences: comparison of pure tones and vocalizations.

Interaural level differences (ILDs) are the dominant cue for localizing the sources of high frequency sounds that differ in azimuth. Neurons in the primary auditory cortex (A1) respond differentially to ILDs of simple stimuli such as tones and noise bands, but the extent to which this applies to complex natural sounds, such as vocalizations, is not known. In sufentanil/N2O anesthetized marmosets, we compared the responses of 76 A1 neurons to three vocalizations (Ock, Tsik, and Twitter) and pure tones at cells' characteristic frequency. Each stimulus was presented with ILDs ranging from 20 dB favoring the contralateral ear to 20 dB favoring the ipsilateral ear to cover most of the frontal azimuthal space. The response to each stimulus was tested at three average binaural levels (ABLs). Most neurons were sensitive to ILDs of vocalizations and pure tones. For all stimuli, the majority of cells had monotonic ILD sensitivity functions favoring the contralateral ear, but we also observed ILD sensitivity functions that peaked near the midline and functions favoring the ipsilateral ear. Representation of ILD in A1 was better for pure tones and the Ock vocalization in comparison to the Tsik and Twitter calls; this was reflected by higher discrimination indices and greater modulation ranges. ILD sensitivity was heavily dependent on ABL: changes in ABL by ±20 dB SPL from the optimal level for ILD sensitivity led to significant decreases in ILD sensitivity for all stimuli, although ILD sensitivity to pure tones and Ock calls was most robust to such ABL changes. Our results demonstrate differences in ILD coding for pure tones and vocalizations, showing that ILD sensitivity in A1 to complex sounds cannot be simply extrapolated from that to pure tones. They also show A1 neurons do not show level-invariant representation of ILD, suggesting that such a representation of auditory space is likely to require population coding, and further processing at subsequent hierarchical stages.

Crossmodal plasticity and hearing capabilities following blindness.

Valuable insights into the role of experience in shaping perception can be obtained by studying the effects of blindness or other forms of sensory deprivation on the intact senses. Blind individuals are particularly dependent on their hearing and there is extensive evidence that they can develop superior auditory skills, either as a result of plasticity within the auditory system or through the recruitment of functionally relevant occipital cortical areas that lack their normal visual inputs. Because spatial processing normally relies on close interactions between vision and hearing, much of the research in this area has focused on the effects of blindness on auditory localization. Although enhanced auditory skills have been reported in many studies, some aspects of spatial hearing are impaired in the absence of vision. In this case, the effects of crossmodal plasticity may reflect a balance between adaptive changes that compensate for blindness and the role vision normally plays, particularly during development, in calibrating the brain’s representation of auditory space.

Neural representation of probabilities for Bayesian inference.

Bayesian models are often successful in describing perception and behavior, but the neural representation of probabilities remains in question. There are several distinct proposals for the neural representation of probabilities, but they have not been directly compared in an example system. Here we consider three models: a non-uniform population code where the stimulus-driven activity and distribution of preferred stimuli in the population represent a likelihood function and a prior, respectively; the sampling hypothesis which proposes that the stimulus-driven activity over time represents a posterior probability and that the spontaneous activity represents a prior; and the class of models which propose that a population of neurons represents a posterior probability in a distributed code. It has been shown that the non-uniform population code model matches the representation of auditory space generated in the owl's external nucleus of the inferior colliculus (ICx). However, the alternative models have not been tested, nor have the three models been directly compared in any system. Here we tested the three models in the owl's ICx. We found that spontaneous firing rate and the average stimulus-driven response of these neurons were not consistent with predictions of the sampling hypothesis. We also found that neural activity in ICx under varying levels of sensory noise did not reflect a posterior probability. On the other hand, the responses of ICx neurons were consistent with the non-uniform population code model. We further show that Bayesian inference can be implemented in the non-uniform population code model using one spike per neuron when the population is large and is thus able to support the rapid inference that is necessary for sound localization.

Enhanced auditory spatial localization in blind echolocators

Echolocation is the extraordinary ability to represent the external environment by using reflected sound waves from self-generated auditory pulses. Blind human expert echolocators show extremely precise spatial acuity and high accuracy in determining the shape and motion of objects by using echoes. In the current study, we investigated whether or not the use of echolocation would improve the representation of auditory space, which is severely compromised in congenitally blind individuals (Gori et al., 2014). The performance of three blind expert echolocators was compared to that of 6 blind non-echolocators and 11 sighted participants. Two tasks were performed: (1) a space bisection task in which participants judged whether the second of a sequence of three sounds was closer in space to the first or the third sound and (2) a minimum audible angle task in which participants reported which of two sounds presented successively was located more to the right. The blind non-echolocating group showed a severe impairment only in the space bisection task compared to the sighted group. Remarkably, the three blind expert echolocators performed both spatial tasks with similar or even better precision and accuracy than the sighted group. These results suggest that echolocation may improve the general sense of auditory space, most likely through a process of sensory calibration.

Exposure to a novel stimulus environment alters patterns of lateralization in avian auditory cortex

Perceptual filters formed early in development provide an initial means of parsing the incoming auditory stream. However, these filters may not remain fixed, and may be updated by subsequent auditory input, such that, even in an adult organism, the auditory system undergoes plastic changes to achieve a more efficient representation of the recent auditory environment. Songbirds are an excellent model system for experimental studies of auditory phenomena due to many parallels between song learning in birds and language acquisition in humans. In the present study, we explored the effects of passive immersion in a novel heterospecific auditory environment on neural responses in caudo-medial neostriatum (NCM), a songbird auditory area similar to the secondary auditory cortex in mammals. In zebra finches, a well-studied species of songbirds, NCM responds selectively to conspecific songs and contains a neuronal memory for tutor and other familiar conspecific songs. Adult male zebra finches were randomly assigned to either a conspecific or heterospecific auditory environment. After 2, 4 or 9days of exposure, subjects were presented with heterospecific and conspecific songs during awake electrophysiological recording. The neural response strength and rate of adaptation to the testing stimuli were recorded bilaterally. Controls exposed to conspecific environment sounds exhibited the normal pattern of hemispheric lateralization with higher absolute response strength and faster adaptation in the right hemisphere. The pattern of lateralization was fully reversed in birds exposed to heterospecific environment for 4 or 9days and partially reversed in birds exposed to heterospecific environment for 2days. Our results show that brief passive exposure to a novel category of sounds was sufficient to induce a gradual reorganization of the left and right secondary auditory cortices. These changes may reflect modification of perceptual filters to form a more efficient representation of auditory space.

Frequency-dependent auditory space representation in the human planum temporale.

Functional magnetic resonance imaging (fMRI) findings suggest that a part of the planum temporale (PT) is involved in representing spatial properties of acoustic information. Here, we tested whether this representation of space is frequency-dependent or generalizes across spectral content, as required from high order sensory representations. Using sounds with two different spectral content and two spatial locations in individually tailored virtual acoustic environment, we compared three conditions in a sparse-fMRI experiment: Single Location, in which two sounds were both presented from one location; Fixed Mapping, in which there was one-to-one mapping between two sounds and two locations; and Mixed Mapping, in which the two sounds were equally likely to appear at either one of the two locations. We surmised that only neurons tuned to both location and frequency should be differentially adapted by the Mixed and Fixed mappings. Replicating our previous findings, we found adaptation to spatial location in the PT. Importantly, activation was higher for Mixed Mapping than for Fixed Mapping blocks, even though the two sounds and the two locations appeared equally in both conditions. These results show that spatially tuned neurons in the human PT are not invariant to the spectral content of sounds.

Cognitive representation of auditory space in blind football experts

ObjectivesWe compared the mental representation of sound directions in blind football players, blind non-athletes and sighted individuals. DesignStanding blindfolded in the middle of a circle with 16 loudspeakers, participants judged whether the directions of two subsequently presented sounds were similar or not. MethodStructure dimensional analysis (SDA) was applied to reveal mean cluster solutions for the groups. ResultsHierarchical cluster analysis via SDA resulted in distinct representation structures of sound directions. The blind football players' mean cluster solution consisted of pairs of neighboring directions. The blind non-athletes also clustered the directions in pairs, but included non-adjacent directions. In the sighted participants' structure, frontal directions were clustered pairwise, the absolute back was singled out, and the side regions accounted for more directions. ConclusionsOur results suggest that the mental representation of egocentric auditory space is influenced by sight and by the level of expertise in auditory-based orientation and navigation.

Accommodating to new ears: the effects of sensory and sensory-motor feedback.

Changing the shape of the outer ear using small in-ear molds degrades sound localization performance consistent with the distortion of monaural spectral cues to location. It has been shown recently that adult listeners re-calibrate to these new spectral cues for locations both inside and outside the visual field. This raises the question as to the teacher signal for this remarkable functional plasticity. Furthermore, large individual differences in the extent and rate of accommodation suggests a number of factors may be driving this process. A training paradigm exploiting multi-modal and sensory-motor feedback during accommodation was examined to determine whether it might accelerate this process. So as to standardize the modification of the spectral cues, molds filling 40% of the volume of each outer ear were custom made for each subject. Daily training sessions for about an hour, involving repetitive auditory stimuli and exploratory behavior by the subject, significantly improved the extent of accommodation measured by both front-back confusions and polar angle localization errors, with some improvement in the rate of accommodation demonstrated by front-back confusion errors. This work has implications for both the process by which a coherent representation of auditory space is maintained and for accommodative training for hearing aid wearers.

Efficient coding of spectrotemporal binaural sounds leads to emergence of the auditory space representation

To date a number of studies have shown that receptive field shapes of early sensory neurons can be reproduced by optimizing coding efficiency of natural stimulus ensembles. A still unresolved question is whether the efficient coding hypothesis explains formation of neurons which explicitly represent environmental features of different functional importance. This paper proposes that the spatial selectivity of higher auditory neurons emerges as a direct consequence of learning efficient codes for natural binaural sounds. Firstly, it is demonstrated that a linear efficient coding transform—Independent Component Analysis (ICA) trained on spectrograms of naturalistic simulated binaural sounds extracts spatial information present in the signal. A simple hierarchical ICA extension allowing for decoding of sound position is proposed. Furthermore, it is shown that units revealing spatial selectivity can be learned from a binaural recording of a natural auditory scene. In both cases a relatively small subpopulation of learned spectrogram features suffices to perform accurate sound localization. Representation of the auditory space is therefore learned in a purely unsupervised way by maximizing the coding efficiency and without any task-specific constraints. This results imply that efficient coding is a useful strategy for learning structures which allow for making behaviorally vital inferences about the environment.

Compression of perceived auditory space during forward self-motion

Humans can perceive a stable auditory environment and appropriately react to a sound source, even when they are moving. This suggests that the inputs are reinterpreted in the brain, while being integrated with information on the movements. Although several studies have shown the influence of the vestibular semicircular canal signals on auditory localization, it is not clear how auditory space representation is modulated by linear accelerations which are obtained from the macular receptors of the otolith system (utricle and saccule). We investigated the effect of the linear acceleration on auditory space representation. During the forward/backward self-motion, a short noise burst was presented from one of the loudspeakers which were aligned parallel to the motion direction when the listener's coronal plane reached the location of one of the speakers (null point). The results showed that the sound position aligned with the subjective coronal plane was displaced ahead of the null point only during forward self-motion. Moreover, all the sounds that were actually located in the traveling direction were perceived as being biased towards the null point. These results suggest a distortion of perceived auditory space in the direction of movement during forward self-motion.

Visual task enhances spatial selectivity in the human auditory cortex

The auditory cortex represents spatial locations differently from other sensory modalities. While visual and tactile cortices utilize topographical space maps, for audition no such cortical map has been found. Instead, auditory cortical neurons have wide spatial receptive fields and together they form a population rate code of sound source location. Recent studies have shown that this code is modulated by task conditions so that during auditory tasks it provides better selectivity to sound source location than during idle listening. The goal of this study was to establish whether the neural representation of auditory space can also be influenced by task conditions involving other sensory modalities than hearing. Therefore, we conducted magnetoencephalography (MEG) recordings in which auditory spatial selectivity of the human cortex was probed with an adaptation paradigm while subjects performed a visual task. Engaging in the task led to an increase in neural selectivity to sound source location compared to when no task was performed. This suggests that an enhancement in the population rate code of auditory space took place during task performance. This enhancement in auditory spatial selectivity was independent of the direction of visual orientation. Together with previous studies, these findings suggest that performing any demanding task, even one in which sounds and their source locations are irrelevant, can lead to enhancements in the neural representation of auditory space. Such mechanisms may have great survival value as sounds are capable of producing location information on potentially relevant events in all directions and over long distances.