Auditory Motion Research Articles

Representation of Auditory Motion Directions and Sound Source Locations in the Human Planum Temporale.

The ability to compute the location and direction of sounds is a crucial perceptual skill to efficiently interact with dynamic environments. How the human brain implements spatial hearing is, however, poorly understood. In our study, we used fMRI to characterize the brain activity of male and female humans listening to sounds moving left, right, up, and down as well as static sounds. Whole-brain univariate results contrasting moving and static sounds varying in their location revealed a robust functional preference for auditory motion in bilateral human planum temporale (hPT). Using independently localized hPT, we show that this region contains information about auditory motion directions and, to a lesser extent, sound source locations. Moreover, hPT showed an axis of motion organization reminiscent of the functional organization of the middle-temporal cortex (hMT+/V5) for vision. Importantly, whereas motion direction and location rely on partially shared pattern geometries in hPT, as demonstrated by successful cross-condition decoding, the responses elicited by static and moving sounds were, however, significantly distinct. Altogether, our results demonstrate that the hPT codes for auditory motion and location but that the underlying neural computation linked to motion processing is more reliable and partially distinct from the one supporting sound source location.SIGNIFICANCE STATEMENT Compared with what we know about visual motion, little is known about how the brain implements spatial hearing. Our study reveals that motion directions and sound source locations can be reliably decoded in the human planum temporale (hPT) and that they rely on partially shared pattern geometries. Our study, therefore, sheds important new light on how computing the location or direction of sounds is implemented in the human auditory cortex by showing that those two computations rely on partially shared neural codes. Furthermore, our results show that the neural representation of moving sounds in hPT follows a "preferred axis of motion" organization, reminiscent of the coding mechanisms typically observed in the occipital middle-temporal cortex (hMT+/V5) region for computing visual motion.

Neural binaural sensitivity at high sound speeds: Single cell responses in cat midbrain to fast-changing interaural time differences of broadband sounds.

Relative motion between the body and the outside world is a rich source of information. Neural selectivity to motion is well-established in several sensory systems, but is controversial in hearing. This study examines neural sensitivity to changes in the instantaneous interaural time difference of sounds at the two ears. Midbrain neurons track such changes up to extremely high speeds, show only a coarse dependence of firing rate on speed, and lack directional selectivity. These results argue against the presence of selectivity to auditory motion at the level of the midbrain, but reveal an acuity which enables coding of fast-fluctuating binaural cues in realistic sound environments.

Sex difference in facilitation effects of auditory motion stimuli on vection

Sex difference in facilitation effects of auditory motion stimuli on vection

Auditory capture of visual apparent motion, both laterally and looming

Traditional tests of multisensory stimuli typically support that vision dominates spatial judgments and audition dominates temporal ones. Here, we examine if unambiguous auditory spatial cues can capture ambiguous visual ones in judgments of direction of apparent motion. The visual motion judgments include both lateral movement and movement in depth, each when coupled with auditory stimuli moving at one of four rates. Experiment 1 tested lateral visual movement judgments (leftward vs rightward) coupled with auditory stimuli that moved laterally. Experiment 2 tested depth visual movement judgments (approaching vs receding) coupled with auditory stimuli that got louder or quieter. Results of Experiment 1 revealed and replicated an overall leftward motion bias, but with additional acoustic capture to experience visual movement away from the side on which sound initially occurred, and no effect of auditory motion speed. Results of Experiment 2 revealed and replicated an approaching motion bias, but with no effect of initial sound intensity, and an additional systematic capture effect of auditory motion speed. Faster changes in acoustic intensity produced larger visual motion capture consistent with the direction of acoustic intensity change. Findings of both experiments generalized over conditions of listening device (head phones vs speakers) and test-setting (Laboratory vs Web-based data-collection). The leftward and approaching motion bias results replicate previous research. Our principal new findings, the auditory motion capture effects, confirm the multisensory nature of dynamic spatial perception and support that extent of inter-sensory capture is a function of the relative reliability of spatial information acquired by each sensory modality.

Doppler Illusion Prevails Over Pratt Effect in Risset Tones.

Changes in frequency such as those found in Risset tones have been associated with moving sound sources in the vertical plane (Pratt effect) and the horizontal plane (Doppler illusion). We investigated the reported origin and motion of unspatialized Risset tones presented monotically and diotically, and Risset tones simulated to be in the sagittal or coronal plane, approaching or receding, from above or horizontally. Independent of the artificial spatialization used (none, spatializing frequency components collectively or individually, elevated or not), upward glissandi were more likely to be judged as approaching than receding, and downward glissandi as receding than approaching, in most cases from the horizon. Glissandi associations with horizontal movements were more common in stimuli simulated on the sagittal plane than in stimuli simulated on the coronal plane. These findings suggest that the Doppler illusion is stronger than the Pratt effect, at least for Risset tones presented over headphones and simulated to be in the sagittal plane. These findings may contribute to better understanding of the association between auditory motion perception and changes in frequency.

Auditory and Visual Motion Processing and Integration in the Primate Cerebral Cortex.

The ability of animals to detect motion is critical for survival, and errors or even delays in motion perception may prove costly. In the natural world, moving objects in the visual field often produce concurrent sounds. Thus, it can highly advantageous to detect motion elicited from sensory signals of either modality, and to integrate them to produce more reliable motion perception. A great deal of progress has been made in understanding how visual motion perception is governed by the activity of single neurons in the primate cerebral cortex, but far less progress has been made in understanding both auditory motion and audiovisual motion integration. Here we, review the key cortical regions for motion processing, focussing on translational motion. We compare the representations of space and motion in the visual and auditory systems, and examine how single neurons in these two sensory systems encode the direction of motion. We also discuss the way in which humans integrate of audio and visual motion cues, and the regions of the cortex that may mediate this process.

Spectrotemporal window of binaural integration in auditory object formation

Binaural integration of interaural temporal information is essential for sound source localization and segregation. Current models of binaural interaction have shown that accurate sound localization in the horizontal plane depends on the resolution of phase ambiguous information by across-frequency integration. However, as such models are mostly static, it is not clear how proximate in time binaural events in different frequency channels should occur to form an auditory object with a unique lateral position. The present study examined the spectrotemporal window required for effective integration of binaural cues across frequency to form the perception of a stationary position. In Experiment 1, listeners judged whether dichotic frequency-modulated (FM) sweeps with a constant large nominal interaural delay (1500 μs), whose perceived laterality was ambiguous depending on the sweep rate (1500, 3000, 6000, and 12,000 Hz/s), produced a percept of continuous motion or a stationary image. Motion detection performance, indexed by d-prime (d') values, showed a clear effect of sweep rate, with auditory motion effects most pronounced for low sweep rates, and a punctate stationary image at high rates. Experiment 2 examined the effect of modulation rate (0.5, 3, 20, and 50 Hz) on lateralizing sinusoidally frequency-modulated (SFM) tones to confirm the effect of sweep rate on motion detection, independent of signal duration. Lateralization accuracy increased with increasing modulation rate up to 20 Hz and saturated at 50 Hz, with poorest performance occurring below 3 Hz depending on modulator phase. Using the transition point where percepts changed from motion to stationary images, we estimated a spectrotemporal integration window of approximately 150 ms per octave required for effective integration of interaural temporal cues across frequency channels. A Monte Carlo simulation based on a cross-correlation model of binaural interaction predicted 90% of the variance on perceptual motion detection performance as a function of FM sweep rate. Findings suggest that the rate of frequency channel convergence of binaural cues is essential to binaural lateralization.

Decoding auditory motion direction and location in hMT+/V5 and Planum Temporale of sighted and blind individuals

The research presented in this thesis addresses the neural mechanisms of auditory motion processing and the impact of early visual deprivation on motion-responsive brain regions, by using functional magnetic resonance imaging. Visual motion, and in particular direction selectivity, is one of the most investigated aspects of mammalian brain function. In comparison, little is known about how the brain processes moving sounds. More precisely, we have a poor understanding of how the human brain codes for the direction of auditory motion and how this process differs from auditory sound-source localization. In the first study, we characterized the neural representations of auditory motion within the Planum Temporale (PT), and how motion direction and sound source location are represented within this auditory motion responsive region. We further explore if the distribution of orientation responsive neurons (topographic representations) within the PT shares similar organizational features to what is observed within the visual motion area MT/V5. The spatial representations would, therefore, be more systematic for axis of motion/space, rather than for within-axis direction/location. Despite the shared representations between auditory spatial conditions, we show that motion directions and sound source locations generate highly distinct patterns of activity. The second study focused on the impact of early visual deprivation on auditory motion processing. Studying visual deprivation-induced plasticity sheds light on how sensory experience alters the functional organization of motion processing areas, and exploits intrinsic computational bias implemented in cortical regions. In addition to enhanced auditory motion responses within the hMT+/V5, we demonstrate that this region maintains direction selectivity tuning, but enhances its modality preference to auditory input in case of early blindness. Crucially, the enhanced computational role of hMT+/V5 is followed by a reduced role of PT for processing both motion direction and sound source location. These results suggest that early blindness triggers interplay between visual and auditory motion areas, and their computational roles could be re-distributed for effective processing of auditory spatial tasks. Overall, our findings suggest (1) auditory motion-specific processing in the typically developed auditory cortex, and (2) interplay between cross- and intra-modal plasticity to compute auditory motion and space in early blind individuals.

A common mechanism processes auditory and visual motion

A common mechanism processes auditory and visual motion

Multi-modal representation of visual and auditory motion directions in hMT+/V5

Multi-modal representation of visual and auditory motion directions in hMT+/V5

Auditory motion does not modulate spiking activity in the middle temporal and medial superior temporal visual areas.

The integration of multiple sensory modalities is a key aspect of brain function, allowing animals to take advantage of concurrent sources of information to make more accurate perceptual judgments. For many years, multisensory integration in the cerebral cortex was deemed to occur only in high-level "polysensory" association areas. However, more recent studies have suggested that cross-modal stimulation can also influence neural activity in areas traditionally considered to be unimodal. In particular, several human neuroimaging studies have reported that extrastriate areas involved in visual motion perception are also activated by auditory motion, and may integrate audiovisual motion cues. However, the exact nature and extent of the effects of auditory motion on the visual cortex have not been studied at the single neuron level. We recorded the spiking activity of neurons in the middle temporal (MT) and medial superior temporal (MST) areas of anesthetized marmoset monkeys upon presentation of unimodal stimuli (moving auditory or visual patterns), as well as bimodal stimuli (concurrent audiovisual motion). Despite robust, direction selective responses to visual motion, none of the sampled neurons responded to auditory motion stimuli. Moreover, concurrent moving auditory stimuli had no significant effect on the ability of single MT and MST neurons, or populations of simultaneously recorded neurons, to discriminate the direction of motion of visual stimuli (moving random dot patterns with varying levels of motion noise). Our findings do not support the hypothesis that direct interactions between MT, MST and areas low in the hierarchy of auditory areas underlie audiovisual motion integration.

Neural tracking of auditory motion is reflected by delta phase and alpha power of EEG

It is of increasing practical interest to be able to decode the spatial characteristics of an auditory scene from electrophysiological signals. However, the cortical representation of auditory space is not well characterized, and it is unclear how cortical activity reflects the time-varying location of a moving sound. Recently, we demonstrated that cortical response measures to discrete noise bursts can be decoded to determine their origin in space. Here we build on these findings to investigate the cortical representation of a continuously moving auditory stimulus using scalp recorded electroencephalography (EEG).In a first experiment, subjects listened to pink noise over headphones which was spectro-temporally modified to be perceived as randomly moving on a semi-circular trajectory in the horizontal plane. While subjects listened to the stimuli, we recorded their EEG using a 128-channel acquisition system. The data were analysed by 1) building a linear regression model (decoder) mapping the relationship between the stimulus location and a training set of EEG data, and 2) using the decoder to reconstruct an estimate of the time-varying sound source azimuth from the EEG data. The results showed that we can decode sound trajectory with a reconstruction accuracy significantly above chance level. Specifically, we found that the phase of delta (<2 Hz) and power of alpha (8–12 Hz) EEG track the dynamics of a moving auditory object.In a follow-up experiment, we replaced the noise with pulse train stimuli containing only interaural level and time differences (ILDs and ITDs respectively). This allowed us to investigate whether our trajectory decoding is sensitive to both acoustic cues. We found that the sound trajectory can be decoded for both ILD and ITD stimuli. Moreover, their neural signatures were similar and even allowed successful cross-cue classification. This supports the notion of integrated processing of ILD and ITD at the cortical level. These results are particularly relevant for application in devices such as cognitively controlled hearing aids and for the evaluation of virtual acoustic environments.

Restoration of spatial hearing in adult cochlear implant users with single-sided deafness

In recent years, cochlear implants (CIs) have been provided in growing numbers to people with not only bilateral deafness but also to people with unilateral hearing loss, at times in order to alleviate tinnitus. This study presents audiological data from 15 adult participants (ages 48 ± 12 years) with single sided deafness. Results are presented from 9/15 adults, who received a CI (SSD-CI) in the deaf ear and were tested in Acoustic or Acoustic + CI hearing modes, and 6/15 adults who are planning to receive a CI, and were tested in the unilateral condition only. Testing included (1) audiometric measures of threshold, (2) speech understanding for CNC words and AzBIO sentences, (3) tinnitus handicap inventory, (4) sound localization with stationary sound sources, and (5) perceived auditory motion. Results showed that when listening to sentences in quiet, performance was excellent in the Acoustic and Acoustic + CI conditions. In noise, performance was similar between Acoustic and Acoustic + CI conditions in 4/6 participants tested, and slightly worse in the Acoustic + CI in 2/6 participants. In some cases, the CI provided reduced tinnitus handicap scores. When testing sound localization ability, the Acoustic + CI condition resulted in improved sound localization RMS error of 29.2° (SD: ±6.7°) compared to 56.6° (SD: ±16.5°) in the Acoustic-only condition. Preliminary results suggest that the perception of motion direction, whereby subjects are required to process and compare directional cues across multiple locations, is impaired when compared with that of normal hearing subjects.

Psychophysical evidence for auditory motion parallax

Distance is important: From an ecological perspective, knowledge about the distance to either prey or predator is vital. However, the distance of an unknown sound source is particularly difficult to assess, especially in anechoic environments. In vision, changes in perspective resulting from observer motion produce a reliable, consistent, and unambiguous impression of depth known as motion parallax. Here we demonstrate with formal psychophysics that humans can exploit auditory motion parallax, i.e., the change in the dynamic binaural cues elicited by self-motion, to assess the relative depths of two sound sources. Our data show that sensitivity to relative depth is best when subjects move actively; performance deteriorates when subjects are moved by a motion platform or when the sound sources themselves move. This is true even though the dynamic binaural cues elicited by these three types of motion are identical. Our data demonstrate a perceptual strategy to segregate intermittent sound sources in depth and highlight the tight interaction between self-motion and binaural processing that allows assessment of the spatial layout of complex acoustic scenes.

Motion processing after sight restoration: No competition between visual recovery and auditory compensation

The present study tested whether or not functional adaptations following congenital blindness are maintained in humans after sight-restoration and whether they interfere with visual recovery. In permanently congenital blind individuals both intramodal plasticity (e.g. changes in auditory cortex) as well as crossmodal plasticity (e.g. an activation of visual cortex by auditory stimuli) have been observed. Both phenomena were hypothesized to contribute to improved auditory functions. For example, it has been shown that early permanently blind individuals outperform sighted controls in auditory motion processing and that auditory motion stimuli elicit activity in typical visual motion areas. Yet it is unknown what happens to these behavioral adaptations and cortical reorganizations when sight is restored, that is, whether compensatory auditory changes are lost and to which degree visual motion processing is reinstalled. Here we employed a combined behavioral-electrophysiological approach in a group of sight-recovery individuals with a history of a transient phase of congenital blindness lasting for several months to several years. They, as well as two control groups, one with visual impairments, one normally sighted, were tested in a visual and an auditory motion discrimination experiment. Task difficulty was manipulated by varying the visual motion coherence and the signal to noise ratio, respectively. The congenital cataract-reversal individuals showed lower performance in the visual global motion task than both control groups. At the same time, they outperformed both control groups in auditory motion processing suggesting that at least some compensatory behavioral adaptation as a consequence of a complete blindness from birth was maintained. Alpha oscillatory activity during the visual task was significantly lower in congenital cataract reversal individuals and they did not show ERPs modulated by visual motion coherence as observed in both control groups. In contrast, beta oscillatory activity in the auditory task, which varied as a function of SNR in all groups, was overall enhanced in congenital cataract reversal individuals. These results suggest that intramodal plasticity elicited by a transient phase of blindness was maintained and might mediate the prevailing auditory processing advantages in congenital cataract reversal individuals. By contrast, auditory and visual motion processing do not seem to compete for the same neural resources. We speculate that incomplete visual recovery is due to impaired neural network turning which seems to depend on early visual input. The present results demonstrate a privilege of the first arriving input for shaping neural circuits mediating both auditory and visual functions.

Modulation frequency as a cue for auditory speed perception.

Unlike vision, the mechanisms underlying auditory motion perception are poorly understood. Here we describe an auditory motion illusion revealing a novel cue to auditory speed perception: the temporal frequency of amplitude modulation (AM-frequency), typical for rattling sounds. Naturally, corrugated objects sliding across each other generate rattling sounds whose AM-frequency tends to directly correlate with speed. We found that AM-frequency modulates auditory speed perception in a highly systematic fashion: moving sounds with higher AM-frequency are perceived as moving faster than sounds with lower AM-frequency. Even more interestingly, sounds with higher AM-frequency also induce stronger motion aftereffects. This reveals the existence of specialized neural mechanisms for auditory motion perception, which are sensitive to AM-frequency. Thus, in spatial hearing, the brain successfully capitalizes on the AM-frequency of rattling sounds to estimate the speed of moving objects. This tightly parallels previous findings in motion vision, where spatio-temporal frequency of moving displays systematically affects both speed perception and the magnitude of the motion aftereffects. Such an analogy with vision suggests that motion detection may rely on canonical computations, with similar neural mechanisms shared across the different modalities.

Auditory motion-specific mechanisms in the primate brain.

This work examined the mechanisms underlying auditory motion processing in the auditory cortex of awake monkeys using functional magnetic resonance imaging (fMRI). We tested to what extent auditory motion analysis can be explained by the linear combination of static spatial mechanisms, spectrotemporal processes, and their interaction. We found that the posterior auditory cortex, including A1 and the surrounding caudal belt and parabelt, is involved in auditory motion analysis. Static spatial and spectrotemporal processes were able to fully explain motion-induced activation in most parts of the auditory cortex, including A1, but not in circumscribed regions of the posterior belt and parabelt cortex. We show that in these regions motion-specific processes contribute to the activation, providing the first demonstration that auditory motion is not simply deduced from changes in static spatial location. These results demonstrate that parallel mechanisms for motion and static spatial analysis coexist within the auditory dorsal stream.

An Adaptive Neural Mechanism for Acoustic Motion Perception with Varying Sparsity.

Biological motion-sensitive neural circuits are quite adept in perceiving the relative motion of a relevant stimulus. Motion perception is a fundamental ability in neural sensory processing and crucial in target tracking tasks. Tracking a stimulus entails the ability to perceive its motion, i.e., extracting information about its direction and velocity. Here we focus on auditory motion perception of sound stimuli, which is poorly understood as compared to its visual counterpart. In earlier work we have developed a bio-inspired neural learning mechanism for acoustic motion perception. The mechanism extracts directional information via a model of the peripheral auditory system of lizards. The mechanism uses only this directional information obtained via specific motor behaviour to learn the angular velocity of unoccluded sound stimuli in motion. In nature however the stimulus being tracked may be occluded by artefacts in the environment, such as an escaping prey momentarily disappearing behind a cover of trees. This article extends the earlier work by presenting a comparative investigation of auditory motion perception for unoccluded and occluded tonal sound stimuli with a frequency of 2.2 kHz in both simulation and practice. Three instances of each stimulus are employed, differing in their movement velocities–0.5°/time step, 1.0°/time step and 1.5°/time step. To validate the approach in practice, we implement the proposed neural mechanism on a wheeled mobile robot and evaluate its performance in auditory tracking.

Audio–visual interaction in visual motion detection: Synchrony versus Asynchrony

ObjectiveDetection and identification of moving targets is of paramount importance in everyday life, even if it is not widely tested in optometric practice, mostly for technical reasons. There are clear indications in the literature that in perception of moving targets, vision and hearing interact, for example in noisy surrounds and in understanding speech. The main aim of visual perception, the ability that optometry aims to optimize, is the identification of objects, from everyday objects to letters, but also the spatial orientation of subjects in natural surrounds. To subserve this aim, corresponding visual and acoustic features from the rich spectrum of signals supplied by natural environments have to be combined. MethodsHere, we investigated the influence of an auditory motion stimulus on visual motion detection, both with a concrete (left/right movement) and an abstract auditory motion (increase/decrease of pitch). ResultsWe found that incongruent audiovisual stimuli led to significantly inferior detection compared to the visual only condition. Additionally, detection was significantly better in abstract congruent than incongruent trials. For the concrete stimuli the detection threshold was significantly better in asynchronous audiovisual conditions than in the unimodal visual condition. ConclusionWe find a clear but complex pattern of partly synergistic and partly inhibitory audio–visual interactions. It seems that asynchrony plays only a positive role in audiovisual motion while incongruence mostly disturbs in simultaneous abstract configurations but not in concrete configurations. As in speech perception in hearing-impaired patients, patients suffering from visual deficits should be able to benefit from acoustic information.

Auditory compensation for head rotation is incomplete.

Hearing is confronted by a similar problem to vision when the observer moves. The image motion that is created remains ambiguous until the observer knows the velocity of eye and/or head. One way the visual system solves this problem is to use motor commands, proprioception, and vestibular information. These “extraretinal signals” compensate for self-movement, converting image motion into head-centered coordinates, although not always perfectly. We investigated whether the auditory system also transforms coordinates by examining the degree of compensation for head rotation when judging a moving sound. Real-time recordings of head motion were used to change the “movement gain” relating head movement to source movement across a loudspeaker array. We then determined psychophysically the gain that corresponded to a perceptually stationary source. Experiment 1 showed that the gain was small and positive for a wide range of trained head speeds. Hence, listeners perceived a stationary source as moving slightly opposite to the head rotation, in much the same way that observers see stationary visual objects move against a smooth pursuit eye movement. Experiment 2 showed the degree of compensation remained the same for sounds presented at different azimuths, although the precision of performance declined when the sound was eccentric. We discuss two possible explanations for incomplete compensation, one based on differences in the accuracy of signals encoding image motion and self-movement and one concerning statistical optimization that sacrifices accuracy for precision. We then consider the degree to which such explanations can be applied to auditory motion perception in moving listeners.