Auditory Motion Perception Research Articles

The perception of auditory motion in sighted and early blind individuals.

Motion perception is a fundamental sensory task that plays a critical evolutionary role. In vision, motion processing is classically described using a motion energy model with spatiotemporally nonseparable filters suited for capturing the smooth continuous changes in spatial position over time afforded by moving objects. However, it is still not clear whether the filters underlying auditory motion discrimination are also continuous motion detectors or infer motion from comparing discrete sound locations over time (spatiotemporally separable). We used a psychophysical reverse correlation paradigm, where participants discriminated the direction of a motion signal in the presence of spatiotemporal noise, to determine whether the filters underlying auditory motion discrimination were spatiotemporally separable or nonseparable. We then examined whether these auditory motion filters were altered as a result of early blindness. We found that both sighted and early blind individuals have separable filters. However, early blind individuals show increased sensitivity to auditory motion, with reduced susceptibility to noise and filters that were more accurate in detecting motion onsets/offsets. Model simulations suggest that this reliance on separable filters is optimal given the limited spatial resolution of auditory input.

The effect of motion frequency and sound source frequency on steady-state auditory motion evoked potential

The ability of humans to perceive motion sound sources is important for accurate response to the living environment. Periodic motion sound sources can elicit steady-state motion auditory evoked potential (SSMAEP). The purpose of this study was to investigate the effects of different motion frequencies and different frequencies of sound source on SSMAEP. The stimulation paradigms for simulating periodic motion of sound sources were designed utilizing head-related transfer function (HRTF) techniques in this study. The motion frequencies of the paradigm are set respectively to 1-10 Hz, 15 Hz, 20 Hz, 30 Hz, 40 Hz, 60 Hz, and 80 Hz. In addition, the frequencies of sound source of the paradigms were set to 500 Hz, 1000 Hz, 2000 Hz, 3000 Hz, and 4000 Hz at motion frequencies of 6 Hz and 40 Hz. Fourteen subjects with normal hearing were recruited for the study. SSMAEP was elicited by 500 Hz pure tone at motion frequencies of 1-10 Hz, 15 Hz, 20 Hz, 30 Hz, 40 Hz, 60 Hz, and 80 Hz. SSMAEP was strongest at motion frequencies of 6 Hz. Moreover, at 6 Hz motion frequency, the SSMAEP amplitude was largest at the tone frequency of 500 Hz and smallest at 4000 Hz. Whilst SSMAEP elicited by 4000 Hz pure tone was significantly the strongest at motion frequency of 40 Hz. SSMAEP can be elicited by periodic motion sound sources at motion frequencies up to 80 Hz. SSMAEP also has a strong response at lower frequency. Low-frequency pure tones are beneficial to enhance SSMAEP at low-frequency sound source motion, whilst high-frequency pure tones help to enhance SSMAEP at high-frequency sound source motion. The study provides new insight into the brain's perception of rhythmic auditory motion.

Doppler Frequency-Shift Information Processing in WOx -Based Memristive Synapse for Auditory Motion Perception.

Auditory motion perception is one crucial capability to decode and discriminate the spatiotemporal information for neuromorphic auditory systems. Doppler frequency-shift feature and interaural time difference (ITD) are two fundamental cues of auditory information processing. In this work, the functions of azimuth detection and velocity detection, as the typical auditory motion perception, are demonstrated in a WOx -based memristive synapse. The WOx memristor presents both the volatile mode (M1) and semi-nonvolatile mode (M2), which are capable of implementing the high-pass filtering and processing the spike trains with a relative timing and frequency shift. In particular, the Doppler frequency-shift information processing for velocity detection is emulated in the WOx memristor based auditory system for the first time, which relies on a scheme of triplet spike-timing-dependent-plasticity in the memristor. These results provide new opportunities for the mimicry of auditory motion perception and enable the auditory sensory system to be applied in future neuromorphic sensing.

Do you hear what I see? How do early blind individuals experience object motion?

One of the most important tasks for 3D vision is tracking the movement of objects in space. The ability of early blind individuals to understand motion in the environment from noisy and unreliable auditory information is an impressive example of cortical adaptation that is only just beginning to be understood. Here, we compare visual and auditory motion processing, and discuss the effect of early blindness on the perception of auditory motion. Blindness leads to cross-modal recruitment of the visual motion area hMT+ for auditory motion processing. Meanwhile, the planum temporale, associated with auditory motion in sighted individuals, shows reduced selectivity for auditory motion. We discuss how this dramatic shift in the cortical basis of motion processing might influence the perceptual experience of motion in early blind individuals. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Contributed Session II: The mechanisms underlying enhanced auditory motion perception in early blind individuals

Previous studies have shown increased sensitivity to auditory motion in early blind (EB) individuals. However, the mechanisms underlying this enhanced performance remains unknown. We estimated auditory motion filters in EB using a psychophysical reverse correlation paradigm-the auditory analogue of Neri (2014).Participants (8 EB and 8 sighted controls (SC)) discriminated the direction of a signal motion (left/right) embedded in broadband noise bursts (500-14,000 Hz) presented over a 10x10 grid (space: -/+30 °; time: 0-800 ms). Amplitude of signal motion was adjusted using staircases to maintain performance at 65%. We made two independent measurements: tuning of auditory motion filters (how noise bursts influence the probability of hearing motion direction), and amplitude threshold for hearing the signal.The estimated filters for both EB and SC were predominantly responsive to sound onsets/offsets, different from the non-separable spatiotemporal selectivity previously observed for visual motion. Also, the filters for EB were more accurate in detecting signal onsets/offsets across both space and time. This more refined tuning successfully predicted the greater sensitivity to hear signal motion in EB, as measured by the amplitude threshold (t(14) = -5.59, p < .001). Thus, EB individuals show subtle qualitative rather than quantitative differences in auditory motion processing that nonetheless result in significant improvements in perception.

Signal envelope and speech intelligibility differentially impact auditory motion perception

Our acoustic environment contains a plethora of complex sounds that are often in motion. To gauge approaching danger and communicate effectively, listeners need to localize and identify sounds, which includes determining sound motion. This study addresses which acoustic cues impact listeners’ ability to determine sound motion. Signal envelope (ENV) cues are implicated in both sound motion tracking and stimulus intelligibility, suggesting that these processes could be competing for sound processing resources. We created auditory chimaera from speech and noise stimuli and varied the number of frequency bands, effectively manipulating speech intelligibility. Normal-hearing adults were presented with stationary or moving chimaeras and reported perceived sound motion and content. Results show that sensitivity to sound motion is not affected by speech intelligibility, but shows a clear difference for original noise and speech stimuli. Further, acoustic chimaera with speech-like ENVs which had intelligible content induced a strong bias in listeners to report sounds as stationary. Increasing stimulus intelligibility systematically increased that bias and removing intelligible content reduced it, suggesting that sound content may be prioritized over sound motion. These findings suggest that sound motion processing in the auditory system can be biased by acoustic parameters related to speech intelligibility.

The impact of temporal fine structure and signal envelope on auditory motion perception.

The majority of psychoacoustic research investigating sound localization has utilized stationary sources, yet most naturally occurring sounds are in motion, either because the sound source itself moves, or the listener does. In normal hearing (NH) listeners, previous research showed the extent to which sound duration and velocity impact the ability of listeners to detect sound movement. By contrast, little is known about how listeners with hearing impairments perceive moving sounds; the only study to date comparing the performance of NH and bilateral cochlear implant (BiCI) listeners has demonstrated significantly poorer performance on motion detection tasks in BiCI listeners. Cochlear implants, auditory protheses offered to profoundly deaf individuals for access to spoken language, retain the signal envelope (ENV), while discarding temporal fine structure (TFS) of the original acoustic input. As a result, BiCI users do not have access to low-frequency TFS cues, which have previously been shown to be crucial for sound localization in NH listeners. Instead, BiCI listeners seem to rely on ENV cues for sound localization, especially level cues. Given that NH and BiCI listeners differentially utilize ENV and TFS information, the present study aimed to investigate the usefulness of these cues for auditory motion perception. We created acoustic chimaera stimuli, which allowed us to test the relative contributions of ENV and TFS to auditory motion perception. Stimuli were either moving or stationary, presented to NH listeners in free field. The task was to track the perceived sound location. We found that removing low-frequency TFS reduces sensitivity to sound motion, and fluctuating speech envelopes strongly biased the judgment of sounds to be stationary. Our findings yield a possible explanation as to why BiCI users struggle to identify sound motion, and provide a first account of cues important to the functional aspect of auditory motion perception.

Dynamic sound field audiometry: Static and dynamic spatial hearing tests in the full horizontal plane

Although spatial hearing is of great importance in everyday life, today’s routine audiological test batteries and static test setups assess sound localization, discrimination and tracking abilities rudimentarily and thus provide only a limited interpretation of treatment outcomes regarding spatial hearing performance. To address this limitation, we designed a dynamic sound field test setup and evaluated the sound localization, discrimination and tracking performance of 12 normal-hearing subjects. During testing, participants provided feedback either through a touchpad or through eye tracking. In addition, the influence of head movement on sound-tracking performance was investigated. Our results show that tracking and discrimination performance was significantly better in the frontal azimuth than in the rear azimuth. Particularly good performance was observed across the rear midline in localization, discrimination and tracking tests. As expected, free head movement improved sound-tracking abilities. Furthermore, feedback via gaze detection led to larger tracking errors than feedback via the touchpad. We found statistically significant correlations between the static and dynamic tests, which favor the snapshot theory for auditory motion perception.

Auditory motion perception emerges from successive sound localizations integrated over time

Humans rely on auditory information to estimate the path of moving sound sources. But unlike in vision, the existence of motion-sensitive mechanisms in audition is still open to debate. Psychophysical studies indicate that auditory motion perception emerges from successive localization, but existing models fail to predict experimental results. However, these models do not account for any temporal integration. We propose a new model tracking motion using successive localization snapshots but integrated over time. This model is derived from psychophysical experiments on the upper limit for circular auditory motion perception (UL), defined as the speed above which humans no longer identify the direction of sounds spinning around them. Our model predicts ULs measured with different stimuli using solely static localization cues. The temporal integration blurs these localization cues rendering them unreliable at high speeds, which results in the UL. Our findings indicate that auditory motion perception does not require motion-sensitive mechanisms.

Doppler Illusion Prevails Over Pratt Effect in Risset Tones.

Changes in frequency such as those found in Risset tones have been associated with moving sound sources in the vertical plane (Pratt effect) and the horizontal plane (Doppler illusion). We investigated the reported origin and motion of unspatialized Risset tones presented monotically and diotically, and Risset tones simulated to be in the sagittal or coronal plane, approaching or receding, from above or horizontally. Independent of the artificial spatialization used (none, spatializing frequency components collectively or individually, elevated or not), upward glissandi were more likely to be judged as approaching than receding, and downward glissandi as receding than approaching, in most cases from the horizon. Glissandi associations with horizontal movements were more common in stimuli simulated on the sagittal plane than in stimuli simulated on the coronal plane. These findings suggest that the Doppler illusion is stronger than the Pratt effect, at least for Risset tones presented over headphones and simulated to be in the sagittal plane. These findings may contribute to better understanding of the association between auditory motion perception and changes in frequency.

Modulation frequency as a cue for auditory speed perception.

Unlike vision, the mechanisms underlying auditory motion perception are poorly understood. Here we describe an auditory motion illusion revealing a novel cue to auditory speed perception: the temporal frequency of amplitude modulation (AM-frequency), typical for rattling sounds. Naturally, corrugated objects sliding across each other generate rattling sounds whose AM-frequency tends to directly correlate with speed. We found that AM-frequency modulates auditory speed perception in a highly systematic fashion: moving sounds with higher AM-frequency are perceived as moving faster than sounds with lower AM-frequency. Even more interestingly, sounds with higher AM-frequency also induce stronger motion aftereffects. This reveals the existence of specialized neural mechanisms for auditory motion perception, which are sensitive to AM-frequency. Thus, in spatial hearing, the brain successfully capitalizes on the AM-frequency of rattling sounds to estimate the speed of moving objects. This tightly parallels previous findings in motion vision, where spatio-temporal frequency of moving displays systematically affects both speed perception and the magnitude of the motion aftereffects. Such an analogy with vision suggests that motion detection may rely on canonical computations, with similar neural mechanisms shared across the different modalities.

An Adaptive Neural Mechanism for Acoustic Motion Perception with Varying Sparsity.

Biological motion-sensitive neural circuits are quite adept in perceiving the relative motion of a relevant stimulus. Motion perception is a fundamental ability in neural sensory processing and crucial in target tracking tasks. Tracking a stimulus entails the ability to perceive its motion, i.e., extracting information about its direction and velocity. Here we focus on auditory motion perception of sound stimuli, which is poorly understood as compared to its visual counterpart. In earlier work we have developed a bio-inspired neural learning mechanism for acoustic motion perception. The mechanism extracts directional information via a model of the peripheral auditory system of lizards. The mechanism uses only this directional information obtained via specific motor behaviour to learn the angular velocity of unoccluded sound stimuli in motion. In nature however the stimulus being tracked may be occluded by artefacts in the environment, such as an escaping prey momentarily disappearing behind a cover of trees. This article extends the earlier work by presenting a comparative investigation of auditory motion perception for unoccluded and occluded tonal sound stimuli with a frequency of 2.2 kHz in both simulation and practice. Three instances of each stimulus are employed, differing in their movement velocities–0.5°/time step, 1.0°/time step and 1.5°/time step. To validate the approach in practice, we implement the proposed neural mechanism on a wheeled mobile robot and evaluate its performance in auditory tracking.

Auditory compensation for head rotation is incomplete.

Hearing is confronted by a similar problem to vision when the observer moves. The image motion that is created remains ambiguous until the observer knows the velocity of eye and/or head. One way the visual system solves this problem is to use motor commands, proprioception, and vestibular information. These “extraretinal signals” compensate for self-movement, converting image motion into head-centered coordinates, although not always perfectly. We investigated whether the auditory system also transforms coordinates by examining the degree of compensation for head rotation when judging a moving sound. Real-time recordings of head motion were used to change the “movement gain” relating head movement to source movement across a loudspeaker array. We then determined psychophysically the gain that corresponded to a perceptually stationary source. Experiment 1 showed that the gain was small and positive for a wide range of trained head speeds. Hence, listeners perceived a stationary source as moving slightly opposite to the head rotation, in much the same way that observers see stationary visual objects move against a smooth pursuit eye movement. Experiment 2 showed the degree of compensation remained the same for sounds presented at different azimuths, although the precision of performance declined when the sound was eccentric. We discuss two possible explanations for incomplete compensation, one based on differences in the accuracy of signals encoding image motion and self-movement and one concerning statistical optimization that sacrifices accuracy for precision. We then consider the degree to which such explanations can be applied to auditory motion perception in moving listeners.

Auditory Motion Elicits a Visual Motion Aftereffect.

The visual motion aftereffect is a visual illusion in which exposure to continuous motion in one direction leads to a subsequent illusion of visual motion in the opposite direction. Previous findings have been mixed with regard to whether this visual illusion can be induced cross-modally by auditory stimuli. Based on research on multisensory perception demonstrating the profound influence auditory perception can have on the interpretation and perceived motion of visual stimuli, we hypothesized that exposure to auditory stimuli with strong directional motion cues should induce a visual motion aftereffect. Here, we demonstrate that horizontally moving auditory stimuli induced a significant visual motion aftereffect—an effect that was driven primarily by a change in visual motion perception following exposure to leftward moving auditory stimuli. This finding is consistent with the notion that visual and auditory motion perception rely on at least partially overlapping neural substrates.

Sensitivity to Auditory Velocity Contrast.

A natural auditory scene often contains sound moving at varying velocities. Using a velocity contrast paradigm, we compared sensitivity to velocity changes between continuous and discontinuous trajectories. Subjects compared the velocities of two stimulus intervals that moved along a single trajectory, with and without a 1 second inter stimulus interval (ISI). We found thresholds were threefold larger for velocity increases in the instantaneous velocity change condition, as compared to instantaneous velocity decreases or thresholds for the delayed velocity transition condition. This result cannot be explained by the current static “snapshot” model of auditory motion perception and suggest a continuous process where the percept of velocity is influenced by previous history of stimulation.

On the robustness of upper limits for circular auditory motion perception

We developed a new rendering system that consists of a image-source reflection model followed by a two-step strategy for modeling moving phantom sound sources (combining a physical propagation model and a spatialization method). This rendering system was used for simulations of pressure signals and acoustical localization cues, and for stimulus generation. We report three perceptual experiments measuring the upper limits, defined as auditory velocity thresholds beyond which listeners are no longer able to perceptually, resolve a smooth circular trajectory. These thresholds were measured in different reverberation conditions using white noise, band-limited white noise and band-limited white noise with a pure tone. Experiment 1 took place in a hemi-anechoic room and compared two conditions: dry and with simulated first-order reflections. In Experiment 2 we compare the thresholds measured in the hemi-anechoic room (dry and simulated second-order reflections) and in a reverberant room using two different loudspeaker configurations. Experiment 3 investigated the effect of audio source type in the dry condition at high velocities. No significant effects were observed among reverberation conditions, loudspeaker configurations and audio source type, suggesting that the upper limit is robust against reverberation.

The Perception of Auditory Motion.

The growing availability of efficient and relatively inexpensive virtual auditory display technology has provided new research platforms to explore the perception of auditory motion. At the same time, deployment of these technologies in command and control as well as in entertainment roles is generating an increasing need to better understand the complex processes underlying auditory motion perception. This is a particularly challenging processing feat because it involves the rapid deconvolution of the relative change in the locations of sound sources produced by rotational and translations of the head in space (self-motion) to enable the perception of actual source motion. The fact that we perceive our auditory world to be stable despite almost continual movement of the head demonstrates the efficiency and effectiveness of this process. This review examines the acoustical basis of auditory motion perception and a wide range of psychophysical, electrophysiological, and cortical imaging studies that have probed the limits and possible mechanisms underlying this perception.

Smooth pursuit eye and gaze movements to moving auditory targets: Evidence for velocity-specific processing in the auditory system

Auditory motion perception remains poorly understood, in contrast with its well-established visual counterpart. Visual smooth pursuit (SP), a velocity-specific behavior, has been well-quantified during both head-fixed (ocular SP) and head-free (eye + head, or gaze SP) conditions. In contrast, auditory SP tracking has received little attention, despite its potential for demonstrating a motion-specific process in audition. We presented constant-velocity (10–40°/s), free-field auditory (0.2–20 kHz white noise), and visual (LED) targets to head-fixed or head-free subjects while recording ocular and gaze SP responses, respectively. To control for possible SP in the absence of a target, subjects were asked to recreate auditory trajectories after priming with a set of auditory ramps (practiced SP). We found that ocular auditory SP is consistently higher in gain than practiced SP, but variable and lower than visual SP. Further, the gain of auditory gaze SP exceeds ocular auditory SP. Finally, SP of periodic (triangular) motion trajectories revealed that auditory, like visual, SP improves rapidly over time, indicating predictive behavior. In sum, auditory SP closely mimics visual SP but with reduced gain. We propose that auditory motion processing exists, is robust, and recruits a velocity-dependent neural process shared with vision.

Auditory motion processing after early blindness.

Studies showing that occipital cortex responds to auditory and tactile stimuli after early blindness are often interpreted as demonstrating that early blind subjects "see" auditory and tactile stimuli. However, it is not clear whether these occipital responses directly mediate the perception of auditory/tactile stimuli, or simply modulate or augment responses within other sensory areas. We used fMRI pattern classification to categorize the perceived direction of motion for both coherent and ambiguous auditory motion stimuli. In sighted individuals, perceived motion direction was accurately categorized based on neural responses within the planum temporale (PT) and right lateral occipital cortex (LOC). Within early blind individuals, auditory motion decisions for both stimuli were successfully categorized from responses within the human middle temporal complex (hMT+), but not the PT or right LOC. These findings suggest that early blind responses within hMT+ are associated with the perception of auditory motion, and that these responses in hMT+ may usurp some of the functions of nondeprived PT. Thus, our results provide further evidence that blind individuals do indeed "see" auditory motion.

Auditory velocity discrimination in the horizontal plane at very high velocities

We determined velocity discrimination thresholds and Weber fractions for sounds revolving around the listener at very high velocities. Sounds used were a broadband white noise and two harmonic sounds with fundamental frequencies of 330 Hz and 1760 Hz. Experiment 1 used velocities ranging between 288°/s and 720°/s in an acoustically treated room and Experiment 2 used velocities between 288°/s and 576°/s in a highly reverberant hall. A third experiment addressed potential confounds in the first two experiments. The results show that people can reliably discriminate velocity at very high velocities and that both thresholds and Weber fractions decrease as velocity increases. These results violate Weber's law but are consistent with the empirical trend observed in the literature. While thresholds for the noise and 330 Hz harmonic stimulus were similar, those for the 1760 Hz harmonic stimulus were substantially higher. There were no reliable differences in velocity discrimination between the two acoustical environments, suggesting that auditory motion perception at high velocities is robust against the effects of reverberation.