Auditory Capture Research Articles

Sleep deprivation moderates neural processes associated with passive auditory capture

Sleep loss has a major effect on cognitive tasks that are dependent on the maintenance of active sustained attention. This study examines the effects of sleep deprivation on automatic information processing, more specifically, its effect on processes leading to involuntary auditory attention capture by task-irrelevant auditory events. Two experiments were run. In the first, 13 participants were totally sleep-deprived (TSD); in the second, 16 participants were partially sleep-deprived (PSD), sleeping only four hours. Event-related potentials were recorded while participants discriminated the duration of equiprobable short and long auditory tones. At rare times, a small change to the pitch of these stimuli occurred. This deviant was however irrelevant to the duration detection task. As expected, TSD had a significant effect on the attention-dependent duration detection task; performance was worse and the P3b, associated with ease of detection, was attenuated. PSD had a similar, but reduced effect. Critically, the small pitch deviant resulted in less behavioural distraction following TSD compared to normal sleep.Consistent with this, the P3a, associated with the attention capture process, was significantly reduced following both TSD and PSD. Processes related to the passive switching of attention to potentially critical, but unattended, stimulus events are moderated by sleep deprivation.

Auditory capture of visual apparent motion, both laterally and looming

Traditional tests of multisensory stimuli typically support that vision dominates spatial judgments and audition dominates temporal ones. Here, we examine if unambiguous auditory spatial cues can capture ambiguous visual ones in judgments of direction of apparent motion. The visual motion judgments include both lateral movement and movement in depth, each when coupled with auditory stimuli moving at one of four rates. Experiment 1 tested lateral visual movement judgments (leftward vs rightward) coupled with auditory stimuli that moved laterally. Experiment 2 tested depth visual movement judgments (approaching vs receding) coupled with auditory stimuli that got louder or quieter. Results of Experiment 1 revealed and replicated an overall leftward motion bias, but with additional acoustic capture to experience visual movement away from the side on which sound initially occurred, and no effect of auditory motion speed. Results of Experiment 2 revealed and replicated an approaching motion bias, but with no effect of initial sound intensity, and an additional systematic capture effect of auditory motion speed. Faster changes in acoustic intensity produced larger visual motion capture consistent with the direction of acoustic intensity change. Findings of both experiments generalized over conditions of listening device (head phones vs speakers) and test-setting (Laboratory vs Web-based data-collection). The leftward and approaching motion bias results replicate previous research. Our principal new findings, the auditory motion capture effects, confirm the multisensory nature of dynamic spatial perception and support that extent of inter-sensory capture is a function of the relative reliability of spatial information acquired by each sensory modality.

Hearing flashes and seeing beeps: Timing audiovisual events.

Many events from daily life are audiovisual (AV). Handclaps produce both visual and acoustic signals that are transmitted in air and processed by our sensory systems at different speeds, reaching the brain multisensory integration areas at different moments. Signals must somehow be associated in time to correctly perceive synchrony. This project aims at quantifying the mutual temporal attraction between senses and characterizing the different interaction modes depending on the offset. In every trial participants saw four beep-flash pairs regularly spaced in time, followed after a variable delay by a fifth event in the test modality (auditory or visual). A large range of AV offsets was tested. The task was to judge whether the last event came before/after what was expected given the perceived rhythm, while attending only to the test modality. Flashes were perceptually shifted in time toward beeps, the attraction being stronger for lagging than leading beeps. Conversely, beeps were not shifted toward flashes, indicating a nearly total auditory capture. The subjective timing of the visual component resulting from the AV interaction could easily be forward but not backward in time, an intuitive constraint stemming from minimum visual processing delays. Finally, matching auditory and visual time-sensitivity with beeps embedded in pink noise produced very similar mutual attractions of beeps and flashes. Breaking the natural auditory preference for timing allowed vision to take over as well, showing that this preference is not hardwired.

Perception of visual apparent motion is modulated by a gap within concurrent auditory glides, even when it is illusory.

Auditory and visual events often happen concurrently, and how they group together can have a strong effect on what is perceived. We investigated whether/how intra- or cross-modal temporal grouping influenced the perceptual decision of otherwise ambiguous visual apparent motion. To achieve this, we juxtaposed auditory gap transfer illusion with visual Ternus display. The Ternus display involves a multi-element stimulus that can induce either of two different percepts of apparent motion: ‘element motion’ (EM) or ‘group motion’ (GM). In “EM,” the endmost disk is seen as moving back and forth while the middle disk at the central position remains stationary; while in “GM,” both disks appear to move laterally as a whole. The gap transfer illusion refers to the illusory subjective transfer of a short gap (around 100 ms) from the long glide to the short continuous glide when the two glides intercede at the temporal middle point. In our experiments, observers were required to make a perceptual discrimination of Ternus motion in the presence of concurrent auditory glides (with or without a gap inside). Results showed that a gap within a short glide imposed a remarkable effect on separating visual events, and led to a dominant perception of GM as well. The auditory configuration with gap transfer illusion triggered the same auditory capture effect. Further investigations showed that visual interval which coincided with the gap interval (50–230 ms) in the long glide was perceived to be shorter than that within both the short glide and the ‘gap-transfer’ auditory configurations in the same physical intervals (gaps). The results indicated that auditory temporal perceptual grouping takes priority over the cross-modal interaction in determining the final readout of the visual perception, and the mechanism of selective attention on auditory events also plays a role.

Auditory capture of visual stimuli in time is statistically optimal

Auditory capture of visual stimuli in time is statistically optimal

Temporal auditory capture does not affect the time course of saccadic mislocalization of visual stimuli

Irrelevant sounds can "capture" visual stimuli to change their apparent timing, a phenomenon sometimes termed "temporal ventriloquism". Here we ask whether this auditory capture can alter the time course of spatial mislocalization of visual stimuli during saccades. We first show that during saccades, sounds affect the apparent timing of visual flashes, even more strongly than during fixation. However, this capture does not affect the dynamics of perisaccadic visual distortions. Sounds presented 50 ms before or after a visual bar (that change perceived timing of the bars by more than 40 ms) had no measurable effect on the time courses of spatial mislocalization of the bars, in four subjects. Control studies showed that with barely visible, low-contrast stimuli, leading, but not trailing, sounds can have a small effect on mislocalization, most likely attributable to attentional effects rather than auditory capture. These findings support previous studies showing that integration of multisensory information occurs at a relatively late stage of sensory processing, after visual representations have undergone the distortions induced by saccades.

Competition between auditory and visual spatial cues during visual task performance

There is debate in the crossmodal cueing literature as to whether capture of visual attention by means of sound is a fully automatic process. Recent studies show that when visual attention is endogenously focused sound still captures attention. The current study investigated whether there is interaction between exogenous auditory and visual capture. Participants preformed an orthogonal cueing task, in which, the visual target was preceded by both a peripheral visual and auditory cue. When both cues were presented at chance level, visual and auditory capture was observed. However, when the validity of the visual cue was increased to 80% only visual capture and no auditory capture was observed. Furthermore, a highly predictive (80% valid) auditory cue was not able to prevent visual capture. These results demonstrate that crossmodal auditory capture does not occur when a competing predictive visual event is presented and is therefore not a fully automatic process.

Synesthetic congruency modulates the temporal ventriloquism effect

People sometimes find it easier to judge the temporal order in which two visual stimuli have been presented if one tone is presented before the first visual stimulus and a second tone is presented after the second visual stimulus. This enhancement of people's visual temporal sensitivity has been attributed to the temporal ventriloquism of the visual stimuli toward the temporally proximate sounds, resulting in an expansion of the perceived interval between the two visual events. In the present study, we demonstrate that the synesthetic congruency between the auditory and visual stimuli (in particular, between the relative pitch of the sounds and the relative size of the visual stimuli) can modulate the magnitude of this multisensory integration effect: The auditory capture of vision is larger for pairs of auditory and visual stimuli that are synesthetically congruent than for pairs of stimuli that are synesthetically incongruent, as reflected by participants' increased sensitivity in discriminating the temporal order of the visual stimuli. These results provide the first evidence that multisensory temporal integration can be affected by the synesthetic congruency between the auditory and visual stimuli that happen to be presented.

Auditory capture in a spatial cueing task

This study examined how an auditory non‐informative spatial cue influences detection and localization of auditory targets. In a go‐no go task subjects attended a centrally placed loudspeaker surrounded by 6 other loudspeakers and were asked to respond only to centre targets. The cue came from one of the loudspeakers and preceded the target tone with an SOA of 100 or 400 ms. A baseline condition with a spatially diffuse cue was included, using out‐of‐phase presentation through two loudspeakers. Results show that, compared to the baseline, cues shortened reaction times at small cue‐target angles (up to 7°) and increased them at larger angles. At the shortest SOA, the reaction time difference was up to 80 ms. Furthermore, cueing seems to have no effect on sound localization: false alarms were normally distributed around the hits. These findings demonstrate that strong auditory capture effects occur that depend on cue‐target distance.

Auditory capture of vision: examining temporal ventriloquism

Four experiments investigated whether irrelevant sounds can influence the perception of lights in a visual temporal order judgment task, where participants judged which of two lights appeared first. In Experiment 1, presenting a sound before the first light and after the second light improved performance relative to baseline (sounds appearing simultaneously with the lights), as if the sounds pulled the perception of lights further apart in time. Experiment 2 ruled out an alerting explanation for this effect and indicated that the performance improvement resulted from the second sound trailing the second light. Experiment 3 excluded the possibility that leading or simultaneous sounds were interfering with performance and revealed that only the second sound had an effect within the temporal window known to support multisensory integration. Experiment 4 demonstrated that sounds intervening between the two lights led to a decline in performance, as if the sounds pulled the lights closer together. The results suggest a ‘temporal ventriloquism’ phenomenon analogous to spatial ventriloquism.

Capturing the suffix: Cognitive streaming in immediate serial recall.

Adding an irrelevant item to the end of an auditory to-be-remembered list increases error on the last list items appreciably, known as the suffix effect. The phenomenon of auditory capture (e.g., Bregman & Rudnicky, 1975), namely, the tendency for a sequence of similar items to form a stream that at the same time isolates perceptually dissimilar members of the sequence, is exploited to explore the suffix effect. Irrelevant items interleaved between to-be-remembered items are used to capture the suffix with the aim of reducing its impact. Four experiments illustrate how the properties of the irrelevant sequence promote capture. The results are problematic for models of the suffix that involve masking of the last list item; instead, models based on grouping are favored.

Capturing the suffix: Cognitive streaming in immediate serial recall.

Adding an irrelevant item to the end of an auditory to-be-remembered list increases error on the last list items appreciably, known as the suffix effect. The phenomenon of auditory capture (e.g., Bregman & Rudnicky, 1975), namely, the tendency for a sequence of similar items to form a stream that at the same time isolates perceptually dissimilar members of the sequence, is exploited to explore the suffix effect. Irrelevant items interleaved between to-be-remembered items are used to capture the suffix with the aim of reducing its impact. Four experiments illustrate how the properties of the irrelevant sequence promote capture. The results are problematic for models of the suffix that involve masking of the last list item; instead, models based on grouping are favored.

The effect of a textured visual field on modality dominance in a ventriloquism situation

Despite the often encountered affirmation that vision completely dominates other modalities in intersensory conflict, there are cases where discordant auditorv information affects the localization of a visual signal. Experiment I shows that “auditory capture” occurs with a visual input reduced to a single luminous point in complete darkness, but not with a textured background. The task was to point at a flashing luminous point alternately in the presence of a synchronous sound coming from a source situated 15° to one side (“conflict trials,” designed to measure immediate reaction to conflict) and in its absence (“test trials,” to measure aftereffects). Adaptive immédiate reactions and aftereffects were observed in the dark, but not with a textured background. In Experiment II, on the other hand, “visual capture” of auditory localization was observed at the levels of both measures in the dark and with the textured background. That visual texture affects the degree of auditory capture of vision, but not the degree of visual capture of audition was confirmed at the level of aftereffects in Experiment III, where bisensory monitoring was substituted for pointing during exposure to conflict. The empirical finding eliminates apparent contradictions in the literature on ventriloquism, but cannot itself be explained in terms either of relative accuracy of visual and auditory localization or attentional adjustments.