Crossmodal Aftereffects Research Articles

Cross-modal aftereffects of visuo-manual prism adaptation: Transfer to auditory divided attention in healthy subjects.

Prism adaptation was shown to modify auditory perception. Using a dichotic listening task, which assesses auditory divided attention, benefits of a rightward prism adaptation were demonstrated in neglect patients (i.e., a syndrome following right hemisphere brain damage) by reducing their left auditory extinction. It is currently unknown whether prism adaptation affects auditory divided attention in healthy subjects. In the present study, we investigated the aftereffects of prism adaptation on dichotic listening. A sample of 47 young adults performed a dichotic listening task, in which pairs of words were presented with two words sounded simultaneously, one in each ear. Three parameters were measured: The percentage of recalled words, the percentage of correctly recalled words, and the laterality index (LI). Prism adaptation to a leftward optical deviation (L-PA) significantly increased the overall percentage of recalled words (p = .044) and that from the right ear (p = .002), and the overall LI (p = .049). For the first time, these findings demonstrate that L-PA produced an orientation of the auditory divided attention in favor of the right ear in healthy participants. This asymmetrical aftereffect provides a new argument in favor of the cross-modal dimension of prism adaptation, although an acclimatization effect of the dichotic listening task is also discussed. Our study opens up a new avenue for using prism adaptation in the field of auditory rehabilitation requiring a modulation of auditory attention. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Perception it is: Processing level in multisensory selection

When repeatedly exposed to simultaneously presented stimuli, associations between these stimuli are nearly always established, both within as well as between sensory modalities. Such associations guide our subsequent actions and may also play a role in multisensory selection. Thus, crossmodal associations (i.e., associations between stimuli from different modalities) learned in a multisensory interference task might affect subsequent information processing. The aim of this study was to investigate the processing level of multisensory stimuli in multisensory selection by means of crossmodal aftereffects. Either feature or response associations were induced in a multisensory flanker task while the amount of interference in a subsequent crossmodal flanker task was measured. The results of Experiment 1 revealed the existence of crossmodal interference after multisensory selection. Experiments 2 and 3 then went on to demonstrate the dependence of this effect on the perceptual associations between features themselves, rather than on the associations between feature and response. Establishing response associations did not lead to a subsequent crossmodal interference effect (Experiment 2), while stimulus feature associations without response associations (obtained by changing the response effectors) did (Experiment 3). Taken together, this pattern of results suggests that associations in multisensory selection, and the interference of (crossmodal) distractors, predominantly work at the perceptual, rather than at the response, level.

Optic Flow Induces Nonvisual Self-Motion Aftereffects

There is strong evidence of shared neurophysiological substrates for visual and vestibular processing that likely support our capacity for estimating our own movement through the environment. We examined behavioral consequences of these shared substrates in the form of crossmodal aftereffects. In particular, we examined whether sustained exposure to a visual self-motion stimulus (i.e., optic flow) induces a subsequent bias in nonvisual (i.e., vestibular) self-motion perception in the opposite direction in darkness. Although several previous studies have investigated self-motion aftereffects, none have demonstrated crossmodal transfer, which is the strongest proof that the adapted mechanisms are generalized for self-motion processing. The crossmodal aftereffect was quantified using a motion-nulling procedure in which observers were physically translated on a motion platform to find the movement required to cancel the visually induced aftereffect. Crossmodal transfer was elicited only with the longest-duration visual adaptor (15 s), suggesting that transfer requires sustained vection (i.e., visually induced self-motion perception). Visual-only aftereffects were also measured, but the magnitudes of visual-only and crossmodal aftereffects were not correlated, indicating distinct underlying mechanisms. We propose that crossmodal aftereffects can be understood as an example of contingent or contextual adaptation that arises in response to correlations across signals and functions to reduce these correlations in order to increase coding efficiency. According to this view, crossmodal aftereffects in general (e.g., visual-auditory or visual-tactile) can be explained as accidental manifestations of mechanisms that constantly function to calibrate sensory modalities with each other as well as with the environment.

Adaptation aftereffects in vocal emotion perception elicited by expressive faces and voices.

The perception of emotions is often suggested to be multimodal in nature, and bimodal as compared to unimodal (auditory or visual) presentation of emotional stimuli can lead to superior emotion recognition. In previous studies, contrastive aftereffects in emotion perception caused by perceptual adaptation have been shown for faces and for auditory affective vocalization, when adaptors were of the same modality. By contrast, crossmodal aftereffects in the perception of emotional vocalizations have not been demonstrated yet. In three experiments we investigated the influence of emotional voice as well as dynamic facial video adaptors on the perception of emotion-ambiguous voices morphed on an angry-to-happy continuum. Contrastive aftereffects were found for unimodal (voice) adaptation conditions, in that test voices were perceived as happier after adaptation to angry voices, and vice versa. Bimodal (voice + dynamic face) adaptors tended to elicit larger contrastive aftereffects. Importantly, crossmodal (dynamic face) adaptors also elicited substantial aftereffects in male, but not in female participants. Our results (1) support the idea of contrastive processing of emotions (2), show for the first time crossmodal adaptation effects under certain conditions, consistent with the idea that emotion processing is multimodal in nature, and (3) suggest gender differences in the sensory integration of facial and vocal emotional stimuli.

Voice aftereffects of adaptation to speaker identity

While adaptation to complex auditory stimuli has traditionally been reported for linguistic properties of speech, the present study demonstrates non-linguistic high-level aftereffects in the perception of voice identity, following adaptation to voices or faces of personally familiar speakers. In Exp. 1, prolonged exposure to speaker A′s voice biased the perception of identity-ambiguous voice morphs between speakers A and B towards speaker B (and vice versa). Significantly biased voice identity perception was also observed in Exp. 2 when adaptors were videos of speakers’ silently articulating faces, although effects were reduced in magnitude relative to those seen in Exp. 1. By contrast, adaptation to an unrelated speaker C elicited an intermediate proportion of speaker A identifications in both experiments. While crossmodal aftereffects on auditory identification ( Exp. 2) dissipated rapidly, unimodal aftereffects ( Exp. 1) were still measurable a few minutes after adaptation. These novel findings suggest contrastive coding of voice identity in long-term memory, with at least two perceptual mechanisms of voice identity adaptation: one related to auditory coding of voice characteristics, and another related to multimodal coding of familiar speaker identity.

Motion Aftereffects Transfer between Touch and Vision

Current views on multisensory motion integration assume separate substrates where visual motion perceptually dominates tactile motion [1, 2]. However, recent neuroimaging findings demonstrate strong activation of visual motion processing areas by tactile stimuli [3-6], implying a potentially bidirectional relationship. To test the relationship between visual and tactile motion processing, we examined the transfer of motion aftereffects. In the well-known visual motion aftereffect, adapting to visual motion in one direction causes a subsequently presented stationary stimulus to be perceived as moving in the opposite direction [7, 8]. The existence of motion aftereffects in the tactile domain was debated [9-11], though robust tactile motion aftereffects have recently been demonstrated [12, 13]. By using a motion adaptation paradigm, we found that repeated exposure to visual motion in a given direction produced a tactile motion aftereffect, the illusion of motion in the opponent direction across the finger pad. We also observed that repeated exposure to tactile motion induces a visual motion aftereffect, biasing the perceived direction of counterphase gratings. These crossmodal aftereffects, operating both from vision to touch and from touch to vision, present strong behavioral evidence that the processing of visual and tactile motion rely on shared representations that dynamically impact modality-specific perception.

Simple and contingent aftereffects of perceived duration in vision and audition.

After repeated presentations of a long inspection tone (800 or 1,000 msec), a test tone of intermediate duration (600 msec) appeared shorter than it would otherwise appear. A short inspection tone (200 or 400 msec) tended to increase the apparent length of the intermediate test tone. Thus, a negative aftereffect of perceived auditory duration occurred, and a similar aftereffect occurred in the visual modality. These aftereffects, each involving a single sensory dimension, aresimple aftereffects. The following procedures producedcontingent aftereffects of perceived duration. A pair of lights, the first short and the second long, was presented repeatedly during an inspection period. When a pair of test lights of intermediate duration was then presented, the first member of the pair appeared longer in relation to the second. A similar aftereffect occurred in the auditory modality. In these latter aftereffects, the perceived duration of a test light or tone is contingent—dependent—on its temporal order, first or second, within a pair of test stimuli. An experiment designed to test the possibility of cross-modal transfer of contingent aftereffects between audition and vision found no significant cross-modal aftereffects.