Cross-modal Conditions Research Articles

Wireless electrocochleography in awake chinchillas: A model to study crossmodal modulations at the peripheral level

The discovery and development of electrocochleography (ECochG) in animal models has been fundamental for its implementation in clinical audiology and neurotology. In our laboratory, the use of round-window ECochG recordings in chinchillas has allowed a better understanding of auditory efferent functioning. In previous works, we gave evidence of the corticofugal modulation of auditory-nerve and cochlear responses during visual attention and working memory. However, whether these cognitive top-down mechanisms to the most peripheral structures of the auditory pathway are also active during audiovisual crossmodal stimulation is unknown. Here, we introduce a new technique, wireless ECochG to record compound-action potentials of the auditory nerve (CAP), cochlear microphonics (CM), and round-window noise (RWN) in awake chinchillas during a paradigm of crossmodal (visual and auditory) stimulation. We compared ECochG data obtained from four awake chinchillas recorded with a wireless ECochG system with wired ECochG recordings from six anesthetized animals. Although ECochG experiments with the wireless system had a lower signal-to-noise ratio than wired recordings, their quality was sufficient to compare ECochG potentials in awake crossmodal conditions. We found non-significant differences in CAP and CM amplitudes in response to audiovisual stimulation compared to auditory stimulation alone (clicks and tones). On the other hand, spontaneous auditory-nerve activity (RWN) was modulated by visual crossmodal stimulation, suggesting that visual crossmodal simulation can modulate spontaneous but not evoked auditory-nerve activity. However, given the limited sample of 10 animals (4 wireless and 6 wired), these results should be interpreted cautiously. Future experiments are required to substantiate these conclusions. In addition, we introduce the use of wireless ECochG in animal models as a useful tool for translational research.

ERP evidence for temporal differences between cross-modal and cross-domain analogical reasoning

Previous studies have shown that individuals not only successfully engage in cross-domain analogies but also accomplish cross-modal reasoning. Yet, the behavioral representation and neurophysiological basis of cross-modal and cross-domain analogical reasoning remain unclear. This study established three analogical reasoning conditions by combining a multi-to-multi learning-test paradigm with a four‑term analogy paradigm: within-domain, cross-domain, and cross-modal conditions. Thirty participants were required to judge whether the relationship between C and D was the same as the learned relationship between A and B. Behavioral results revealed no significant differences in reaction times and accuracy between cross-domain and cross-modal conditions, but both conditions showed significantly lower accuracy than within-domain condition. ERP results indicated a larger P2 amplitude in the cross-modal condition, while a larger N400 amplitude was observed in the cross-domain condition. These findings suggest: (1) The P2 in cross-modal analogical reasoning is associated with more difficult access to cross-modal information. (2) The N400 in cross-domain analogical reasoning is related to more challenging semantic processing. This study provides the first evidence of behavioral and ERP differences between cross-modal and cross-domain analogical reasoning, deepening our understanding of the cognitive processes involved in cross-modal analogical reasoning.

Crossmodal Texture Perception Is Illumination-Dependent.

Visually perceived roughness of 3D textures varies with illumination direction. Surfaces appear rougher when the illumination angle is lowered resulting in a lack of roughness constancy. Here we aimed to investigate whether the visual system also relies on illumination-dependent features when judging roughness in a crossmodal matching task or whether it can access illumination-invariant surface features that can also be evaluated by the tactile system. Participants ( N = 32) explored an abrasive paper of medium physical roughness either tactually, or visually under two different illumination conditions (top vs oblique angle). Subsequently, they had to judge if a comparison stimulus (varying in physical roughness) matched the previously explored standard. Matching was either performed using the same modality as during exploration (intramodal) or using a different modality (crossmodal). In the intramodal conditions, participants performed equally well independent of the modality or illumination employed. In the crossmodal conditions, participants selected rougher tactile matches after exploring the standard visually under oblique illumination than under top illumination. Conversely, after tactile exploration, they selected smoother visual matches under oblique than under top illumination. These findings confirm that visual roughness perception depends on illumination direction and show, for the first time, that this failure of roughness constancy also transfers to judgements made crossmodally.

Cochlear Preconditioning as a Modulator of Susceptibility to Hearing Loss.

Significance: Acquired sensorineural hearing loss is a major public health problem worldwide. The leading causes of sensorineural hearing loss are noise, aging, and ototoxic medications, with the key underlying pathology being damage to the cochlea. The review focuses on the phenomenon of preconditioning, in which the susceptibility to cochlear injury is reduced by exposing the ear to a stressful stimulus. Recent Advances: Cochlear conditioning has focused on the use of mono-modal conditioning, specifically conditioning the cochlea with moderate noise exposures before a traumatic exposure that causes permanent hearing loss. Recently, cross-modal conditioning has been explored more thoroughly, to prevent not only noise-induced hearing loss, but also age-related and drug-induced hearing losses. Critical Issues: Noise exposures that cause only temporary threshold shifts (TTSs) can cause long-term synaptopathy, injury to the synapses between the inner hair cells and spiral ganglion cells. This discovery has the potential to significantly alter the field of cochlear preconditioning with noise. Further, cochlear preconditioning can be the gateway to the development of clinically deployable therapeutics. Therefore, understanding the underlying mechanisms of conditioning is crucial for optimizing clinical protection against sensorineural hearing loss. Future Directions: Before the discovery of synaptopathy, noise exposures that caused only TTSs were believed to be either harmless or potentially beneficial. Any considerations of preconditioning with noise must consider the potential for injury to the synapses. Further, the discovery of different methods to precondition the cochlea against injury will yield new avenues for protection against hearing loss in the vulnerable populations. Antioxid. Redox Signal. 36, 1215-1228.

Crossmodal Pattern Discrimination in Humans and Robots: A Visuo-Tactile Case Study.

The quality of crossmodal perception hinges on two factors: The accuracy of the independent unimodal perception and the ability to integrate information from different sensory systems. In humans, the ability for cognitively demanding crossmodal perception diminishes from young to old age. Here, we propose a new approach to research to which degree the different factors contribute to crossmodal processing and the age-related decline by replicating a medical study on visuo-tactile crossmodal pattern discrimination utilizing state-of-the-art tactile sensing technology and artificial neural networks (ANN). We implemented two ANN models to specifically focus on the relevance of early integration of sensory information during the crossmodal processing stream as a mechanism proposed for efficient processing in the human brain. Applying an adaptive staircase procedure, we approached comparable unimodal classification performance for both modalities in the human participants as well as the ANN. This allowed us to compare crossmodal performance between and within the systems, independent of the underlying unimodal processes. Our data show that unimodal classification accuracies of the tactile sensing technology are comparable to humans. For crossmodal discrimination of the ANN the integration of high-level unimodal features on earlier stages of the crossmodal processing stream shows higher accuracies compared to the late integration of independent unimodal classifications. In comparison to humans, the ANN show higher accuracies than older participants in the unimodal as well as the crossmodal condition, but lower accuracies than younger participants in the crossmodal task. Taken together, we can show that state-of-the-art tactile sensing technology is able to perform a complex tactile recognition task at levels comparable to humans. For crossmodal processing, human inspired early sensory integration seems to improve the performance of artificial neural networks. Still, younger participants seem to employ more efficient crossmodal integration mechanisms than modeled in the proposed ANN. Our work demonstrates how collaborative research in neuroscience and embodied artificial neurocognitive models can help to derive models to inform the design of future neurocomputational architectures.

Listen to Your Heart: Examining Modality Dominance Using Cross-Modal Oddball Tasks.

The current study used cross-modal oddball tasks to examine cardiac and behavioral responses to changing auditory and visual information. When instructed to press the same button for auditory and visual oddballs, auditory dominance was found with cross-modal presentation slowing down visual response times more than auditory response times (Experiment 1). When instructed to make separate responses to auditory and visual oddballs, visual dominance was found with cross-modal presentation decreasing auditory discrimination, and participants also made more visual-based than auditory-based errors on cross-modal trials (Experiment 2). Experiment 3 increased task demands while requiring a single button press and found evidence of auditory dominance, suggesting that it is unlikely that increased task demands can account for the reversal in Experiment 2. Auditory processing speed was the best predictor of auditory dominance, with auditory dominance being stronger in participants who were slower at processing the sounds, whereas auditory and visual processing speed and baseline heart rate variability did not predict visual dominance. Examination of cardiac responses that were time-locked with stimulus onset showed cross-modal facilitation effects, with auditory and visual discrimination occurring earlier in the course of processing in the cross-modal condition than in the unimodal conditions. The current findings showing that response demand manipulations reversed modality dominance and that time-locked cardiac responses show cross-modal facilitation, not interference, suggest that auditory and visual dominance effects may both be occurring later in the course of processing, not from disrupted encoding.

Tracking the Transitions of Brain States: An Analytical Approach Using EEG

Classification of the neural activity of the brain is a well known problem in the field of brain computer interface. Machine learning based approaches for classification of brain activities do not reveal the underlying dynamics of the human brain. Since eigen decomposition has been found useful in a variety of applications, we conjecture that change of brain states would manifest in terms of changes in the invariant spaces spanned by eigen vectors as well as amount of variance along them. Based on this, our first approach is to track the brain state transitions by analysing invariant space variations over time. Whereas, our second approach analyses sub-band characteristic response vector formed using eigen values along with the eigen vectors to capture the dynamics. We have taken two real time EEG datasets to demonstrate the efficacy of proposed approaches. It has been observed that in case of unimodal experiment, invariant spaces explicitly show the transitions of brain states. Whereas sub-band characteristic response vector approach gives better performance in the case of cross-modal conditions. Evolution of invariant spaces along with the eigen values may help in understanding and tracking the brain state transitions. The proposed approaches can track the activity transitions in real time. They do not require any training dataset.

Selective visual and crossmodal impairment in the discrimination of anger and fear expressions in severe alcohol use disorder

BackgroundSevere alcohol use disorder (SAUD) is associated with impaired discrimination of emotional expressions. This deficit appears increased in crossmodal settings, when simultaneous inputs from different sensory modalities are presented. However, so far, studies exploring emotional crossmodal processing in SAUD relied on static faces and unmatched face/voice pairs, thus offering limited ecological validity. Our aim was therefore to assess emotional processing using a validated and ecological paradigm relying on dynamic audio-visual stimuli, manipulating the amount of emotional information available. MethodThirty individuals with SAUD and 30 matched healthy controls performed an emotional discrimination task requiring to identify five emotions (anger, disgust, fear, happiness, sadness) expressed as visual, auditory, or auditory-visual segments of varying length. Sensitivity indices (d’) were computed to get an unbiased measure of emotional discrimination and entered in a Generalized Linear Mixed Model. Incorrect emotional attributions were also scrutinized through confusion matrices. ResultsDiscrimination levels varied across sensory modalities and emotions, and increased with stimuli duration. Crucially, performances also improved from unimodal to crossmodal conditions in both groups, but discrimination for anger crossmodal stimuli and fear crossmodal/visual stimuli remained selectively impaired in SAUD. These deficits were not influenced by stimuli duration, suggesting that they were not modulated by the amount of emotional information available. Moreover, they were not associated with systematic emotional error patterns reflecting specific confusions between emotions. ConclusionsThese results clarify the nature and extent of crossmodal impairments in SAUD and converge with earlier findings to ascribe a specific role for anger and fear in this pathology.

Unimodal and crossmodal working memory binding is not differentially affected by age or Alzheimer's disease.

In this study, we investigated whether Crossmodal WMB is differentially affected by normal or pathological aging compared to Unimodal WMB. Experiment 1: 26 older and 26 younger adults recalled the target feature matching the test probe to complete a previously displayed color-shape binding (visually presented in the Unimodal condition; auditorily and visually presented in the Crossmodal condition). Experiment 2: 35 older and 35 younger adults undertook the same paradigm while carrying out articulatory suppression to limit verbal recoding. Experiment 3: 24 Alzheimer's disease (AD) patients and two groups of 24 healthy matched controls (tested respectively with the same and an increased memory load compared to the patients) were recruited to perform a similar task. Results show no age-related additional cost in Crossmodal WMB in respect to Unimodal WMB. AD patients had poor attainment in both WMB tasks regardless of specific binding condition. These findings provide evidence identifying WMB per se to be impaired in AD, regardless of the type of to-be-bound material. This supports the view that WMB is a suitable cognitive marker for AD. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

How modality processing differences affect cross-modal nonspatial repetition inhibition.

Although previous studies have demonstrated that identity-based repetition inhibition could occur across modalities, whether the modality processing difference or attentional set caused differences between the unimodal and cross-modal conditions was unknown. To investigate this question in both visual-auditory and auditory-visual patterns, the present study adopted a cross-modal "prime-neutral cue-target" priming paradigm, in which a neutral event was presented between the prime and the target. The relationships of the identities and modalities between the prime and the target were manipulated such that their modalities and identities could either be the same or different. Our results showed that (a) identity-based repetition inhibition occurred under both unimodal and cross-modal conditions, (b) response times to auditory targets were slower than those to visual targets, and (c) identity-based repetition inhibition was larger while discriminating repeated auditory targets than visual targets regardless of whether the prime was visual or auditory. These results suggested that nonspatial repetition inhibition can occur across modalities and that it was not in general larger or smaller than unimodal repetition inhibition, as this difference was due to modality processing differences.

Cross-modal processing of emotions in severe alcohol use disorders: Impaired discrimination of anger and fear under dynamic audiovisual conditions

Severe alcohol use disorders (SAUD) are associated with a large variety of affective disturbances, among which a well-established decoding deficit for facial and vocal emotional expressions. This deficit has recently been found to be increased in cross-modal settings, namely when inputs from different sensory modalities have to be combined. Compared to unimodal emotional stimuli, cross-modal ones allow for faster and more accurate emotional predictions, and therefore constitute critical cues for social interactions. However, so far, studies exploring emotional cross-modal processing in SAUD relied on static faces, associated with voices from a different individual, largely hampering ecological validity. Besides, in real life conditions, emotions are often not fully expressed, so that we have to make guesses based on incomplete information. Accordingly, the aim of this study was to assess cross-modal emotional processing using a new ecological paradigm with dynamic audiovisual stimuli, manipulating the amount of emotional information available to the individual. Thirty individuals with SAUD and 30 matched healthy controls performed an emotional discrimination task requiring to identify emotions (anger, disgust, fear, happiness, sadness) expressed in short movies containing visual, auditory or auditory-visual information of various durations. The shortest excerpts revealed the very early emotional sketch (i.e., initial facial movements and prosody) while the longest ones depicted a more complete emotion. Sensitivity analyses (d’) showed that discrimination levels varied across sensory modalities and emotions, and increased with stimuli duration in both groups. Individuals with SAUD's performances improved from unimodal to cross-modal conditions, but their discrimination for cross-modal stimuli was impaired for anger and fear. This deficit was not influenced by the amount of information displayed, suggesting that it persists even when more emotional information is available. Results are discussed in light of the predictive mechanisms underlying emotion recognition, and converge with earlier findings to ascribe a specific role for anger and fear in SAUD.

Within- and Cross-Modal Integration and Attention in the Autism Spectrum.

Although impairment in sensory integration is suggested in the autism spectrum (AS), empirical evidences remain equivocal. We assessed the integration of low-level visual and tactile information within and across modalities in AS and typically developing (TD) individuals. TD individualsdemonstrated increased redundancy gain for cross-modal relative to double tactile or visual stimulation, while AS individualsshowed similar redundancy gain between cross-modal and double tactile conditions. We further observed that violation of the race model inequality for cross-modal conditions was observed over a wider proportion of the reaction times distribution in TD than ASindividuals. Importantly, the reduced cross-modal integration in AS individualswas not related to atypical attentional shift between modalities. We conclude that AS individualsdisplays selective decrease of cross-modal integration of low-level information.

Explaining the Effect of Likelihood Manipulation and Prior Through a Neural Network of the Audiovisual Perception of Space.

Results in the recent literature suggest that multisensory integration in the brain follows the rules of Bayesian inference. However, how neural circuits can realize such inference and how it can be learned from experience is still the subject of active research. The aim of this work is to use a recent neurocomputational model to investigate how the likelihood and prior can be encoded in synapses, and how they affect audio-visual perception, in a variety of conditions characterized by different experience, different cue reliabilities and temporal asynchrony. The model considers two unisensory networks (auditory and visual) with plastic receptive fields and plastic crossmodal synapses, trained during a learning period. During training visual and auditory stimuli are more frequent and more tuned close to the fovea. Model simulations after training have been performed in crossmodal conditions to assess the auditory and visual perception bias: visual stimuli were positioned at different azimuth (±10° from the fovea) coupled with an auditory stimulus at various audio-visual distances (±20°). The cue reliability has been altered by using visual stimuli with two different contrast levels. Model predictions are compared with behavioral data. Results show that model predictions agree with behavioral data, in a variety of conditions characterized by a different role of prior and likelihood. Finally, the effect of a different unimodal or crossmodal prior, re-learning, temporal correlation among input stimuli, and visual damage (hemianopia) are tested, to reveal the possible use of the model in the clarification of important multisensory problems.

Modeling implicit learning in a cross-modal audio-visual serial reaction time task

This study examined implicit learning in a cross-modal condition, where visual and auditory stimuli were presented in an alternating fashion. Each cross-modal transition occurred with a probability of 0.85, enabling participants to gain a reaction time benefit by learning the cross-modal predictive information between colors and tones. Motor responses were randomly remapped to ensure that pure perceptual learning took place. The effect for the implicit learning was extracted from the data by fitting five different models to the data, which was highly variable due to motor variability. To examine individual learning rates for stimulus types of different discriminability and modality, the models were fitted per stimulus type and individually for each participant. The model selection identified the model that included motor variability, surprise effects for deviants and a serial position for effect onset as the most explanatory (Akaike weight 0.87). Further, there was a significant global cross-modal implicit learning effect for predictable versus deviant transitions (40 ms reaction time difference, p < 0.004). The learning rates over time differed for both modality and the stimuli within modalities, although there was no correlation to global error rates or reaction time differences between the stimulus types. These results demonstrate a modeling method that is well suited to extract detailed information about the success of implicit learning from high variability data. It further shows a cross-modal implicit learning effect, which extends the understanding of the implicit learning system and highlights the possibility for information to be processed in a cross-modal representation without conscious processing.

Response Modality vs. Target Modality: Sensory Transformations and Comparisons in Cross-modal Slant Matching Tasks

Humans constantly combine multi-sensory spatial information to successfully interact with objects in peripersonal space. Previous studies suggest that sensory inputs of different modalities are encoded in different reference frames. In cross-modal tasks where the target and response modalities are different, it is unclear which reference frame these multiple sensory signals are transformed to for comparison. The current study used a slant perception and parallelity paradigm to explore this issue. Participants perceived (either visually or haptically) the slant of a reference board and were asked to either adjust an invisible test board by hand manipulation or to adjust a visible test board through verbal instructions to be physically parallel to the reference board. We examined the patterns of constant error and variability of unimodal and cross-modal tasks with various reference slant angles at different reference/test locations. The results revealed that rather than a mixture of the patterns of unimodal conditions, the pattern in cross-modal conditions depended almost entirely on the response modality and was not substantially affected by the target modality. Deviations in haptic response conditions could be predicted by the locations of the reference and test board, whereas the reference slant angle was an important predictor in visual response conditions.

Effects of modality and repetition in a continuous recognition memory task: Repetition has no effect on auditory recognition memory

Previous research has shown that auditory recognition memory is poorer compared to visual and cross-modal (visual and auditory) recognition memory. The effect of repetition on memory has been robust in showing improved performance. It is not clear, however, how auditory recognition memory compares to visual and cross-modal recognition memory following repetition. Participants performed a recognition memory task, making old/new discriminations to new stimuli, stimuli repeated for the first time after 4–7 intervening items (R1), or repeated for the second time after 36–39 intervening items (R2). Depending on the condition, participants were either exposed to visual stimuli (2D line drawings), auditory stimuli (spoken words), or cross-modal stimuli (pairs of images and associated spoken words). Results showed that unlike participants in the visual and cross-modal conditions, participants in the auditory recognition did not show improvements in performance on R2 trials compared to R1 trials. These findings have implications for pedagogical techniques in education, as well as for interventions and exercises aimed at boosting memory performance.

Beta oscillations reflect supramodal information during perceptual judgment

Previous work on perceptual decision making in the sensorimotor system has shown population dynamics in the beta band, corresponding to the encoding of stimulus properties and the final decision outcome. Here, we asked how oscillatory dynamics in the medial premotor cortex (MPC) contribute to supramodal perceptual decision making. We recorded local field potentials (LFPs) and spikes in two monkeys trained to perform a tactile-acoustic frequency discrimination task, including both unimodal and crossmodal conditions. We studied the role of oscillatory activity as a function of stimulus properties (frequency and sensory modality), as well as decision outcome. We found that beta-band power correlated with relevant stimulus properties: there was a significant modulation by stimulus frequency during the working-memory (WM) retention interval, as well as modulation by stimulus modality-the latter was observed only in the case of a purely unimodal task, where modality information was relevant to prepare for the upcoming second stimulus. Furthermore, we found a significant modulation of beta power during the comparison and decision period, which was predictive of decision outcome. Finally, beta-band spike-field coherence (SFC) matched these LFP observations. In conclusion, we demonstrate that beta power in MPC is reflective of stimulus features in a supramodal, context-dependent manner, and additionally reflects the decision outcome. We propose that these beta modulations are a signature of the recruitment of functional neuronal ensembles, which encode task-relevant information.

Development of a Bayesian Estimator for Audio-Visual Integration: A Neurocomputational Study

The brain integrates information from different sensory modalities to generate a coherent and accurate percept of external events. Several experimental studies suggest that this integration follows the principle of Bayesian estimate. However, the neural mechanisms responsible for this behavior, and its development in a multisensory environment, are still insufficiently understood. We recently presented a neural network model of audio-visual integration (Neural Computation, 2017) to investigate how a Bayesian estimator can spontaneously develop from the statistics of external stimuli. Model assumes the presence of two unimodal areas (auditory and visual) topologically organized. Neurons in each area receive an input from the external environment, computed as the inner product of the sensory-specific stimulus and the receptive field synapses, and a cross-modal input from neurons of the other modality. Based on sensory experience, synapses were trained via Hebbian potentiation and a decay term. Aim of this work is to improve the previous model, including a more realistic distribution of visual stimuli: visual stimuli have a higher spatial accuracy at the central azimuthal coordinate and a lower accuracy at the periphery. Moreover, their prior probability is higher at the center, and decreases toward the periphery. Simulations show that, after training, the receptive fields of visual and auditory neurons shrink to reproduce the accuracy of the input (both at the center and at the periphery in the visual case), thus realizing the likelihood estimate of unimodal spatial position. Moreover, the preferred positions of visual neurons contract toward the center, thus encoding the prior probability of the visual input. Finally, a prior probability of the co-occurrence of audio-visual stimuli is encoded in the cross-modal synapses. The model is able to simulate the main properties of a Bayesian estimator and to reproduce behavioral data in all conditions examined. In particular, in unisensory conditions the visual estimates exhibit a bias toward the fovea, which increases with the level of noise. In cross modal conditions, the SD of the estimates decreases when using congruent audio-visual stimuli, and a ventriloquism effect becomes evident in case of spatially disparate stimuli. Moreover, the ventriloquism decreases with the eccentricity.

Impaired facial and vocal emotion decoding in schizophrenia is underpinned by basic perceptivo-motor deficits

ABSTRACTIntroduction: Emotional decoding impairments have been largely demonstrated in schizophrenia for facial and prosodic stimuli, when presented separately. Nevertheless, the exploration of crossmodal integration has been far less considered, despite its omnipresence in daily social interactions. Moreover, the role played by basic visuo-motor impairments in unimodal and crossmodal decoding remains unexplored.Methods: Thirty-two patients were compared with 32 matched controls in an emotional decoding task including unimodal (visual and auditory) and crossmodal (congruent and incongruent) conditions. A control perceptive task was also conducted to take potential low-level perceptual deficits into account.Results: Schizoprenic patients presented lower performance and higher reaction times for both unimodal tasks (visual and auditory) and crossmodal conditions. Moreover, reaction times for the visuo-perceptive task were also significantly longer for patients compared to controls.Conclusions: The consistency of the results across unimodal and crossmodal tasks suggests a globalised emotional impairment in schizophrenia, independent of the sensorial modality and crossmodal nature of the stimuli. Centrally, given the results in the visuo-perceptive task, the impairments observed for emotional recognition appears at least partly explained by primary cognitive deficits, namely reduced processing speed.

Crossmodal processing of emotions in alcohol-dependence and Korsakoff syndrome

ABSTRACTIntroduction: Decoding emotional information from faces and voices is crucial for efficient interpersonal communication. Emotional decoding deficits have been found in alcohol-dependence (ALC), particularly in crossmodal situations (with simultaneous stimulations from different modalities), but are still underexplored in Korsakoff syndrome (KS). The aim of this study is to determine whether the continuity hypothesis, postulating a gradual worsening of cognitive and brain impairments from ALC to KS, is valid for emotional crossmodal processing.Methods: Sixteen KS, 17 ALC and 19 matched healthy controls (CP) had to detect the emotion (anger or happiness) displayed by auditory, visual or crossmodal auditory-visual stimuli. Crossmodal stimuli were either emotionally congruent (leading to a facilitation effect, i.e. enhanced performance for crossmodal condition compared to unimodal ones) or incongruent (leading to an interference effect, i.e. decreased performance for crossmodal condition due to discordant information across modalities). Reaction times and accuracy were recorded.Results: Crossmodal integration for congruent information was dampened only in ALC, while both ALC and KS demonstrated, compared to CP, decreased performance for decoding emotional facial expressions in the incongruent condition.Conclusions: The crossmodal integration appears impaired in ALC but preserved in KS. Both alcohol-related disorders present an increased interference effect. These results show the interest of more ecological designs, using crossmodal stimuli, to explore emotional decoding in alcohol-related disorders. They also suggest that the continuum hypothesis cannot be generalised to emotional decoding abilities.