Multisensory Integration Processes Research Articles

Is that me or my twin? Lack of self-face recognition advantage in identical twins.

Despite the increasing interest in twin studies and the stunning amount of research on face recognition, the ability of adult identical twins to discriminate their own faces from those of their co-twins has been scarcely investigated. One’s own face is the most distinctive feature of the bodily self, and people typically show a clear advantage in recognizing their own face even more than other very familiar identities. Given the very high level of resemblance of their faces, monozygotic twins represent a unique model for exploring self-face processing. Herein we examined the ability of monozygotic twins to distinguish their own face from the face of their co-twin and of a highly familiar individual. Results show that twins equally recognize their own face and their twin’s face. This lack of self-face advantage was negatively predicted by how much they felt physically similar to their co-twin and by their anxious or avoidant attachment style. We speculate that in monozygotic twins, the visual representation of the self-face overlaps with that of the co-twin. Thus, to distinguish the self from the co-twin, monozygotic twins have to rely much more than control participants on the multisensory integration processes upon which the sense of bodily self is based. Moreover, in keeping with the notion that attachment style influences perception of self and significant others, we propose that the observed self/co-twin confusion may depend upon insecure attachment.

Listening to another sense: somatosensory integration in the auditory system.

Conventionally, sensory systems are viewed as separate entities, each with its own physiological process serving a different purpose. However, many functions require integrative inputs from multiple sensory systems and sensory intersection and convergence occur throughout the central nervous system. The neural processes for hearing perception undergo significant modulation by the two other major sensory systems, vision and somatosensation. This synthesis occurs at every level of the ascending auditory pathway: the cochlear nucleus, inferior colliculus, medial geniculate body and the auditory cortex. In this review, we explore the process of multisensory integration from (1) anatomical (inputs and connections), (2) physiological (cellular responses), (3) functional and (4) pathological aspects. We focus on the convergence between auditory and somatosensory inputs in each ascending auditory station. This review highlights the intricacy of sensory processing and offers a multisensory perspective regarding the understanding of sensory disorders.

Evidence for brain morphometric changes during the migraine cycle: a magnetic resonance-based morphometry study.

Neurophysiological investigations have demonstrated that there are unique fluctuations in the migraine brain functional activity between the ictal and interictal periods. Here we investigated the possibility that there are fluctuations over time also in whole brain morphometry of patients affected by episodic migraine without aura (MO).Twenty-four patients with untreated MO underwent 3T MRI scans during (n = 10) or between attacks (n = 14) and were compared to a group of 15 healthy volunteers (HVs). We then performed voxel-based-morphometry (VBM) analysis of structural T1-weighted MRI scans to determine if changes in brain structure were observed over the course of the migraine cycle.Interictally, MO patients had a significantly lower gray matter (GM) density within the right inferior parietal lobule, right temporal inferior gyrus, right superior temporal gyrus, and left temporal pole than did HVs. Ictally, GM density increased within the left temporal pole, bilateral insula, and right lenticular nuclei, but no areas exhibited decreased GM density.These morphometric GM changes between ictal and interictal phases suggest that abnormal structural plasticity may be an important mechanism of migraine pathology. Given the functional neuroanatomy of these areas, our findings suggest that migraine is a condition associated with global dysfunction of multisensory integration and memory processing.

Increased functional connectivity between superior colliculus and brain regions implicated in bodily self-consciousness during the rubber hand illusion.

Bodily self-consciousness refers to bodily processes operating at personal, peripersonal, and extrapersonal spatial dimensions. Although the neural underpinnings of representations of personal and peripersonal space associated with bodily self-consciousness were thoroughly investigated, relatively few is known about the neural underpinnings of representations of extrapersonal space relevant for bodily self-consciousness. In the search to unravel brain structures generating a representation of the extrapersonal space relevant for bodily self-consciousness, we developed a functional magnetic resonance imaging (fMRI) study to investigate the implication of the superior colliculus (SC) in bodily illusions, and more specifically in the rubber hand illusion (RHi), which constitutes an established paradigm to study the neural underpinnings of bodily self-consciousness. We observed activation of the colliculus ipsilateral to the manipulated hand associated with eliciting of RHi. A generalized form of context-dependent psychophysiological interaction analysis unravelled increased illusion-dependent functional connectivity between the SC and some of the main brain areas previously involved in bodily self-consciousness: right temporoparietal junction (rTPJ), bilateral ventral premotor cortex (vPM), and bilateral postcentral gyrus. We hypothesize that the collicular map of the extrapersonal space interacts with maps of the peripersonal and personal space generated at rTPJ, vPM and the postcentral gyrus, producing a unified representation of space that is relevant for bodily self-consciousness. We suggest that processes of multisensory integration of bodily-related sensory inputs located in this unified representation of space constitute one main factor underpinning emergence of bodily self-consciousness.

Slow changing postural cues cancel visual field dependence on self-tilt detection

Interindividual differences influence the multisensory integration process involved in spatial perception. Here, we assessed the effect of visual field dependence on self-tilt detection relative to upright, as a function of static vs. slow changing visual or postural cues. To that aim, we manipulated slow rotations (i.e., 0.05°s−1) of the body and/or the visual scene in pitch. Participants had to indicate whether they felt being tilted forward at successive angles. Results show that thresholds for self-tilt detection substantially differed between visual field dependent/independent subjects, when only the visual scene was rotated. This difference was no longer present when the body was actually rotated, whatever the visual scene condition (i.e., absent, static or rotated relative to the observer). These results suggest that the cancellation of visual field dependence by dynamic postural cues may rely on a multisensory reweighting process, where slow changing vestibular/somatosensory inputs may prevail over visual inputs.

EHMTI-0177. Evidence for plastic brain morphometric changes during the migraine cycle

Background Neurophysiological investigations have demonstrated that there are distinctive fluctuations in the brain’ se lectric signals between the ictal and interictal periods of recurrent migraine. Whether structural plasticity of the brain is also an important feature of episodic migraine remains unresolved. Aim We therefore investigated the possibility that there are fluctuations over time in whole brain grey matter morphometry of patients affected by episodic migraine without aura (MO). Method Twenty-four patients with untreated MO underwent MRI scans (3-Tesla Siemens Verio) during (n = 10) or between attacks (n = 14) and were compared to a group of 15 healthy volunteers (HV). We then performed voxelbased-morphometry (VBM) analysis of structural T1-weighted MRI scans to determine if changes in brain structure were observed over the course of the migraine cycle. Results During the interictal phase, MO patients had a significantly lower grey matter (GM) density within the right inferior parietal lobule, right temporal inferior gyrus, right superior temporal gyrus, and left temporal pole than did HV. During attacks, GM density increased within the left temporal pole, bilateral insula, and right lenticular nuclei, but no areas exhibited decreased GM density. Conclusion The morphometric GM changes between ictal and interictal phases reported in the present study suggest that abnormal structural plasticity may be an important mechanism of migraine pathology. Given the functional neuroanatomy of these areas, our findings suggest that migraine is a condition associated with global dysfunction of multisensory integration and memory processing. No conflict of interest.

Tuning and disrupting the brain-modulating the McGurk illusion with electrical stimulation.

In the so-called McGurk illusion, when the synchronized presentation of the visual stimulus /ga/ is paired with the auditory stimulus /ba/, people in general hear it as /da/. Multisensory integration processing underlying this illusion seems to occur within the Superior Temporal Sulcus (STS). Herein, we present evidence demonstrating that bilateral cathodal transcranial direct current stimulation (tDCS) of this area can decrease the McGurk illusion-type responses. Additionally, we show that the manipulation of this audio-visual integrated output occurs irrespective of the number of eye-fixations on the mouth of the speaker. Bilateral anodal tDCS of the Parietal Cortex also modulates the illusion, but in the opposite manner, inducing more illusion-type responses. This is the first demonstration of using non-invasive brain stimulation to modulate multisensory speech perception in an illusory context (i.e., both increasing and decreasing illusion-type responses to a verbal audio-visual integration task). These findings provide clear evidence that both the superior temporal and parietal areas contribute to multisensory integration processing related to speech perception. Specifically, STS seems fundamental for the temporal synchronization and integration of auditory and visual inputs. For its part, posterior parietal cortex (PPC) may adjust the arrival of incoming audio and visual information to STS thereby enhancing their interaction in this latter area.

The interaction between stimulus factors and cognitive factors during multisensory integration of audiovisual speech.

The interaction between stimulus factors and cognitive factors during multisensory integration of audiovisual speech.

A visual or tactile signal makes auditory speech detection more efficient by reducing uncertainty

Acoustic speech is easier to detect in noise when the talker can be seen. This finding could be explained by integration of multisensory inputs or refinement of auditory processing from visual guidance. In two experiments, we studied two-interval forced-choice detection of an auditory 'ba' in acoustic noise, paired with various visual and tactile stimuli that were identically presented in the two observation intervals. Detection thresholds were reduced under the multisensory conditions vs. the auditory-only condition, even though the visual and/or tactile stimuli alone could not inform the correct response. Results were analysed relative to an ideal observer for which intrinsic (internal) noise and efficiency were independent contributors to detection sensitivity. Across experiments, intrinsic noise was unaffected by the multisensory stimuli, arguing against the merging (integrating) of multisensory inputs into a unitary speech signal, but sampling efficiency was increased to varying degrees, supporting refinement of knowledge about the auditory stimulus. The steepness of the psychometric functions decreased with increasing sampling efficiency, suggesting that the 'task-irrelevant' visual and tactile stimuli reduced uncertainty about the acoustic signal. Visible speech was not superior for enhancing auditory speech detection. Our results reject multisensory neuronal integration and speech-specific neural processing as explanations for the enhanced auditory speech detection under noisy conditions. Instead, they support a more rudimentary form of multisensory interaction: the otherwise task-irrelevant sensory systems inform the auditory system about when to listen.

Possible anatomical pathways for short-latency multisensory integration processes in primary sensory cortices

Multisensory integration does not only recruit higher-level association cortex, but also low-level and even primary sensory cortices. Here, we will describe and quantify two types of anatomical pathways, a thalamocortical and a corticocortical that possibly underlie short-latency multisensory integration processes in the primary auditory (A1), somatosensory (S1), and visual cortex (V1). Results were obtained from Mongolian gerbils, a common model-species in neuroscience, using simultaneous injections of different retrograde tracers into A1, S1, and V1. Several auditory, visual, and somatosensory thalamic nuclei project not only to the primary sensory area of their own (matched) but also to areas of other (non-matched) modalities. The crossmodal output ratios of these nuclei, belonging to both core and non-core sensory pathways, vary between 0.4 and 63.5% of the labeled neurons. Approximately 0.3% of the sensory thalamic input to A1, 5.0% to S1, and 2.1% to V1 arise from non-matched nuclei. V1 has most crossmodal corticocortical connections, projecting strongest to S1 and receiving a similar amount of moderate inputs from A1 and S1. S1 is mainly interconnected with V1. A1 has slightly more projections to V1 than S1, but gets just faint inputs from there. Concerning the layer-specific distribution of the retrogradely labeled somata in cortex, V1 provides the most pronounced feedforward-type outputs and receives (together with S1) most pronounced feedback-type inputs. In contrast, A1 has most pronounced feedback-type outputs and feedforward-type inputs in this network. Functionally, the different sets of thalamocortical and corticocortical connections could underlie distinctive types of integration mechanisms for different modality pairings.

Visual-somatosensory integration and balance: evidence for psychophysical integrative differences in aging.

Research detailing multisensory integration (MSI) processes in aging and their association with clinically relevant outcomes is virtually non-existent. To our knowledge, the relationship between MSI and balance has not been well-established in aging. Given known alterations in unisensory processing with increasing age, the aims of the current study were to determine differential behavioral patterns of MSI in aging and investigate whether MSI was significantly associated with balance and fall-risk. Seventy healthy older adults (M = 75 years; 58% female) participated in the current study. Participants were instructed to make speeded responses to visual, somatosensory, and visual-somatosensory (VS) stimuli. Based on reaction times (RTs) to all stimuli, participants were classified into one of two groups (MSI or NO MSI), depending on their MSI RT benefit. Static balance was assessed using mean unipedal stance time. Overall, results revealed that RTs to VS stimuli were significantly shorter than those elicited to constituent unisensory conditions. Further, the current experimental design afforded differential patterns of multisensory processing, with 75% of the elderly sample demonstrating multisensory enhancements. Interestingly, 25% of older adults did not demonstrate multisensory RT facilitation; a finding that was attributed to extremely fast RTs overall and specifically in response to somatosensory inputs. Individuals in the NO MSI group maintained significantly better unipedal stance times and reported less falls, compared to elders in the MSI group. This study reveals the existence of differential patterns of multisensory processing in aging, while describing the clinical translational value of MSI enhancements in predicting balance and falls risk.

The role of the parietal cortex in multisensory and response integration: evidence from transcranial direct current stimulation (tDCS).

The question of how the brain forms unified representations from multisensory data that are processed in distinct cortical regions is known in the literature as 'the binding problem'. In the last decade, several studies have suggested possible neural mechanisms and brain regions that might be involved in integration processes. One of the brain regions that is implicated with multisensory perception is the posterior parietal cortex (PPC). Evidence from patients with parietal lesions suggests the involvement of the PPC in coherent perception. Here, we investigated the role of the PPC in multisensory feature integration through experimental manipulation of non-invasive brain stimulation with healthy participants using transcranial direct current stimulation (tDCS). In different sessions, healthy participants received anodal, cathodal, or sham stimulation (2 mA, 20 min) over the right PPC while performing an audio-visual event-file task. The results underscore two interesting observations. Firstly, there was a significant difference in integration effects between features from different modalities in the anodal stimulation compared to sham, suggesting interference of the multisensory integration processes during the brain stimulation. And secondly, after anodal stimulation, the unattended feature became more likely to be integrated with the response feature compared to the other conditions, presumably through an interference of attentional processes. Hence, these findings emphasize the role of the right PPC in multisensory integration. Furthermore, from a methodological perspective, tDCS can be used as an experimental tool by creating a temporary, reversible disruption in cognitive processes in order to explore the mechanisms underlying cognitive functions.

N1 enhancement in synesthesia during visual and audio–visual perception in semantic cross-modal conflict situations: an ERP study

Synesthesia entails a special kind of sensory perception, where stimulation in one sensory modality leads to an internally generated perceptual experience of another, not stimulated sensory modality. This phenomenon can be viewed as an abnormal multisensory integration process as here the synesthetic percept is aberrantly fused with the stimulated modality. Indeed, recent synesthesia research has focused on multimodal processing even outside of the specific synesthesia-inducing context and has revealed changed multimodal integration, thus suggesting perceptual alterations at a global level. Here, we focused on audio–visual processing in synesthesia using a semantic classification task in combination with visually or auditory–visually presented animated and in animated objects in an audio–visual congruent and incongruent manner. Fourteen subjects with auditory-visual and/or grapheme-color synesthesia and 14 control subjects participated in the experiment. During presentation of the stimuli, event-related potentials were recorded from 32 electrodes. The analysis of reaction times and error rates revealed no group differences with best performance for audio-visually congruent stimulation indicating the well-known multimodal facilitation effect. We found enhanced amplitude of the N1 component over occipital electrode sites for synesthetes compared to controls. The differences occurred irrespective of the experimental condition and therefore suggest a global influence on early sensory processing in synesthetes.

P 21. Temporal cortex transcranial direct current stimulation modulates the performance in an audio-visual integration task

Organisms are able to combine information from different sensory channels. This enables them to obtain a coherent representation of the environment by a process termed multisensory integration. The interaction of different sensory inputs minimizes perceptual uncertainty in localizing and detecting objects in the observers surrounding ( MacLeod and Summerfield, 1990 ). Thus, the integration of information arising from the auditory and visual domain is a crucial cognitive function for the perception of objects in a certain distance ( Welch and Warren, 1986 ). While audio-visual integration has been studied in the past showing multimodal and unimodal areas involved in this dual process, the influence of transcranial direct current stimulation (tDCS) on these cortical regions has been poorly explored. The aim of the present study was to investigate the behavioral and electrophysiological effects of tDCS applied over the left temporal cortex during an audio-visual integration task and in parallel to determine electrophysiological correlates of multisensory integration by means of electroencephalography (EEG). 10 healthy subjects received in a randomized order at three consecutive days anodal, cathodal or sham tDCS (referenced over the right supraorbital area) over the left temporal cortex for 30 min with a current strength of 2 mA. Subsequently, an audio-visual task ( Noesselt et al., 2010 ) and simultaneous EEG recording started. Participants had to detect Gabor Patches which were presented either solely or with a tone (cf. Fig. 1 ). These Gabor Patches differ in contrast sensitivity (high or low salience). Results show that tDCS over the left temporal cortex modulates behavioral performance in a multisensory integration task. In parallel, we found tDCS-induced modulations of event-related potential components demonstrating that tDCS can interact with the process of multisensory audio-visual integration. This study demonstrates tDCS-induced changes in multisensory processing. Modulating the reactivity of the auditory cortex by means of tDCS resulted in a modified performance and related electrophysiological responses in an audio-visual integration task.

Auditory-driven phase reset in visual cortex: Human electrocorticography reveals mechanisms of early multisensory integration

Findings in animal models demonstrate that activity within hierarchically early sensory cortical regions can be modulated by cross-sensory inputs through resetting of the phase of ongoing intrinsic neural oscillations. Here, subdural recordings evaluated whether phase resetting by auditory inputs would impact multisensory integration processes in human visual cortex. Results clearly showed auditory-driven phase reset in visual cortices and, in some cases, frank auditory event-related potentials (ERP) were also observed over these regions. Further, when audiovisual bisensory stimuli were presented, this led to robust multisensory integration effects which were observed in both the ERP and in measures of phase concentration. These results extend findings from animal models to human visual cortices, and highlight the impact of cross-sensory phase resetting by a non-primary stimulus on multisensory integration in ostensibly unisensory cortices.

Le sens de verticalité est-il vestibulaire ?

The vestibular system constitutes an inertial sensor, which detects linear (otoliths) and angular (semicircular canals) accelerations of the head in the three dimensions. The otoliths are specialized in the detection of linear accelerations and can be used by the brain as a "plumb line" coding earth gravity acceleration (direction). This property of otolithic system suggested that the sense of verticality is supported by the vestibular system. The preeminence of vestibular involvement in the sense of verticality stated in the 1900s was progressively supplanted by the notion of internal models of verticality. The internal models of verticality involve rules and properties of integration of vestibular graviception, somaesthesic graviception, and vision. The construction of a mental representation of verticality was mainly modeled as a bottom-up organization integrating visual, somatosensory and vestibular information without any cognitive modulations. Recent studies reported that the construction of internal models of verticality is not an automatic multi-sensory integration process but corresponds to more complex mechanisms including top-down influences such as awareness of body orientation or spatial representations.

Multisensory integration mechanisms during aging

The rapid demographical shift occurring in our society implies that understanding of healthy aging and age-related diseases is one of our major future challenges. Sensory impairments have an enormous impact on our lives and are closely linked to cognitive functioning. Due to the inherent complexity of sensory perceptions, we are commonly presented with a complex multisensory stimulation and the brain integrates the information from the individual sensory channels into a unique and holistic percept. The cerebral processes involved are essential for our perception of sensory stimuli and becomes especially important during the perception of emotional content. Despite ongoing deterioration of the individual sensory systems during aging, there is evidence for an increase in, or maintenance of, multisensory integration processing in aging individuals. Within this comprehensive literature review on multisensory integration we aim to highlight basic mechanisms and potential compensatory strategies the human brain utilizes to help maintain multisensory integration capabilities during healthy aging to facilitate a broader understanding of age-related pathological conditions. Further our goal was to identify where further research is needed.

Visual dominance and multisensory integration changes with age

Objects comprise of visual and auditory signatures that arrive through distinct sensory channels. Exposure to cross-modal events sets up expectations about what a given object most likely "sounds" like, and vice versa, thereby facilitating detection and recognition. Whereas episodic and working memory functions decline with age, the extent to which multisensory integration processes change with age remains an open question. In the present study, we examined whether multisensory integration processes play a compensatory role in normal aging. Magnetoencephalography recordings of semantically-related cross-modal and unimodal auditory and visual stimuli captured the spatiotemporal dynamics of multisensory responses in young and older adults. Whereas sensory-specific regions showed increased activity in response to cross-modal compared to unimodal stimuli 100 ms after stimulus onset in both age groups, posterior parietal and medial prefrontal regions responded preferentially to cross-modal stimuli between 150 and 300 ms in the older group only. Additionally, faster detection of cross-modal stimuli correlated with increased activity in inferior parietal and medial prefrontal regions 100 ms after stimulus onset in older compared to younger adults. Age-related differences in visual dominance were also observed with older adults exhibiting significantly larger multisensory facilitation effects relative to the auditory modality. Using structural equation modeling, we showed that age-related increases in parietal and medial prefrontal source activity predicted faster detection of cross-modal stimuli. Furthermore, the relationship between performance and source activity was mediated by age-related reductions in gray matter volume in those regions. Thus, we propose that multisensory integration processes change with age such that posterior parietal and medial prefrontal activity underlies the integrated response in older adults.

Parietal disruption alters audiovisual binding in the sound-induced flash illusion

Selective attention and multisensory integration are fundamental to perception, but little is known about whether, or under what circumstances, these processes interact to shape conscious awareness. Here, we used transcranial magnetic stimulation (TMS) to investigate the causal role of attention-related brain networks in multisensory integration between visual and auditory stimuli in the sound-induced flash illusion. The flash illusion is a widely studied multisensory phenomenon in which a single flash of light is falsely perceived as multiple flashes in the presence of irrelevant sounds. We investigated the hypothesis that extrastriate regions involved in selective attention, specifically within the right parietal cortex, exert an influence on the multisensory integrative processes that cause the flash illusion. We found that disruption of the right angular gyrus, but not of the adjacent supramarginal gyrus or of a sensory control site, enhanced participants' veridical perception of the multisensory events, thereby reducing their susceptibility to the illusion. Our findings suggest that the same parietal networks that normally act to enhance perception of attended events also play a role in the binding of auditory and visual stimuli in the sound-induced flash illusion.

A possible role of the locus coeruleus in complex regional pain syndrome

Heightened sensitivity to painful stimulation commonly spreads from the affected limb to the ipsilateral forehead in patients with complex regional pain syndrome (CRPS). In addition, acoustic startle evokes greater auditory discomfort and increases in limb pain when presented on the affected than unaffected side. In contrast, limb pain ordinarily evokes analgesia in the ipsilateral forehead of healthy participants, and acoustic startle suppresses limb pain. Together, these findings suggest that hemilateral and generalized pain control mechanisms are disrupted in CRPS, and that multisensory integrative processes are compromised. Failure to inhibit nociceptive input from the CRPS-affected limb could sensitize spinal and supraspinal neurons that receive convergent nociceptive and auditory information from hemilateral body sites. Somatosensory, auditory, and emotional inputs may then aggravate pain by feeding into this sensitized nociceptive network. In particular, a disturbance in hemilateral pain processing that involves the locus coeruleus could exacerbate the symptoms of CRPS in some patients.