Integration Of Multisensory Information Research Articles

MRI resting-state signature of the propensity to experience meaningful coincidences: a functional coupling analysis.

The propensity to experience meaningful patterns in random arrangements and unrelated events shows considerable interindividual differences. Reduced inhibitory control (over sensory processes) and decreased working memory capacities are associated with this trait, which implies that the activation of frontal as well as posterior brain regions may be altered during rest and working memory tasks. In addition, people experiencing more meaningful coincidences showed reduced gray matter of the left inferior frontal gyrus (IFG), which is linked to the inhibition of irrelevant information in working memory and the control and integration of multisensory information. To study deviations in the functional connectivity of the IFG with posterior associative areas, the present study investigated the fMRI resting state in a large sample of n = 101 participants. We applied seed-to-voxel analysis and found that people who perceive more meaningful coincidences showed negative functional connectivity of the left IFG (i.e. pars triangularis) with areas of the left posterior associative cortex (e.g. superior parietal cortex). A data-driven multivoxel pattern analysis further indicated that functional connectivity of a cluster located in the right cerebellum with a cluster including parts of the left middle frontal gyrus, left precentral gyrus, and the left IFG (pars opercularis) was associated with meaningful coincidences. These findings add evidence to the neurocognitive foundations of the propensity to experience meaningful coincidences, which strengthens the idea that deviations of working memory functions and inhibition of sensory and motor information explain why people experience more meaning in meaningless noise.

I-BaR: integrated balance rehabilitation framework.

Neurological diseases are observed in approximately 1 billion people worldwide. A further increase is foreseen at the global level as a result of population growth and aging. Individuals with neurological disorders often experience cognitive, motor, sensory, and lower extremity dysfunctions. Thus, the possibility of falling and balance problems arise due to the postural control deficiencies that occur as a result of the deterioration in the integration of multi-sensory information. We propose a novel rehabilitation framework, Integrated Balance Rehabilitation (I-BaR), to improve the effectiveness of the rehabilitation with objective assessment, individualized therapy, convenience with different disability levels and adoption of assist-as-needed paradigm and, with integrated rehabilitation process as whole, that is, ankle-foot preparation, balance, and stepping phases, respectively. Integrated Balance Rehabilitation allows patients to improve their balance ability by providing multi-modal feedback: visual via utilization of virtual reality; vestibular via anteroposterior and mediolateral perturbations with the robotic platform; proprioceptive via haptic feedback.

Decentralized Neural Circuits of Multisensory Information Integration in the Brain.

The brain combines multisensory inputs together to obtain a complete and reliable description of the world. Recent experiments suggest that several interconnected multisensory brain areas are simultaneously involved to integrate multisensory information. It was unknown how these mutually connected multisensory areas achieve multisensory integration. To answer this question, using biologically plausible neural circuit models we developed a decentralized system for information integration that comprises multiple interconnected multisensory brain areas. Through studying an example of integrating visual and vestibular cues to infer heading direction, we show that such a decentralized system is well consistent with experimental observations. In particular, we demonstrate that this decentralized system can optimally integrate information by implementing sampling-based Bayesian inference. The Poisson variability of spike generation provides appropriate variability to drive sampling, and the interconnections between multisensory areas store the correlation prior between multisensory stimuli. The decentralized system predicts that optimally integrated information emerges locally from the dynamics of the communication between brain areas and sheds new light on the interpretation of the connectivity between multisensory brain areas.

Neural correlates of the sound facilitation effect in the modified Simon task in older adults.

The ability to resolve interference declines with age and is attributed to neurodegeneration and reduced cognitive function and mental alertness in older adults. Our previous study revealed that task-irrelevant but environmentally meaningful sounds improve performance on the modified Simon task in older adults. However, little is known about neural correlates of this sound facilitation effect. Twenty right-handed older adults [mean age = 72 (SD = 4), 11 female] participated in the fMRI study. They performed the modified Simon task in which the arrows were presented either in the locations matching the arrow direction (congruent trials) or in the locations mismatching the arrow direction (incongruent trials). A total of 50% of all trials were accompanied by task-irrelevant but environmentally meaningful sounds. Participants were faster on the trials with concurrent sounds, independently of whether trials were congruent or incongruent. The sound effect was associated with activation in the distributed network of auditory, posterior parietal, frontal, and limbic brain regions. The magnitude of the behavioral facilitation effect due to sound was associated with the changes in activation of the bilateral auditory cortex, cuneal cortex, and occipital fusiform gyrus, precuneus, left superior parietal lobule (SPL) for No Sound vs. Sound trials. These changes were associated with the corresponding changes in reaction time (RT). Older adults with a recent history of falls showed greater activation in the left SPL than those without falls history. Our findings are consistent with the dedifferentiation hypothesis of cognitive aging. The facilitatory effect of sound could be achieved through recruitment of excessive neural resources, which allows older adults to increase attention and mental alertness during task performance. Considering that the SPL is critical for integration of multisensory information, individuals with slower task responses and those with a history of falls may need to recruit this region more actively than individuals with faster responses and those without a fall history to overcome increased difficulty with interference resolution. Future studies should examine the relationship among activation in the SPL, the effect of sound, and falls history in the individuals who are at heightened risk of falls.

Altered functional connectivity of the hippocampus with the sensorimotor cortex induced by long-term experience of virtual hand illusion

Synchronizing the movement of a virtual hand with an unseen real hand in a virtual environment is an effective method to induce a sense of ownership of the virtual hand. Although several neuroimaging studies have revealed neural mechanisms related to the ongoing process of ownership illusion, the effect of the long-term experience of ownership illusion on brain activity remains to be investigated. Here, we developed an apparatus based on real-time image matting and virtual reality technology which allowed us to use the image of participants’ real hands as their virtual hands and synchronize the movement of the virtual hands with the unseen real hands in a virtual scene. Resting-state functional imaging data were acquired immediately after participants completed several light office tasks with the virtual hands in either the virtual environment (virtual hand condition) or real environment (real hand condition). Significant positive functional connectivity of the hippocampus with the primary somatosensory and motor cortex was only observed in the virtual hand condition. The results provided novel evidence for the involvement of hippocampal-sensorimotor connection in the long-term experience of virtual hand ownership. The functional connection reorganization of the hippocampus might promote multi-sensory information integration into memory and updated the sense of body ownership, which offered important insights into the neural network underlying the availability of long-term use of VR technology in healthcare and rehabilitation.

Molecular sensors in the taste system of Drosophila.

Most animals, including humans and insects, consume foods based on their senses. Feeding is mostly regulated by taste and smell. Recent insect studies shed insight into the cross-talk between taste and smell, sweetness and temperature, sweetness and texture, and other sensory modality pairings. Five canonical tastes include sweet, umami, bitter, salty, and sour. Furthermore, other receptors that mediate the detection of noncanonical sensory attributes encoded by taste stimuli, such as Ca2+, Zn2+, Cu2+, lipid, and carbonation, have been characterized. Deorphanizing receptors and interactions among different modalities are expanding the taste field. Our study explores the taste system of Drosophila melanogaster and perception processing in insects to broaden the neuroscience of taste. Attractive and aversive taste cues and their chemoreceptors are categorized as tables. In addition, we summarize the recent progress in animal behavior as affected by the integration of multisensory information in relation to different gustatory receptor neuronal activations, olfaction, texture, and temperature. We mainly focus on peripheral responses and insect decision-making. Drosophila is an excellent model animal to study the cellular and molecular mechanism of the taste system. Despite the divergence in the receptors to detect chemicals, taste research in the fruit fly can offer new insights into the many different taste sensors of animals and how to test the interaction among different sensory modalities.

Approaching or Decentering? Differential Neural Networks Underlying Experiential Emotion Regulation and Cognitive Defusion.

The current study investigated the bottom-up experiential emotion regulation in comparison to the cognitiveve top down-approach of cognitive defusion. Rooted in an experiential- and client-centered psychotherapeutic approach, experiential emotion regulation involves an active, non-intervening, accepting, open and welcoming approach towards the bodily felt affective experience in a welcoming, compassionate way, expressed in ‘experiential awareness’ in a first phase, and its verbalization or ‘experiential expression’ in a second phase. Defusion refers to the ability to observe one’s thoughts and feelings in a detached manner. Nineteen healthy participants completed an emotion regulation task during fMRI scanning by processing highly arousing negative events by images. Both experiential emotion regulation and cognitive defusion resulted in higher negative emotion compared to a ‘watch’ control condition. On the neurophysiological level, experiential emotion regulation recruited brain areas that regulate attention towards affective- and somatosensorial experience such as the anterior cingulate cortex, the paracingulate gyrus, the inferior frontal gyrus, and the prefrontal pole, areas underlying multisensory information integration (e.g., angular gyrus), and linking body states to emotion recognition and awareness (e.g., postcentral gyrus). Experiential emotion regulation, relative to the control condition, also resulted in a higher interaction between the anterior insular cortex and left amygdala while participants experienced less negative emotion. Cognitive defusion decreased activation in the subcortical areas such as the brainstem, the thalamus, the amygdala, and the hippocampus. In contrast to cognitive defusion, experiential emotion regulation relative to demonstrated greater activation in the left angular gyrus, indicating more multisensory information integration. These findings provide insight into different and specific neural networks underlying psychotherapy-based experiential emotion regulation and cognitive defusion.

Directed network analysis reveals changes in cortical and muscular connectivity caused by different standing balance tasks

Objective. Standing balance forms the basis of daily activities that require the integration of multi-sensory information and coordination of multi-muscle activation. Previous studies have confirmed that the cortex is directly involved in balance control, but little is known about the neural mechanisms of cortical integration and muscle coordination in maintaining standing balance. Approach. We used a direct directed transfer function (dDTF) to analyze the changes in the cortex and muscle connections of healthy subjects (15 subjects: 13 male and 2 female) corresponding to different standing balance tasks. Main results. The results show that the topology of the EEG brain network and muscle network changes significantly as the difficulty of the balancing tasks increases. For muscle networks, the connection analysis shows that the connection of antagonistic muscle pairs plays a major role in the task. For EEG brain networks, graph theory-based analysis shows that the clustering coefficient increases significantly, and the characteristic path length decreases significantly with increasing task difficulty. We also found that cortex-to-muscle connections increased with the difficulty of the task and were significantly stronger than the muscle-to-cortex connections. Significance. These results show that changes in the difficulty of balancing tasks alter EEG brain networks and muscle networks, and an analysis based on the directed network can provide rich information for exploring the neural mechanisms of balance control.

Intermittent Visual Occlusions Increase Balance Training Effectiveness.

Improving dynamic balance can prevent falls in humans with neurological and mechanical deficits. Dynamic balance requires the neural integration of multisensory information to constantly assess the state of body mechanics. Prior research found that intermittent visual rotations improved balance training during walking on a narrow beam, but limitations from the immersive virtual reality headset hindered balance training effectiveness overall. We theorized that intermittent visual occlusions with electrically controlled liquid crystal glasses would overcome the previous limitations of the immersive virtual reality headset and provide a means to enhance dynamic balance training efficacy. Forty healthy young individuals walked on a treadmill-mounted balance beam for 30 min (20 subjects with intermittent visual occlusions and 20 subjects with unperturbed vision). Balance performance, in number of step-offs of the beam, improved by 78% for the visual occlusions group on the same day of the training, a near fourfold improvement compared to the 21% improvement for the unperturbed vision group (t(38) = –5.2, p < 0.001). The difference between groups was also apparent 2 weeks later testing for retention (60% improvement for the visual occlusions group, 5% for the unperturbed vision group; t(38) = –4.2, p < 0.001). Intermittent visual occlusions are likely a simple method for enhancing balance training in dynamic motor tasks.

Temporal dynamics of fMRI signal changes during conditioned interoceptive pain-related fear and safety acquisition and extinction

Associative learning and memory mechanisms drive interoceptive signaling along the gut-brain axis, thus shaping affective-emotional reactions and behavior. Specifically, learning to predict potentially harmful, visceral pain is assumed to succeed within very few trials. However, the temporal dynamics of cerebellar and cerebral fMRI signal changes underlying early acquisition and extinction of learned fear signals and the concomitant evolvement of safety learning remain incompletely understood.3 T fMRI data of healthy individuals from three studies were uniformly processed across the whole brain and the cerebellum. All studies employed differential delay conditioning (N = 94) with one visual cue (CS+) being repeatedly paired with visceral pain as unconditioned stimulus (US) while a second cue remained unpaired (CS-). During subsequent extinction (N = 51), all CS were presented without US.Behavioral results revealed increased CS+-aversiveness and CS--pleasantness after conditioning and diminished valence ratings for both CS following extinction. During early acquisition, the CS- induced linearly increasing neural activation in the insula, midcingulate cortex, hippocampus, precuneus as well as cerebral and cerebellar somatomotor regions. The comparison between acquisition and extinction phases yielded a CS--induced linear increase in the posterior cingulate cortex and precuneus during early acquisition, while there was no evidence for linear fMRI signal changes for the CS+ during acquisition and for both CS during extinction.Based on theoretical accounts of discrimination and temporal difference learning, these results suggest a gradual evolvement of learned safety cues that engage emotional arousal, memory, and cortical modulatory networks. As safety signals are presumably more difficult to learn and to discriminate from learned threat cues, the underlying temporal dynamics may reflect enhanced salience and prediction processing as well as increasing demands for attentional resources and the integration of multisensory information. Maladaptive responses to learned safety signals are a clinically relevant phenotype in multiple conditions, including chronic visceral pain, and can be exceptionally resistant to modification or extinction. Through sustained hypervigilance, safety seeking constitutes one key component in pain and stress-related avoidance behavior, calling for future studies targeting the mechanisms of safety learning and extinction to advance current cognitive-behavioral treatment approaches.

Altered Resting-State Intranetwork and Internetwork Functional Connectivity in Patients With Chronic Unilateral Vestibulopathy.

BackgroundChronic unilateral vestibulopathy (CUVP) is often accompanied by dizziness and postural instability, which restrict patients' daily activities. It is important to understand central compensation mechanisms underlying these symptoms in patients with CUVP by evaluating their brain functional status.PurposeTo analyze the changes in resting‐state intranetwork and internetwork functional connectivity (FC) and explore the state of central vestibular compensation in patients with CUVP.Study typeRetrospective.Population Eighteen patients with right‐sided CUVP and 18 age‐ and sex‐matched healthy controls.Field strength/sequenceA 3.0 T, three‐dimensional magnetization‐prepared rapid gradient‐echo (MP‐RAGE) and resting‐state echo‐planar imaging (EPI) functional MRI sequences.AssessmentFC alterations were explored using independent component analysis (ICA). Twelve independent components were identified via ICA. Dizziness Handicap Inventory (DHI) score for all patients was determined.Statistical testsTwo‐sample t test, family‐wise error (FWE) correction, Pearson correlation coefficient (r). A P value <0.05 was considered statistically significant.ResultsCompared with healthy controls, patients with CUVP showed significantly decreased FC in the right middle occipital gyrus within the lateral visual network, and significantly increased FC in the right supplementary motor area within the sensorimotor network. The FC was decreased between the medial visual and auditory networks, the right frontoparietal and posterior default networks, as well as the sensorimotor and auditory networks. There was a significant negative correlation between the FC changes in the visual, auditory networks and the DHI score in patients with CUVP (r = −0.583).Data conclusionCompared to healthy controls, the FC was significantly decreased in the right visual cortex and significantly enhanced in the right sensorimotor network in patients with CUVP. Patients with CUVP showed decreased FC between multiple whole‐brain networks, suggesting that abnormal integration of multisensory information may be involved in the occurrence of chronic dizziness and instability in patients with CUVP.Level of evidence4Technical EfficacyStage 2.

Neurophysiological correlates of age differences in driving behavior during concurrent subtask performance

Driving is a complex cognitive-motor task that requires the continuous integration of multisensory information, cognitive processes, and motor actions. With higher age, driving becomes increasingly challenging as a result of naturally declining neurophysiological resources. Performing additional subtasks, such as conversations with passengers or interactions with in-vehicle devices (e.g., adjusting the radio), may further challenge neurocognitive resources that are required to maintain driving performance. Based on declining brain physiological resources and inferior neurocognitive functioning, older adults (OA) may show higher brain activation and larger performance decrements than younger adults (YA) when engaging in additional subtasks during driving. Age differences, however, may further vary for different neurocognitive task demands, such that driving performance of OA might be particularly affected by certain subtasks. In this study, we hence investigated the brain functional correlates of age differences in driving behavior during concurrent subtask performance in YA and OA. Our final sample consisted of thirty younger (21.80 ± 1.73y, 15 female) and thirty older (69.43 ± 3.30y, 12 female) regular drivers that drove along a typical rural road (25 - 30 min) in a driving simulator and performed three different concurrent subtasks that were presented auditorily or visually: typing a 3-digit number (TYPE), comparing traffic news and gas station prices (working memory, WM), and stating arguments (ARG). We measured variability in lateral car position, velocity, and following distance to a frontal lead car as the standard deviation from 0 to 15 s after subtask onset. Brain activity was continuously recorded using functional near-infrared spectroscopy over the dorsolateral prefrontal cortex. Both YA and OA particularly varied in their lateral position during TYPE with a more pronounced effect in OA. For YA, in contrast, ARG led to higher variability in velocity compared to TYPE and WM, whereas OA showed no task-specific differences. Substantiating our behavioral findings, OA revealed the largest brain functional response to TYPE, while YA demonstrated a very distinct activation during ARG and smaller hemodynamic responses to TYPE and WM. Brain activity in the DLPFC was, overall, not significantly, but small to moderately related to certain behavioral performance parameters (mainly lateral position). We conclude that both OA and YA are vulnerable to distractive subtasks while driving. Age differences, however, seem to largely depend on neurocognitive task demands. OA may be at higher risk for accidents when performing visuo-motor subtasks (e.g., interacting with navigational systems) during driving while YA may be more (cognitively) distracted when talking to passengers.

From statistical regularities in multisensory inputs to peripersonal space representation and body ownership: Insights from a neural network model.

Peripersonal space (PPS), the interface between the self and the environment, is represented by a network of multisensory neurons with visual (or auditory) receptive fields anchored to specific body parts, and tactile receptive fields covering the same body parts. Neurophysiological and behavioural features of hand PPS representation have been previously modelled through a neural network constituted by one multisensory population integrating tactile inputs with visual/auditory external stimuli. Reference frame transformations were not explicitly modelled, as stimuli were encoded in pre‐computed hand‐centred coordinates. Here we present a novel model, aiming to overcome this limitation by including a proprioceptive population encoding hand position. We confirmed behaviourally the plausibility of the proposed architecture, showing that visuo‐proprioceptive information is integrated to enhance tactile processing on the hand. Moreover, the network's connectivity was spontaneously tuned through a Hebbian‐like mechanism, under two minimal assumptions. First, the plasticity rule was designed to learn the statistical regularities of visual, proprioceptive and tactile inputs. Second, such statistical regularities were simply those imposed by the body structure. The network learned to integrate proprioceptive and visual stimuli, and to compute their hand‐centred coordinates to predict tactile stimulation. Through the same mechanism, the network reproduced behavioural correlates of manipulations implicated in subjective body ownership: the invisible and the rubber hand illusion. We thus propose that PPS representation and body ownership may emerge through a unified neurocomputational process; the integration of multisensory information consistently with a model of the body in the environment, learned from the natural statistics of sensory inputs.

Neural Representations of the Voice Tremor Spectrum.

Voice tremor is a common movement disorder that manifests as involuntary oscillations of laryngeal muscles, leading to rhythmic alterations in voice pitch and loudness. Differential diagnosis of essential tremor of voice (ETv) is often challenging and includes dystonic tremor of voice (DTv), which is characterized by irregular, isometric contractions of laryngeal muscles during dystonic activity. Although clinical characteristics of voice tremor are well described, the pathophysiology underlying its heterogeneous phenomenology remains limited. We used a multimodal approach of functional magnetic resonance imaging for assessment of brain activity during symptomatic speech production, high-resolution magnetic resonance imaging for the examination of cortical thickness and gray matter volume, and diffusion-weighted imaging for evaluation of white matter integrity to identify disorder-specific neural alterations and their relationships with the symptomatology of ETv and DTv. We found a broad overlap between cortical alterations in ETv and DTv, involving sensorimotor regions responsible for the integration of multisensory information during speech production, such as primary sensorimotor, inferior/superior parietal, and inferior temporal cortices. In addition, ETv and DTv showed unique patterns of abnormalities in regions controlling speech motor preparation, which were localized in the cerebellum in ETv and the premotor cortex, insula, and superior temporal gyrus in DTv. Neural alterations in superior parietal and inferior temporal cortices were correlated with ETv severity, whereas changes in the left premotor cortex were associated with DTv severity. Our findings point to the pathophysiological spectrum underlying ETv and DTv and favor a more heterogeneous rather than dichotomous diagnostic classification of these voice tremor disorders. © 2020 International Parkinson and Movement Disorder Society.

Action force modulates action binding: evidence for a multisensory information integration explanation

Action binding refers to the observation that the perceived time of an action (e.g., a keypress) is shifted towards the distal sensory feedback (usually a sound) triggered by that action. Surprisingly, the role of somatosensory feedback for this phenomenon has been largely ignored. We fill this gap by showing that the somatosensory feedback, indexed by keypress peak force, is functional in judging keypress time. Specifically, the strength of somatosensory feedback is positively correlated with reported keypress time when the keypress is not associated with an auditory feedback and negatively correlated when the keypress triggers an auditory feedback. The result is consistent with the view that the reported keypress time is shaped by sensory information from different modalities. Moreover, individual differences in action binding can be explained by a sensory information weighting between somatosensory and auditory feedback. At the group level, increasing the strength of somatosensory feedback can decrease action binding to a level not being detected statistically. Therefore, a multisensory information integration account (between somatosensory and auditory inputs) explains action binding at both a group level and an individual level.

NeuroBayesSLAM: Neurobiologically inspired Bayesian integration of multisensory information for robot navigation

Spatial navigation depends on the combination of multiple sensory cues from idiothetic and allothetic sources. The computational mechanisms of mammalian brains in integrating different sensory modalities under uncertainty for navigation is enlightening for robot navigation. We propose a Bayesian attractor network model to integrate visual and vestibular inputs inspired by the spatial memory systems of mammalian brains. In the model, the pose of the robot is encoded separately by two sub-networks, namely head direction network for angle representation and grid cell network for position representation, using similar neural codes of head direction cells and grid cells observed in mammalian brains. The neural codes in each of the sub-networks are updated in a Bayesian manner by a population of integrator cells for vestibular cue integration, as well as a population of calibration cells for visual cue calibration. The conflict between vestibular cue and visual cue is resolved by the competitive dynamics between the two populations. The model, implemented on a monocular visual simultaneous localization and mapping (SLAM) system, termed NeuroBayesSLAM, successfully builds semi-metric topological maps and self-localizes in outdoor and indoor environments of difference characteristics, achieving comparable performance as previous neurobiologically inspired navigation systems but with much less computation complexity. The proposed multisensory integration method constitutes a concise yet robust and biologically plausible method for robot navigation in large environments. The model provides a viable Bayesian mechanism for multisensory integration that may pertain to other neural subsystems beyond spatial cognition.

The motor system (partially) deceives body representation biases in absence of visual correcting cues

The internal models of our body dimensions are prone to bias, but little evidence exists to explain how the motor system achieves fine-grained control despite these distortions. Previous work showed that the hand representation, assessed in a dynamic task (Proprioceptive Matching Task), was less distorted compared to that measured through a static body representation task (Localization Task), suggesting that either the hand representation was updated or the motor trajectory was adjusted during movement. The present study set out to shed light on this phenomenon by administering the Localization Task before and after either the Proprioceptive Matching Task or a control condition in a within-subjects design. Our results showed that hand map biases decreased during the Proprioceptive Matching Task, but that this increase in accuracy did not carry over to the Localization Task. In other words, more accurate performance in the dynamic body representation task does not reflect a change in how the hand is represented. Rather, it likely reflects a refinement of the motor trajectory, due to the integration of multisensory information, providing interesting insights into how the motor system partially overcomes biases in body representations.

Complex multi-modal sensory integration and context specificity in colour preferences of a pierid butterfly.

Innate colour preferences in insects were long considered to be a non-flexible representation of a floral 'search image' guiding them to flowers during initial foraging trips. However, these colour preferences have recently been shown to be modulated by multi-sensory integration of information. Using experiments on the butterfly Catopsilia pomona (common emigrant), we demonstrate that cross-modal integration of information not only affects colour preferences but also colour learning, and in a sex-specific manner. We show that spontaneous colour preference in this species is sexually dimorphic, with males preferring both blue and yellow while females prefer yellow. With minimal training (two training sessions), both males and females learned to associate blue with reward, but females did not learn green. This suggests that the aversion to green, in the context of foraging, is stronger in females than in males, probably because green is used as a cue to find oviposition sites in butterflies. However, females learned green after extensive training (five training sessions). Intriguingly, when a floral odour was present along with green during training, female colour preference during the subsequent choice tests resembled their innate preference (preference for yellow). Our results show that multi-sensory integration of information can influence preference, sensory bias, learning and memory in butterflies, thus modulating their behaviour in a context-specific manner.

A Laboratory Study of the McGurk Effect in 324 Monozygotic and Dizygotic Twins.

Multisensory integration of information from the talker’s voice and the talker’s mouth facilitates human speech perception. A popular assay of audiovisual integration is the McGurk effect, an illusion in which incongruent visual speech information categorically changes the percept of auditory speech. There is substantial interindividual variability in susceptibility to the McGurk effect. To better understand possible sources of this variability, we examined the McGurk effect in 324 native Mandarin speakers, consisting of 73 monozygotic (MZ) and 89 dizygotic (DZ) twin pairs. When tested with 9 different McGurk stimuli, some participants never perceived the illusion and others always perceived it. Within participants, perception was similar across time (r = 0.55 at a 2-year retest in 150 participants) suggesting that McGurk susceptibility reflects a stable trait rather than short-term perceptual fluctuations. To examine the effects of shared genetics and prenatal environment, we compared McGurk susceptibility between MZ and DZ twins. Both twin types had significantly greater correlation than unrelated pairs (r = 0.28 for MZ twins and r = 0.21 for DZ twins) suggesting that the genes and environmental factors shared by twins contribute to individual differences in multisensory speech perception. Conversely, the existence of substantial differences within twin pairs (even MZ co-twins) and the overall low percentage of explained variance (5.5%) argues against a deterministic view of individual differences in multisensory integration.

Respiration Entrainment in Virtual Reality by Using a Breathing Avatar.

Increasing evidence points to the role of interoception in body perception and in constructing the bodily self. Body ownership illusion (BOI) in virtual reality (VR) is a widely used paradigm to study body perception. However, existing research has focused mainly on exteroceptive sensory modalities, and studies on interoception and BOI remain scarce. The postulated mechanism of BOI is related to the multisensory integration of information and visuo-tactile or visuo-kinesthetic sensory conflict resolution. In this within-subjects experimental study, we introduced systematic visuo-interoceptive (visuo-respiratory) conflicts and tested if participants would resolve them by adjusting their respiration rate (RR). Participants observed a virtual breathing avatar body from the first-person perspective while their own RR was recorded. The VR system was connected to a respiration monitor. The avatar was first breathing for 60 seconds in accordance with the participant's RR; then, it was either slowing down or speeding up, each condition lasting for 180 seconds. The dependent variable was a change in participants' RR, expressed as a linear regression slope coefficient. Forty participants were included in each experimental condition, in a counterbalanced order. There was a change in RR in the predicted direction in both conditions. Participants' RR decreased on average by 0.48 breaths/minute and increased by 0.64 breaths/minute, leading to a change of 1.45 and 1.93 breaths/minute, respectively, over the entire timespan of the experiment. The difference between conditions was statistically significant (V = 192, p < 0.01). Because a change in RR of even 1 breath/minute is considered clinically significant, the results of this study-apart from demonstrating visuo-respiratory conflict resolution-may have an applied significance.