Multisensory Behavior Research Articles

Perfluorooctane Sulfonate (PFOS) Negatively Impacts Prey Capture Capabilities in Larval Zebrafish.

Per- and polyfluoroalkyl substances (PFAS) are widely used in many industrial and domestic applications, which has resulted in unintentional human exposures and bioaccumulation in blood and other organs. Perfluorooctane sulfonate (PFOS) is among the most prevalent PFAS in the environment and has been postulated to affect brain functions in exposed organisms. However, the impacts of PFOS in early neural development have not been well described. We used zebrafish larvae to assess the effects of PFOS on two fundamental complex behaviors, prey capture and learning. Zebrafish exposed to PFOS concentrations ranging from 2 to 20 µM for differing 48-h periods were viable through early larval stages. In addition, PFOS uptake was unaffected by the presence of a chorion. We employed two different experimental paradigms; first we assessed the impacts of increasing organismal PFOS bioaccumulation on prey capture and learning, and second, we probed stage-specific sensitivity to PFOS by exposing zebrafish at different developmental stages (0-2 vs. 3-5 days post fertilization). Following both assays we measured the amount of PFOS present in each larva and found that PFOS levels varied in larvae from different groups within each experimental paradigm. Significant negative correlations were observed between larval PFOS accumulation and percentage of captured prey, whereas nonsignificant negative correlations were observed between PFOS accumulation and experienced-induced prey capture learning. These findings suggest that PFOS accumulation negatively affects larval zebrafish's ability to perform complicated multisensory behaviors and highlights the potential risks of PFOS exposure to animals in the wild, with implications for human health. Environ Toxicol Chem 2024;43:847-855. © 2023 SETAC.

Simulating tactile and visual multisensory behaviour in humans based on an MoS2 field effect transistor

Simulating tactile and visual multisensory behaviour in humans based on an MoS2 field effect transistor

Network-Based Differences in Top-Down Multisensory Integration between Adult ADHD and Healthy Controls-A Diffusion MRI Study.

Attention-deficit-hyperactivity disorder (ADHD) is a neurodevelopmental disorder neurobiologically conceptualized as a network disorder in white and gray matter. A relatively new branch in ADHD research is sensory processing. Here, altered sensory processing i.e., sensory hypersensitivity, is reported, especially in the auditory domain. However, our perception is driven by a complex interplay across different sensory modalities. Our brain is specialized in binding those different sensory modalities to a unified percept-a process called multisensory integration (MI) that is mediated through fronto-temporal and fronto-parietal networks. MI has been recently described to be impaired for complex stimuli in adult patients with ADHD. The current study relates MI in adult ADHD with diffusion-weighted imaging. Connectome-based and graph-theoretic analysis was applied to investigate a possible relationship between the ability to integrate multimodal input and network-based ADHD pathophysiology. Multishell, high-angular resolution diffusion-weighted imaging was performed on twenty-five patients with ADHD (six females, age: 30.08 (SD: 9.3) years) and twenty-four healthy controls (nine females; age: 26.88 (SD: 6.3) years). Structural connectome was created and graph theory was applied to investigate ADHD pathophysiology. Additionally, MI scores, i.e., the percentage of successful multisensory integration derived from the McGurk paradigm, were groupwise correlated with the structural connectome. Structural connectivity was elevated in patients with ADHD in network hubs mirroring altered default-mode network activity typically reported for patients with ADHD. Compared to controls, MI was associated with higher connectivity in ADHD between Heschl's gyrus and auditory parabelt regions along with altered fronto-temporal network integrity. Alterations in structural network integrity in adult ADHD can be extended to multisensory behavior. MI and the respective network integration in ADHD might represent the maturational cortical delay that extends to adulthood with respect to sensory processing.

Aberrant causal inference and presence of a compensatory mechanism in autism spectrum disorder.

Autism spectrum disorder (ASD) is characterized by a panoply of social, communicative, and sensory anomalies. As such, a central goal of computational psychiatry is to ascribe the heterogenous phenotypes observed in ASD to a limited set of canonical computations that may have gone awry in the disorder. Here, we posit causal inference - the process of inferring a causal structure linking sensory signals to hidden world causes - as one such computation. We show that audio-visual integration is intact in ASD and in line with optimal models of cue combination, yet multisensory behavior is anomalous in ASD because this group operates under an internal model favoring integration (vs. segregation). Paradoxically, during explicit reports of common cause across spatial or temporal disparities, individuals with ASD were less and not more likely to report common cause, particularly at small cue disparities. Formal model fitting revealed differences in both the prior probability for common cause (p-common) and choice biases, which are dissociable in implicit but not explicit causal inference tasks. Together, this pattern of results suggests (i) different internal models in attributing world causes to sensory signals in ASD relative to neurotypical individuals given identical sensory cues, and (ii) the presence of an explicit compensatory mechanism in ASD, with these individuals putatively having learned to compensate for their bias to integrate in explicit reports.

Polyglots’ multisensory language learning behaviour and its application in teaching languages

The article describes the study of Hebrew by Alexander Arguelles, one of the outstanding polyglots of our time. The monitoring results demonstrate that the polyglot himself is not always aware of his effective strategies of learning the language. For example, the active use of sensory-motor input of new language information was identified as one of the unconscious language acquisition strategies.

Multisensory enhancement of overt behavior requires multisensory experience.

The superior colliculus (SC) is richly endowed with neurons that integrate cues from different senses to enhance their physiological responses and the overt behaviors they mediate. However, in the absence of experience with cross-modal combinations (e.g., visual-auditory), they fail to develop this characteristic multisensory capability: Their multisensory responses are no greater than their most effective unisensory responses. Presumably, this impairment in neural development would be reflected as corresponding impairments in SC-mediated behavioral capabilities such as detection and localization performance. Here, we tested that assumption directly in cats raised to adulthood in darkness. They, along with a normally reared cohort, were trained to approach brief visual or auditory stimuli. The animals were then tested with these stimuli individually and in combination under ambient light conditions consistent with their rearing conditions and home environment as well as under the opposite lighting condition. As expected, normally reared animals detected and localized the cross-modal combinations significantly better than their individual component stimuli. However, dark-reared animals showed significant defects in multisensory detection and localization performance. The results indicate that a physiological impairment in single multisensory SC neurons is predictive of an impairment in overt multisensory behaviors.

Caenorhabditis elegans learning in a structured maze is a multisensory behavior

SummaryWe show that C. elegans nematodes learn to associate food with a combination of proprioceptive cues and information on the structure of their surroundings (maze), perceived through mechanosensation. By using the custom-made Worm-Maze platform, we demonstrate that C. elegans young adults can locate food in T-shaped mazes and, following that experience, learn to reach a specific maze arm. C. elegans learning inside the maze is possible after a single training session, it resembles working memory, and it prevails over conflicting environmental cues. We provide evidence that the observed learning is a food-triggered multisensory behavior, which requires mechanosensory and proprioceptive input, and utilizes cues about the structural features of nematodes' environment and their body actions. The CREB-like transcription factor and dopamine signaling are also involved in maze performance. Lastly, we show that the observed aging-driven decline of C. elegans learning ability in the maze can be reversed by starvation.

Sensory- and memory-related drivers for altered ventriloquism effects and aftereffects in older adults

The manner in which humans exploit multisensory information for subsequent decisions changes with age. Multiple causes for such age-effects are being discussed, including a reduced precision in peripheral sensory representations, changes in cognitive inference about causal relations between sensory cues, and a decline in memory contributing to altered sequential patterns of multisensory behaviour. To dissociate these putative contributions, we investigated how healthy young and older adults integrate audio-visual spatial information within trials (the ventriloquism effect) and between trials (the ventriloquism aftereffect). With both a model-free and (Bayesian) model-based analyses we found that both biases differed between groups. Our results attribute the age-change in the ventriloquism bias to a decline in spatial hearing rather than a change in cognitive processes. This decline in peripheral function, combined with a more prominent influence from preceding responses rather than preceding stimuli in the elderly, can also explain the observed age-effect in the ventriloquism aftereffect. Our results suggest a transition from a sensory-to a behavior-driven influence of past multisensory experience on perceptual decisions with age, due to reduced sensory precision and change in memory capacity.

Caenorhabditis Elegans Learning and Decision-Making in a Structured Environment is a Multisensory Behavior

C. elegans' ability to exhibit associative, non-associative and imprinted memory in the context of chemical stimuli is well studied. Here, we show that C. elegans nematodes are capable of spatial associative learning, as they learn to associate food with a combination of proprioceptive cues, and information on the structure of their surroundings (maze), perceived through mechanosensation. By using the custom-made Worm-Maze platform, we demonstrate that C. elegans young adults can locate food in T-shaped mazes and, following that experience, learn to reach a specific maze arm. We show that C. elegans learning of space-related information is possible after a single training session, it resembles working memory, and it prevails over conflicting environmental cues. We provide evidence that C. elegans maze learning is a food-triggered multisensory behavior, which requires mechanosensory and proprioceptive input, and utilizes cues about the structural features of nematodes’ environment and their body actions. The CREB-like transcription factor, known for its role in learning, and dopamine signaling are also involved in maze performance. Interestingly, the observed aging-driven decline of C. elegans spatial learning ability can be reversed by starvation.

Multitasking as multisensory behavior: Revisiting media multitasking in the perspective of media ecology theory

Media multitasking has become an inseparable part of the media routine for many Internet users as it is being retribalized, as a multisensory computer-mediated behavior, in the mobile media age. Guided by McLuhan's media ecology theory, this study examined the hierarchical relationships between media multitasking and media generations, mobile media use, and need for cognition among users in the United States and China (N = 1,340). After gender was controlled, media generations (defined as those growing up with or without the pervasive use of mobile media) was directly associated with media multitasking. Moreover, mobile media power use mediated the relationship between media generations and media multitasking. The personality construct of need for cognition was also found to play a critical role in performing multitasking. These findings indicate that mobile media technologies are reactivating multiple human senses in multitaskers, which might be a result of the evolutionary process spearheaded by the ubiquity of mobile media technologies. Thus, it is high time to revisit media multitasking as a new normal and access it as a multisensory behavior. McLuhan's media ecology theory helps understand the evolution of media multitasking, which may no longer be as simple as a technology distraction to users.

Spatial bias in estimating the position of visual and proprioceptive targets.

When people match an unseen hand to a visual or proprioceptive target, they make both variable and systematic (bias) errors. Variance is a well-established factor in behavior, but the origin and implications of bias, and its connection to variance, are poorly understood. Eighty healthy adults matched their unseen right index finger to proprioceptive (left index finger) and visual targets with no performance feedback. We asked whether matching bias was related to target modality and to the magnitude or spatial properties of matching variance. Bias errors were affected by target modality, with subjects estimating visual and proprioceptive targets 20 mm apart. We found three pieces of evidence to suggest a connection between bias and variable errors: 1) for most subjects, the target modality that yielded greater spatial bias was also estimated with greater variance; 2) magnitudes of matching bias and variance were somewhat correlated for each target modality ( R = 0.24 and 0.29); and 3) bias direction was closely related to the angle of the major axis of the confidence ellipse ( R = 0.60 and 0.63). However, whereas variance was significantly correlated with visuo-proprioceptive weighting as predicted by multisensory integration theory ( R = -0.29 and 0.27 for visual and proprioceptive variance, respectively), bias was not. In a second session, subjects improved their matching variance, but not bias, for both target modalities, indicating a difference in stability. Taken together, these results suggest bias and variance are related only in some respects, which should be considered in the study of multisensory behavior. NEW & NOTEWORTHY People matching visual or proprioceptive targets make both variable and systematic (bias) errors. Multisensory integration is thought to minimize variance, but if the less variable modality has more bias, behavioral accuracy will decrease. Our data set suggests this is unusual. However, although bias and variable errors were spatially related, they differed in both stability and correlation with multisensory weighting. This suggests the bias-variance relationship is not straightforward, and both should be considered in multisensory behavior.

Better than Optimal

Using a novel visual-tactile paradigm in rats, Nikbakht etal. (2018) describe multisensory behavior that outperformed predictions of optimal cue combination (indicating cross-modal synergy) and exposed encoding in PPC neurons (of stimulus and choice signals) that was independent of stimulus modality.

Integration of parallel mechanosensory and visual pathways resolved through sensory conflict

The acquisition of information from parallel sensory pathways is a hallmark of coordinated movement in animals. Insect flight, for example, relies on both mechanosensory and visual pathways. Our challenge is to disentangle the relative contribution of each modality to the control of behavior. Toward this end, we show an experimental and analytical framework leveraging sensory conflict, a means for independently exciting and modeling separate sensory pathways within a multisensory behavior. As a model, we examine the hovering flower-feeding behavior in the hawkmoth Manduca sexta In the laboratory, moths feed from a robotically actuated two-part artificial flower that allows independent presentation of visual and mechanosensory cues. Freely flying moths track lateral flower motion stimuli in an assay spanning both coupled motion, in which visual and mechanosensory cues follow the same motion trajectory, and sensory conflict, in which the two sensory modalities encode different motion stimuli. Applying a frequency-domain system identification analysis, we find that the tracking behavior is, in fact, multisensory and arises from a linear summation of visual and mechanosensory pathways. The response dynamics are highly preserved across individuals, providing a model for predicting the response to novel multimodal stimuli. Surprisingly, we find that each pathway in and of itself is sufficient for driving tracking behavior. When multiple sensory pathways elicit strong behavioral responses, this parallel architecture furnishes robustness via redundancy.

Altered multisensory temporal integration in obesity.

Eating is a multisensory behavior. The act of placing food in the mouth provides us with a variety of sensory information, including gustatory, olfactory, somatosensory, visual, and auditory. Evidence suggests altered eating behavior in obesity. Nonetheless, multisensory integration in obesity has been scantily investigated so far. Starting from this gap in the literature, we seek to provide the first comprehensive investigation of multisensory integration in obesity. Twenty male obese participants and twenty male healthy-weight participants took part in the study aimed at describing the multisensory temporal binding window (TBW). The TBW is defined as the range of stimulus onset asynchrony in which multiple sensory inputs have a high probability of being integrated. To investigate possible multisensory temporal processing deficits in obesity, we investigated performance in two multisensory audiovisual temporal tasks, namely simultaneity judgment and temporal order judgment. Results showed a wider TBW in obese participants as compared to healthy-weight controls. This holds true for both the simultaneity judgment and the temporal order judgment tasks. An explanatory hypothesis would regard the effect of metabolic alterations and low-grade inflammatory state, clinically observed in obesity, on the temporal organization of brain ongoing activity, which one of the neural mechanisms enabling multisensory integration.

Rheotaxis of Larval Zebrafish: Behavioral Study of a Multi-Sensory Process.

Awake animals unceasingly perceive sensory inputs with great variability of nature and intensity, and understanding how the nervous system manages this continuous flow of diverse information to get a coherent representation of the environment is arguably a central question in systems neuroscience. Rheotaxis, the ability shared by most aquatic species to orient toward a current and swim to hold position, is an innate and robust multi-sensory behavior that is known to involve the lateral line and visual systems. To facilitate the neuroethological study of rheotaxic behavior in larval zebrafish we developed an assay for freely swimming larvae that allows for high experimental throughtput, large statistic and a fine description of the behavior. We show that there exist a clear transition from exploration to counterflow swim, and by changing the sensory modalities accessible to the fishes (visual only, lateral line only or both) and comparing the swim patterns at different ages we were able to detect and characterize two different mechanisms for position holding, one mediated by the lateral line and one mediated by the visual system. We also found that when both sensory modalities are accessible the visual system overshadows the lateral line, suggesting that at the larval stage the sensory inputs are not merged to finely tune the behavior but that redundant information pathways may be used as functional fallbacks.

The lateral line is necessary for blind cavefish rheotaxis in non-uniform flow.

When encountering a unidirectional flow, many fish exhibit an unconditioned orienting response known as rheotaxis. This multisensory behavior can reportedly involve visual, vestibular, tactile and lateral line cues. However, the precise circumstances under which different senses contribute are still unclear and there is considerable debate, in particular, about the contributions of the lateral line. In this study, we investigate the rheotactic behavior of blind cavefish under conditions of spatially non-uniform flow (a jet stream), which in theory, should promote reliance on lateral line cues. The behavior of individual lateral line enabled and disabled fish was videorecorded under IR light in a square arena that prevented streamwise biases and that contained a narrow jet stream in the center of the tank. Whereas the stream's peak velocity (8 cm s(-1)) declined very little in the streamwise direction, it declined steeply in the cross-stream direction (∼3-4.5 cm s(-1)cm(-1)). Lateral line enabled fish showed higher levels of orientation to the stream and its source (a 1-cm-wide nozzle) when in the central (jet stream) region of the tank compared with surrounding regions, whereas lateral line disabled fish showed random orientations in all regions of the tank. The results of this study indicate that the spatial characteristics of flow play a role in determining the sensory basis of rheotaxis.

Cortical EEG components that reflect inverse effectiveness during visuotactile integration processing

Multisensory integration is required to interpret information from multiple senses and then produce the appropriate behavioral output. Inverse effectiveness, a phenomenon in which weaker stimuli induce greater activation of multisensory neurons than do stronger stimuli, is an essential component of multisensory integration. The superior colliculus is especially abundant in multisensory neurons with properties of inverse effectiveness. However, multisensory neurons are distributed throughout the brain, suggesting that a wide range of brain regions are involved in generating multisensory behavior in addition to the superior colliculus. In this study, we aimed to use scalp EEG to elucidate the cortical responses that respond to multisensory stimuli according to the principle of inverse effectiveness. By modulating the intensity of the tactile aspect of a simultaneous visuotactile stimulus, we explored the time–frequency component of scalp EEG waveforms produced during multi-sensory stimulation. Using this method, we determined the relative values and temporal dynamics of the increment of power spectrum (event-related spectrum perturbation) and phase coherence across trials (e.g., inter-trial phase coherency) produced by multisensory stimulation compared to unisensory stimulation. In the somatosensory and anterior cingulate cortices, we observed significant differences in the inter-trial phase coherence of the theta band oscillation 200–400ms post-stimulus, in response to visuotactile (multisensory) and tactile (unisensory) stimulation, while no differences were found in later time windows and other cortical areas. These results suggest that inverse effectiveness is an important aspect of multi-sensory processing in the somato-sensory and frontal cerebral cortices.

Brief cortical deactivation early in life has long-lasting effects on multisensory behavior.

Detecting and locating environmental events are markedly enhanced by the midbrain's ability to integrate visual and auditory cues. Its capacity for multisensory integration develops in cats 1-4 months after birth but only after acquiring extensive visual-auditory experience. However, briefly deactivating specific regions of association cortex during this period induced long-term disruption of this maturational process, such that even 1 year later animals were unable to integrate visual and auditory cues to enhance their behavioral performance. The data from this animal model reveal a window of sensitivity within which association cortex mediates the encoding of cross-modal experience in the midbrain. Surprisingly, however, 3 years later, and without any additional intervention, the capacity appeared fully developed. This suggests that, although sensitivity degrades with age, the potential for acquiring or modifying multisensory integration capabilities extends well into adulthood.

Young Children’s Engagement With E-Books at School

Differences between digital devices on children’s engagement with e-books are examined. The sample included 24 typical 4-year olds enrolled in Head Start. Over a 1-month period, video captures of children’s multi-sensory behaviors during shared reading at a tabletop touchscreen computer and teacher-facilitated book browsing with iPads and iPods were obtained. Data were coded on each child at 1-min intervals, examining the simultaneity of behaviors present, then aggregated to determine frequencies of each behavior by device and format. Differences between media devices on median percent of observation time were evaluated. Looking, touching, moving, and gesturing behaviors were significantly different among different devices. Large effect sizes indicated considerable variability attributable to the device. Mobiles support more looking and touching but less moving and gesturing than the tabletop touchscreen; none of the devices favored listening over another. Given the role of haptic perception in digital reading experience, access to mobiles may favor behaviors that support literacy motivation, sense of control, and interaction.

Sedentary behavior as a factor in determining lateral line contributions to rheotaxis.

Rheotaxis is a robust, multisensory behavior with many potential benefits for fish and other aquatic animals. Visual (optic flow) cues appear to be sufficient for rheotaxis, but other sensory cues can clearly compensate for the loss of vision. The role of various non-visual sensory systems, in particular the flow-sensing lateral line, is poorly understood, largely because of widely varying methods and sensory conditions for studying rheotaxis. Here, we examine how sedentary behavior under visually deprived conditions affects the relative importance of lateral line cues in two species: one that is normally sedentary (the three-lined corydoras, Corydoras trilineatus) and one that normally swims continuously along the substrate (the blind cavefish, Astyanax mexicanus). No effect of lateral line disruption on rheotactic performance was found in blind cavefish, which were significantly more mobile than three-lined corydoras. By contrast, rheotaxis was significantly impaired at low, but not high, flow speeds in lateral-line-disabled corydoras. In addition, lateral-line-enabled corydoras were characterized by decreased mobility and increased rheotactic performance relative to lateral-line-disabled fish. Taken together, these results suggest that sedentary behavior is an important factor in promoting reliance on lateral line cues.