Bimodal Neurons Research Articles

Contributed Session III: Identifying Specific Neural Substrates for Bayesian-like Computations in Binocular Vision and Multisensory Processing.

Bayesian thinking is influential in vision but the grounding of Bayesian computation in wetware is poorly understood. Bayesian reliability (inverse variance) weighting of inputs is predicted by Maximum Likelihood Estimation Theory and has some psychophysical support, but evidence for neural reliability weighting is sparse and neural modeling of reliability weighting is tricky. However, reliability averaging is just one possible perceptual weighted average. An alternative - nonlinear magnitude-weighted averaging - was suggested by Schrodinger in 1926 to account for suprathreshold binocular perception and is available to repurpose for other sensory cue combinations. We identified macaque suppressive binocular neurons that implement nonlinear magnitude-weighted averaging and approximate Bayesian averaging, without suffering the computational difficulties that Bayesian averaging implies. We then applied the binocular modeling to suppressive multisensory (visual-tactile, audio-tactile, and audio-visual) neurons. Although magnitude-weighting is a better fit than reliability-weighted averaging for cortical firing rates (in all four cases and in three different species), nonlinear magnitude-weighted averaging is well correlated with reliability averaging. Magnitude-weighted averaging could serve as a surrogate for Bayesian calculations; mildly suppressive binocular and multisensory bimodal neurons could be neural correlates of Bayesian-like computation in the brain.

A novel microseismic classification model based on bimodal neurons in an artificial neural network

Microseismic monitoring systems deployed in deep underground engineering can capture massive waveform signals in real time. However, some noise signals are highly deceptive and similar to microfracture signals. This problem usually requires engineers to compare the signal characteristics from different domains and poses a challenge for the rapid detection of microseismic data. To address this problem, we establish a new waveform multiclassification model based on the multimodal feature extraction technology. The proposed model constructs a dual-branch convolutional neural network to learn the bimodal features from the time and frequency domains. Furthermore, the attention mechanism (channel and spatial attention) with skip connection is used to increase the focus on the prominent features of the waveform and reduce the interference caused by irrelevant information. We conduct experimental studies using microseismic data from deep tunnel projects under complex geological conditions and classify the recorded data into three target types: microseismic, blast, and noise waveforms. The results show that the proposed approach achieves good multiclassification performance for noise signals highly similar to microseismic signals as well as for noisy microseismic signals with different signal-to-noise ratios. In summary, this study lays the foundation for further exploration of the potential application of multimodal fusion in seismic engineering and various rock engineering fields.

Understanding the Concept of Biomimetics for Geoinformatics Technology and its utilization for Nature-based Solutions

This study aims to introduce a wide range of readers to some of animals' exceptional remote sensing and navigation abilities. A comparison of remote sensors employed by animals and those built by humans is presented in this research. Thermal infrared sensors used by snakes, echolocation used by bats and dolphins, and navigation systems used by birds are all part of the research and comparison. Prey countermeasures to prevent capture are also taken into account. Remote sensing and navigation capabilities of certain animals are far superior to those provided by the human body or devised by humans. The use of the biometric method in creating novel sensors may be promoted. The paper contributes to a better understanding of animal behavior, particularly their unique abilities to perceive, echolocate, and travel over long distances with great accuracy. Studying animal sensors within a biomimetic framework can help remote sensor designers improve their designs for the purpose of border management, forest fire control (for example, the photo mechanic infrared receptor for detecting forest fires in the beetle), jamming the signal from the tiger moth which clogs bat sonar, integration of visual and infrared information in bimodal neurons in the rattlesnake optic tectum, and biomimetic sonar that locates and recognizes objects.

A simple vector-like law for perceptual information combination is also followed by a class of cortical multisensory bimodal neurons

SummaryAn interdisciplinary approach to sensory information combination shows a correspondence between perceptual and neural measures of nonlinear multisensory integration. In psychophysics, sensory information combinations are often characterized by the Minkowski formula, but the neural substrates of many psychophysical multisensory interactions are unknown. We show that audiovisual interactions – for both psychophysical detection threshold data and cortical bimodal neurons – obey similar vector-like Minkowski models, suggesting that cortical bimodal neurons could underlie multisensory perceptual sensitivity. An alternative Bayesian model is not a good predictor of cortical bimodal response. In contrast to cortex, audiovisual data from superior colliculus resembles the ‘City-Block’ combination rule used in perceptual similarity metrics. Previous work found a simple power law amplification rule is followed for perceptual appearance measures and by cortical subthreshold multisensory neurons. The two most studied neural cell classes in cortical multisensory interactions may provide neural substrates for two important perceptual modes: appearance-based and performance-based perception.

Neuronal Encoding of Multisensory Motion Features in the Rat Associative Parietal Cortex.

Motion perception is facilitated by the interplay of various sensory channels. In rodents, the cortical areas involved in multisensory motion coding remain to be identified. Using voltage-sensitive-dye imaging, we revealed a visuo-tactile convergent region that anatomically corresponds to the associative parietal cortex (APC). Single unit responses to moving visual gratings or whiskers deflections revealed a specific coding of motion characteristics strikingly found in both sensory modalities. The heteromodality of this region was further supported by a large proportion of bimodal neurons and by a classification procedure revealing that APC carries information about motion features, sensory origin and multisensory direction-congruency. Altogether, the results point to a central role of APC in multisensory integration for motion perception.

Differential responses of neurons in the rat caudal ventrolateral medulla to visceral and somatic noxious stimuli and their alterations in colitis

Visceral and somatic types of pain have been reported to manifest crucial differences not only in the experience, but also in their peripheral and central processing. However, the precise neuronal mechanisms that responsible for the modality-specific transmission of pain signals, especially at the supraspinal level, remain unclear. Very little is known also about the potential involvement of such mechanisms in the development of viscero-somatic hyperalgesia. Therefore, in the present study performed on urethane-anesthetized adult male Wistar rats we examined responses of neurons in the caudal ventrolateral medulla (CVLM)—the first site for supraspinal processing of both internal and external pain signals—to visceral (colorectal distension, CRD) and somatic (squeezing of the tail) noxious stimulations and evaluated alterations in response properties of these cells after the induction of colitis. It has been found out that the CVLM of healthy control rats, along with harboring of cells excited by both stimulations (23.7%), contained neurons that were activated by either visceral (31.9%) or somatic noxious stimuli (44.4%). In inflamed animals, the percentages of the visceral and somatic nociceptive cells were decreased (to 18.3% and 34.3%, correspondingly) and the number of bimodal neurons was increased (up to 47.4%); these alterations were associated with substantially enhanced responses of both the modality-specific and convergent CVLM neurons not only to CRD, but also to squeezing of the tail. Under these conditions, visceral and somatic pain stimuli induced similar changes in arterial blood pressure and respiratory rate, whereas in the absence of intestinal inflammation noxious CRD and tail stimulation evoked predominantly divergent autonomic reactions. The data obtained can benefit to a deeper understanding of the neuronal mechanisms that underlie differential supraspinal processing of visceral and somatic noxious stimuli and can potentially contribute to the realization of specific cardiovascular and respiratory accompaniments inherent to a particular type of pain. Therewith, results of the study elucidate colitis-induced alterations in these mechanisms, which may be responsible for the combined development of visceral hypersensitivity and somatic hyperalgesia.

Spatial Cues Influence Time Estimations in Deaf Individuals.

SummaryRecent studies have reported a strong interaction between spatial and temporal representation when visual experience is missing: blind people use temporal representation of events to represent spatial metrics. Given the superiority of audition on time perception, we hypothesized that when audition is not available complex temporal representations could be impaired, and spatial representation of events could be used to build temporal metrics. To test this hypothesis, deaf and hearing subjects were tested with a visual temporal task where conflicting and not conflicting spatiotemporal information was delivered. As predicted, we observed a strong deficit of deaf participants when only temporal cues were useful and space was uninformative with respect to time. However, the deficit disappeared when coherent spatiotemporal cues were presented and increased for conflicting spatiotemporal stimuli. These results highlight that spatial cues influence time estimations in deaf participants, suggesting that deaf individuals use spatial information to infer temporal environmental coordinates.

Heteromodal Motion Coding in the Associative Parietal Cortex in Rats

Motion perception is facilitated by the interplay of various sensory channels. In rodents, the cortical areas involved in multisensory motion coding remains to be identified. Using voltage-sensitive-dye imaging, we revealed a visuo-tactile convergent region which anatomically corresponds to the associative parietal cortex (APC). Our electrophysiological unit recordings provide the first single-unit evidence of direction and speed tuning of APC visual neurons during moving visual gratings which suggest the APC as a motion processing area. Strikingly, using whisker deflections, we also found somatosensory neurons exhibiting selectivity to motion characteristics. Using a classification procedure, we revealed that APC population carries information about motion features in both sensory modalities. The presence of a large proportion of bimodal neurons underpins potential integrative functions of APC and suggests its role in the elaboration of a multimodal motion percept.

Hand function, not proximity, biases visuotactile integration later in object processing: An ERP study.

Behavioral studies document a functional hand proximity effect: objects near the palm, but not the back of the hand, affect visual processing. Although visuotactile bimodal neurons integrate visual and haptic inputs, their receptive fields in monkey cortex encompass the whole hand, not just the palm. Using ERPs, we investigated whether hand function influenced the topology of integrated space around the hand. In a visual detection paradigm, target and non-target stimuli appeared equidistantly in front or in back of the hand. Equivalent N1 amplitudes were found for both conditions. P3 target versus non-target amplitude differences were greater for palm conditions. Hand proximity biased processing of visual targets equidistant from the hand early in processing. However, hand function biases emerged later when targets were selected for potential action. Thus, early hand proximity effects on object processing depend on sensory-reliant neural responses, whereas later multisensory integration depend more on the hand's functional expertise.

Do the Different Sensory Areas Within the Cat Anterior Ectosylvian Sulcal Cortex Collectively Represent a Network Multisensory Hub?

Current theory supports that the numerous functional areas of the cerebral cortex are organized and function as a network. Using connectional databases and computational approaches, the cerebral network has been demonstrated to exhibit a hierarchical structure composed of areas, clusters and, ultimately, hubs. Hubs are highly connected, higher-order regions that also facilitate communication between different sensory modalities. One region computationally identified network hub is the visual area of the Anterior Ectosylvian Sulcal cortex (AESc) of the cat. The Anterior Ectosylvian Visual area (AEV) is but one component of the AESc that also includes the auditory (Field of the Anterior Ectosylvian Sulcus - FAES) and somatosensory (Fourth somatosensory representation - SIV). To better understand the nature of cortical network hubs, the present report reviews the biological features of the AESc. Within the AESc, each area has extensive external cortical connections as well as among one another. Each of these core representations is separated by a transition zone characterized by bimodal neurons that share sensory properties of both adjoining core areas. Finally, core and transition zones are underlain by a continuous sheet of layer 5 neurons that project to common output structures. Altogether, these shared properties suggest that the collective AESc region represents a multiple sensory/multisensory cortical network hub. Ultimately, such an interconnected, composite structure adds complexity and biological detail to the understanding of cortical network hubs and their function in cortical processing.

Processing of Intraoral Olfactory and Gustatory Signals in the Gustatory Cortex of Awake Rats.

Food perception and choice depend upon the concurrent processing of olfactory and gustatory signals from the mouth. The primary gustatory cortex has been proposed to integrate chemosensory stimuli; however, no study has examined the single-unit responses to intraoral odorant presentation. Here we found that neurons in gustatory cortex can respond either exclusively to tastants, exclusively to odorants, or to both (bimodal). Several differences exist between these groups' responses; notably, bimodal neurons code palatability significantly better than unimodal neurons. This group of neurons might represent a substrate for how odorants gain the quality of tastants.

Approaching threat modulates visuotactile interactions in peripersonal space

The region surrounding our body (i.e. peripersonal space) is coded in a multimodal representation by fronto-parietal bimodal neurons integrating tactile stimuli on the body with nearby visual stimuli. This has often been suggested to serve a defensive purpose, which we propose could be mediated through visuotactile predictions. An approaching threat would then be of particular interest to peripersonal space processing. To investigate this, we asked participants to respond as fast as possible to a tactile stimulus on the hand, while looking at an animation of an approaching or receding spider or butterfly. Tactile stimulation was applied at one of 25 possible time points during the animation. Tactile reaction times were faster when an approaching stimulus was closer to the hand at the time of tactile presentation. Critically, this effect of distance on reaction times was larger when participants saw an approaching spider compared to an approaching butterfly, but only for participants who were afraid of spiders. This finding demonstrates that the perceived threat of an approaching stimulus modulates visuotactile interactions in peripersonal space and is consistent with the idea that visuotactile predictions are important for defensive purposes and maintaining bodily integrity.

Intentional control of visual processing benefits from referential objects.

Arguments have been made that enhanced visual processing occurs in the area of the palms of the hands due to greater density of bimodal neurons. An alternative is that the hands serve as reference objects relative to which attentional resources are allocated. Two experiments were conducted to determine whether the palms are unique in speeding responses in an Eriksen flanker-type task compared with other parts of the hands and objects used as barriers. In Experiment 1, the hands were crossed and positioned so that the palms faced outward toward letters located in the outer positions. Trial blocks differed in whether the centrally located letter or outer letters were designated as the target for responding. Results yielded reductions in flanker interference much as obtained when the palms face inward. This reduction occurred regardless of whether the center or outer positions of the letters were designated as the target. Experiment 2 replicated these results using as reference objects wooden blocks that mimicked the hands' physical contours, positioned with a curve-edge facing outwards. The results lend support to the referential coding account of the reduction of flanker interference.

Crossmodal adaptation in right posterior superior temporal sulcus during face-voice emotional integration.

The integration of emotional information from the face and voice of other persons is known to be mediated by a number of "multisensory" cerebral regions, such as the right posterior superior temporal sulcus (pSTS). However, whether multimodal integration in these regions is attributable to interleaved populations of unisensory neurons responding to face or voice or rather by multimodal neurons receiving input from the two modalities is not fully clear. Here, we examine this question using functional magnetic resonance adaptation and dynamic audiovisual stimuli in which emotional information was manipulated parametrically and independently in the face and voice via morphing between angry and happy expressions. Healthy human adult subjects were scanned while performing a happy/angry emotion categorization task on a series of such stimuli included in a fast event-related, continuous carryover design. Subjects integrated both face and voice information when categorizing emotion-although there was a greater weighting of face information-and showed behavioral adaptation effects both within and across modality. Adaptation also occurred at the neural level: in addition to modality-specific adaptation in visual and auditory cortices, we observed for the first time a crossmodal adaptation effect. Specifically, fMRI signal in the right pSTS was reduced in response to a stimulus in which facial emotion was similar to the vocal emotion of the preceding stimulus. These results suggest that the integration of emotional information from face and voice in the pSTS involves a detectable proportion of bimodal neurons that combine inputs from visual and auditory cortices.

The cortical distribution of multisensory neurons was modulated by multisensory experience

Previous studies have indicated a sparse distribution of multisensory neurons in the transition zones between cortical areas associated with specific sensory modalities. However, little is known about the distribution and functional properties of such neurons. The bimodal visual–auditory neurons in the transition area between visual and auditory cortices in rats were examined to determine whether these neurons are modulated by simultaneous input from visual and auditory modalities. Visual–auditory neurons were found to have a non-uniform distribution within this region, instead gathering together and forming a small zone. Electrophysiological recordings revealed that visual–auditory neurons possess integrative characteristics similar to neurons of the superior colliculus, a midbrain structure in the visual pathway. Exposing adult animals to combined visual and auditory stimuli resulted in an expansion of bimodal neuron distribution in the visual–auditory transition area. These effects were more pronounced in young animals; in this case, the distribution of visual–auditory neurons extended past the limits of the transition area and invaded the flanking modality-specific cortical areas. These results provide a direct demonstration of the role of sensory experience in shaping cortical structure, which can have implications for neuronal integration and cognitive function.

Multisensory integration in early vestibular processing in mice: the encoding of passive vs. active motion

The mouse has become an important model system for studying the cellular basis of learning and coding of heading by the vestibular system. Here we recorded from single neurons in the vestibular nuclei to understand how vestibular pathways encode self-motion under natural conditions, during which proprioceptive and motor-related signals as well as vestibular inputs provide feedback about an animal's movement through the world. We recorded neuronal responses in alert behaving mice focusing on a group of neurons, termed vestibular-only cells, that are known to control posture and project to higher-order centers. We found that the majority (70%, n = 21/30) of neurons were bimodal, in that they responded robustly to passive stimulation of proprioceptors as well as passive stimulation of the vestibular system. Additionally, the linear summation of a given neuron's vestibular and neck sensitivities predicted well its responses when both stimuli were applied simultaneously. In contrast, neuronal responses were suppressed when the same motion was actively generated, with the one striking exception that the activity of bimodal neurons similarly and robustly encoded head on body position in all conditions. Our results show that proprioceptive and motor-related signals are combined with vestibular information at the first central stage of vestibular processing in mice. We suggest that these results have important implications for understanding the multisensory integration underlying accurate postural control and the neural representation of directional heading in the head direction cell network of mice.

Visuospatial properties of caudal area 7b in Maca ca fascicularis

To proceed from sensation to movement, integration and transformation of information from different senses and reference frames are required. Several brain areas are involved in this transformation process, but previous neuroanatomical and neurophysiological studies have implicated the caudal area 7b as one particular component of this transformation system. In this study, we present the first quantitative report on the spatial coding properties of caudal area 7b. The results showed that neurons in this area had intermediate component characteristics in the transformation system; the area contained bimodal neurons, and neurons in this area encode spatial information using a hybrid reference frame. These results provide evidence that caudal area 7b may belong to the reference frame transformation system, thus contributing to our general understanding of the transformation system.

Multisensory Integration in Non-Human Primates during a Sensory-Motor Task

Daily our central nervous system receives inputs via several sensory modalities, processes them and integrates information in order to produce a suitable behavior. The amazing part is that such a multisensory integration brings all information into a unified percept. An approach to start investigating this property is to show that perception is better and faster when multimodal stimuli are used as compared to unimodal stimuli. This forms the first part of the present study conducted in a non-human primate’s model (n = 2) engaged in a detection sensory-motor task where visual and auditory stimuli were displayed individually or simultaneously. The measured parameters were the reaction time (RT) between stimulus and onset of arm movement, successes and errors percentages, as well as the evolution as a function of time of these parameters with training. As expected, RTs were shorter when the subjects were exposed to combined stimuli. The gains for both subjects were around 20 and 40 ms, as compared with the auditory and visual stimulus alone, respectively. Moreover the number of correct responses increased in response to bimodal stimuli. We interpreted such multisensory advantage through redundant signal effect which decreases perceptual ambiguity, increases speed of stimulus detection, and improves performance accuracy. The second part of the study presents single-unit recordings derived from the premotor cortex (PM) of the same subjects during the sensory-motor task. Response patterns to sensory/multisensory stimulation are documented and specific type proportions are reported. Characterization of bimodal neurons indicates a mechanism of audio-visual integration possibly through a decrease of inhibition. Nevertheless the neural processing leading to faster motor response from PM as a polysensory association cortical area remains still unclear.

Multisensory and unisensory neurons in ferret parietal cortex exhibit distinct functional properties

Despite the fact that unisensory and multisensory neurons are comingled in every neural structure in which they have been identified, no systematic comparison of their response features has been conducted. Towards that goal, the present study was designed to examine and compare measures of response magnitude, latency, duration and spontaneous activity in unisensory and bimodal neurons from the ferret parietal cortex. Using multichannel single-unit recording, bimodal neurons were observed to demonstrate significantly higher response levels and spontaneous discharge rates than did their unisensory counterparts. These results suggest that, rather than merely reflect different connectional arrangements, unisensory and multisensory neurons are likely to differ at the cellular level. Thus, it can no longer be assumed that the different populations of bimodal and unisensory neurons within a neural region respond similarly to a given external stimulus.

Corticospinal Facilitation during Observation of Graspable Objects: A Transcranial Magnetic Stimulation Study

In 1979, Gibson first advanced the idea that the sight of graspable objects automatically activates in the observer the repertoire of actions necessary to interact with them, even in the absence of any intention to act (“affordance effect”). The neurophysiological substrate of this effect was later identified in a class of bimodal neurons, the so-called "canonical" neurons, located within monkey premotor cortex. In humans, even if different behavioral studies supported the existence of affordance effect, neurophysiological investigations exploring its neural substrates showed contradictory results. Here, by means of Transcranial Magnetic Stimulation (TMS), we explored the time-course of the “affordance effect” elicited by the observation of everyday-life graspable objects on motor cortex of resting observers. We recorded motor evoked potentials (MEP) from three intrinsic hand muscles (two "synergic" for grasping, OP and FDI and one "neutral", ADM). We found that objects’ vision determined an increased excitability at 120 milliseconds after their presentation. Moreover, this modulation was proved to be specific to the cortical representations of synergic muscles. From an evolutionary perspective, this timing perfectly fits with a fast recruitment of the motor system aimed at rapidly and accurately choosing the appropriate motor plans in a competitive environment filled with different opportunities.