Cross-modal Research Articles

Thalamocortical Sensorimotor Circuit Damage Associated with Disorders of Consciousness for Diffuse Axonal Injury Patients

The relationship of structural and functional brain damage and disorders of consciousness (DOC) for diffuse axonal injury (DAI) is still not fully explored. We employed diffusion tensor imaging (DTI) and resting-state fMRI (RS-fMRI) to examine the changes of resting activations and white matter (WM) integrity for DAI with DOC. WM damages were observed in the body and genu of the corpus callosum, right external capsule (EC) and superior corona radiate (SCR), left superior cerebellar peduncle (SCP) and posterior thalamic radiation (PTR). The RS-fMRI revealed augmented amplitude of low-frequency fluctuation (ALFF) in the anterior cingulate cortex, hippocampus, insula, amygdala and putamen, and reduced ALFF in the precuneus, thalamus, pre-central and post-central gyri. Correlation analysis identified positive associations between the Glasgow Coma Scale (GCS) and activation of the precuneus and between GCS and DTI measurements in the left PTR and SCP, but a negative correlation was found between GCS and activation of the thalamus. Cross modality association analyses indicated that activations of the amygdala and postcentral gyrus were correlated with DTI measurements of the right EC and left PTR respectively. These results implicate that the WM damages in thalamocortical sensorimotor circuit and aberrant brain activity responding to self-awareness and sensation are critical factors to DOC, which expand the current understanding of the neural mechanisms underlying DAI.

Neuroanatomical correlates of cross-modal transfer performance in object categorization: from vision to touch

Neuroanatomical correlates of cross-modal transfer performance in object categorization: from vision to touch

Stochastic Resonance in Time-to-Contact Judgments

Stochastic resonance (SR) is a counterintuitive phenomenon in which additive noise enhances performance of a nonlinear system. Previous studies demonstrated SR effect on human tactile sensitivity by adding noise of same modality and cross modality. Similarly, enhancement of human hearing through additive noise has been studied. In this study, we investigate the effect of noise in visual perception, specifically time-to-contact (TTC) judgments. This study explores four research questions: (1) Does noise help in TTC judgments? (2) How does noise affect speed and accuracy of TTC judgments? (3) Does cross modal noise help in TTC judgments? (4) How does cross modal noise affect speed and accuracy of TTC judgments? Through simulation, we show that noise in optical cue can enhance weak signals. We also demonstrate that noise can improve speed of TTC judgments at the expense of accuracy. Similarly, we demonstrate SR by adding noise of cross modality. These findings provide plausible hypotheses regarding how much noise should be added to enhance TTC judgments.

嗅覚の非定常応答に関する基礎的研究 (その15）音の大きさと線分長を用いたクロスモダリティーマッチングに対する嗅覚反応モデルの適用

嗅覚の非定常応答に関する基礎的研究 (その15）音の大きさと線分長を用いたクロスモダリティーマッチングに対する嗅覚反応モデルの適用

Combined Evaluation of FDG-PET/CT and CT Imaging Characteristics of Cervical Lymph Nodes to Increase the Interpretation Accuracy for Nodal Metastatic Involvement in Head and Neck Cancer

Rationale and objectives: Accurate staging of regional lymph nodes in patient with head and neck squamous cell carcinoma (HNSCC) has important therapeutic and prognostic implications. The objective of this study was to assess qualitative and quantitative imaging characteristics of cervical lymph nodes on FDG PET/CT and contrast enhanced CT (CECT), either alone or in cross-modality combinations, for more accurate diagnosis of nodal metastatic involvement in HNSCC patients who had hypermetabolic cervical lymph nodes on PET/CT imaging. Materials and methods: Lymph nodes were evaluated using quantitative PET/CT parameters (SUVmax, SUVmean, metabolic tumor volume, total lesion glycolysis) and qualitative and quantitative CT parameters (shape, margin, focal hypoattenuation, fatty hilum, short axis, long axis, volume, eccentricity factor). A subset analysis was performed on FDG-avid lymph nodes that lacked imaging findings suggestive of extracapsular spread or necrosis. Statistical outcomes included sensitivity, specificity, accuracy, positive likelihood ratio, and odds ratio. Results: In the full cohort (174 nodes), the most malignancy specific parameters were ill-defined margin and focal hypoattenuation, and the most accurate variables included SUVmax, SUVmean, ill-defined margin, short axis, and focal hypoattenuation. On multivariate analysis, ill-defined margin, short axis measurement, and SUVmax independently predicted nodal metastasis. In the subgroup of smaller lymph nodes without ill-defined margin or focal hypoattenuation (100 nodes), round shape was among the most accurate and specific criteria for malignancy. Cross-modality combination criteria with SUVmax and shape, eccentricity factor, or long to short axis ratio resulted in statistically significant improvements in specificity and accuracy for lymph node metastasis. Conclusion: A combined evaluation strategy using individual PET/CT and CECT characteristics in cross modality combinations, particularly SUVmax with round shape on axial imaging or with loss of eccentricity, can improve diagnostic accuracy of malignant nodal involvement in the evaluation of indeterminate cervical lymph nodes for malignancy.

Dominance of apparent motion in binocular rivalry is modulated by crossmodal synchrony

Dominance of apparent motion in binocular rivalry is modulated by crossmodal synchrony

Cross-modal comparison of seed-based structural and functional covariance

Cross-modal comparison of seed-based structural and functional covariance

Emotional mediation of cross-modal associations in audioVisual synesthesia

Emotional mediation of cross-modal associations in audioVisual synesthesia

Simplification of olfactory stimuli in pseudo-gustatory displays.

In this paper, we propose a technique that simplifies pseudo-gustatory systems by using the cross-modal effect between vision and olfaction to change only the visual and olfactory stimuli without altering the ingredients of the food. Conventional pseudo-gustatory simulations require one olfactory stimulus for each flavor. In contrast, we hypothesize that the cross-modal effect between vision and olfaction in a pseudo-gustatory presentation can be utilized to reduce the required number of olfactory stimuli, and verify this hypothesis using a pseudo-gustatory display with our proposed method. This pseudo-gustatory display uses the visual-olfactory cross-modal effect and the key scent components decided on the basis of similarity analysis of olfactory perception. We also investigate how users perceive the taste of a drink under various visual and olfactory conditions, both with and without the visual-olfactory interactions. Our results indicate that the cross-modal effect between vision and olfaction can effectively simplify pseudo-gustatory simulations.

Molyneux's Question: A Window on Crossmodal Interplay in Blindness

Abstract: In 1688, the Irish scientist William Molyneux sent a letter to the philosopher John Locke in which he asked whether a man who had been born blind and whose experience of the world was based on senses other than vision, would be able to distinguish and name a globe and a cube by sight alone, once he had been enabled to see. This issue immediately raised considerable interest amongst philosophers, who, for centuries, have continued to speculate about how this issue might be resolved. More recently, the possibility of corrective surgery for people born with congenital cataracts, has offered a valuable opportunity to explore this topic experimentally. A discussion of how Molyneux’s question has been addressed over the centuries, allows us to investigate a number of intellectual challenges, involving a variety of fields, ranging from philosophy, psychology and the cognitive sciences, to neuroscience. For instance, Molyenux’s question raises the question of whether sensory experience is specific to each sensory modality, or rather supramodal, and if a transfer of knowledge across modalities can be established. Molyneux’s question is also relevant to the discussion of whether, how and to what extent a concept of space can be developed by blind people. More generally, it concerns the idea of how conceptual knowledge is acquired and processed when vision is lacking. The present paper aims to provide an overview of all the issues raised by Molyenux’s question. Keywords: Molyneux; Multisensory; Blindness; Nativism; Empiricism. Il problema di Molyneux: una finestra sull’interazione intermodale nella cecita Riassunto : Nel 1688 lo scienziato irlandese William Molyneux invio una lettera al filosofo John Locke in cui chiedeva se un uomo nato cieco e la cui esperienza del mondo e basata su sensi non visivi, avrebbe potuto discriminare e nominare una sfera e un cubo semplicemente guardandoli, una volta acquistata la vista. La questione desto immediatamente grande interesse tra i filosofi, i quali per secoli avevano cercato di trovare una soluzione speculativa a questo problema. In tempi piu recenti, la possibilita di sottoporre a un intervento individui nati con cataratta congenita ha offerto un’occasione propizia per indagare il tema in via sperimentale. Discutere il modo in cui il problema di Molyeux e stato affrontato nei secoli, permette di considerare un gran numero di questioni intellettuali che abbracciano campi diversi, dalla filosofia, alle neuroscienze, passando per la psicologia e le scienze cognitive. Per esempio, il problema di Molyenux investe la questione se l’esperienza sensibile sia specifica per ogni modalita sensibile o se, invece, sia supermodale, e se si possa dare un passaggio di conoscenza da una modalita all’altra. Il problema di Molyneux investe anche come e fino e che punto il concetto di spazio possa essere sviluppato da persone non vedenti. Piu in generale, riguarda l’idea di come la conoscenza concettuale e acquisita e processata quando la vista manca. Questo articolo intende fornire una visione d’insieme che abbraccia i diversi temi legati al problema di Molyenux. Parole chiave: Molyneux; Multisensorialita; Cecita; Innatismo; Empirismo.

Visualization of sounds: Crossmodal transfer of affective information

Visualization of sounds: Crossmodal transfer of affective information

Visual, Auditory and Cross Modal Lexical Decision: A Comparison between Dyslexic and Typical Readers

This study examined the differences in processing between typical and dyslexic readers in a lexical decision task in different modalities (visual, auditory, and cross modality). In general, dyslexics exhibited slower reaction times and made more mistakes when performing the task. The biggest gap between the two groups of readers was found on the visual task. However, the dyslexics benefited from the cross modality presentation, with differences between the two groups minimized. The results of this study indicate that the slowness characteristic of dyslexics’ stems mainly from slow processing of visual linguistic information and that supporting both processing systems relevant to reading will assist inter-sensory integration and may ease the word decoding process among dyslexics.

A segmentation guide and probabilistic atlas of the C57BL/6J mouse brain from magnetic resonance imaging

A segmentation guide and probabilistic atlas of the C57BL/6J mouse brain from magnetic resonance imaging

An Approach for the Cross Modality Content-Based Image Retrieval between Different Image Modalities

CBIR is an effective tool to search and extract image contents in a large remote sensing image database queried by an operator or end user. However, as imaging principles are different by sensors, their visual representation thus varies among image modality type. Considering images of various modalities archived in the database, image modality difference has to be tackled for the successful CBIR implementation. However, this topic has been seldom dealt with and thus still poses a practical challenge. This study suggests a cross modality CBIR (termed as the CM-CBIR) method that transforms given query feature vector by a supervised procedure in order to link between modalities. This procedure leverages the skill of analyst in training steps after which the transformed query vector is created for the use of searching in target images with different modalities. Current initial results show the potential of the proposed CM-CBIR method by delivering the image content of interest from different modality images. Despite its retrieval capability is outperformed by that of same modality CBIR (abbreviated as the SM-CBIR), the lack of retrieval performance can be compensated by employing the user's relevancy feedback, a conventional technique for retrieval enhancement.

Unconscious imitation as a metric for crossmodal influences: Visibility of the mouth can enhance alignment to audiovisual speech.

Unconscious imitation as a metric for crossmodal influences: Visibility of the mouth can enhance alignment to audiovisual speech.

Violins are Green, Pianos are Blue: Cross-modal Sound-to-Sight Associations with Timbre in Synesthetes & Non-Synesthetes

Violins are Green, Pianos are Blue: Cross-modal Sound-to-Sight Associations with Timbre in Synesthetes & Non-Synesthetes

The Cross Modality Asymmetrical Effect of Stimulus-driven Capture in Audiovisual Additional Singleton Search Task

The Cross Modality Asymmetrical Effect of Stimulus-driven Capture in Audiovisual Additional Singleton Search Task

The function of consciousness in multisensory integration

The function of consciousness was explored in two contexts of audio–visual speech, cross-modal visual attention guidance and McGurk cross-modal integration. Experiments 1, 2, and 3 utilized a novel cueing paradigm in which two different flash suppressed lip-streams cooccured with speech sounds matching one of these streams. A visual target was then presented at either the audio-visually congruent or incongruent location. Target recognition differed for the congruent versus incongruent trials, and the nature of this difference depended on the probabilities of a target appearing at these respective locations. Thus, even though the lip-streams were never consciously perceived, they were nevertheless meaningfully integrated with the consciously perceived sounds, and participants learned to guide their attention according to statistical regularities between targets and these unconsciously perceived cross-modal cues. In Experiment 4, McGurk stimuli were presented in which the lip-streams were either flash suppressed (4a) or unsuppressed (4b), and the McGurk effect was found to vanish under conditions of flash suppression. Overall, these results suggest a simple yet fundamental principle regarding the function of consciousness in multisensory integration – cross-modal effects can occur in the absence of consciousness, but the influencing modality must be consciously perceived for its information to cross modalities.

A cross-modal, cross-species comparison of connectivity analyses in the primate cortex

A cross-modal, cross-species comparison of connectivity analyses in the primate cortex

Visual mismatch negativity elicited by facial expressions: new evidence from the equiprobable paradigm

BackgroundConverging evidence revealed that facial expressions are processed automatically. Recently, there is evidence that facial expressions might elicit the visual mismatch negativity (MMN), expression MMN (EMMN), reflecting that facial expression could be processed under non-attentional condition. In the present study, using a cross modality task we attempted to investigate whether there is a memory-comparison-based EMMN.Methods12 normal adults were instructed to simultaneously listen to a story and pay attention to a non-patterned white circle as a visual target interspersed among face stimuli. In the oddball block, the sad face was the deviant with a probability of 20% and the neutral face was the standard with a probability of 80%; in the control block, the identical sad face was presented with other four kinds of face stimuli with equal probability (20% for each). Electroencephalogram (EEG) was continuously recorded and ERPs (event-related potentials) in response to each kind of face stimuli were obtained. Oddball-EMMN in the oddball block was obtained by subtracting the ERPs elicited by the neutral faces (standard) from those by the sad faces (deviant), while controlled-EMMN was obtained by subtracting the ERPs elicited by the sad faces in the control block from those by the sad faces in the oddball block. Both EMMNs were measured and analyzed by ANOVAs (Analysis of Variance) with repeated measurements. sLORETA (standardized low-resolution brain electromagnetic tomography) was used to investigate the cortical generators of controlled-EMMN.ResultsBoth the oddball-EMMN in deviant-standard comparison and the controlled-EMMN in deviant-control comparison were observed at occipital-temporal regions with right hemisphere predominance. The oddball-EMMN was bigger and earlier than the controlled-EMMN because, besides the memory-based comparison, the former included a difference of refractoriness due to the distinction of presented probability between the deviant and standard face stimuli. The source analysis of controlled-EMMN indicated a current source primarily involved in posterior areas including superior temporal gyrus, postcentral gyrus, inferior parietal lobule as well as the insula.ConclusionsThe valid EMMN properly reflecting the memory-based comparison of facial expressions could be obtained, i.e., the controlled-EMMN.