Modality-specific Brain Areas Research Articles

Different kinds of simulation during literary reading: Insights from a combined fMRI and eye-tracking study

Mental simulation is an important aspect of narrative reading. In a previous study, we found that gaze durations are differentially impacted by different kinds of mental simulation. Motor simulation, perceptual simulation, and mentalizing as elicited by literary short stories influenced eye movements in distinguishable ways (Mak & Willems, 2019). In the current study, we investigated the existence of a common neural locus for these different kinds of simulation. We additionally investigated whether individual differences during reading, as indexed by the eye movements, are reflected in domain-specific activations in the brain. We found a variety of brain areas activated by simulation-eliciting content, both modality-specific brain areas and a general simulation area. Individual variation in percent signal change in activated areas was related to measures of story appreciation as well as personal characteristics (i.e., transportability, perspective taking). Taken together, these findings suggest that mental simulation is supported by both domain-specific processes grounded in previous experiences, and by the neural mechanisms that underlie higher-order language processing (e.g., situation model building, event indexing, integration).

Neural processes underlying statistical learning for speech segmentation in dogs

To learn words, humans extract statistical regularities from speech. Multiple species use statistical learning also to process speech, but the neural underpinnings of speech segmentation in non-humans remain largely unknown. Here, we investigated computational and neural markers of speech segmentation in dogs, a phylogenetically distant mammal that efficiently navigates humans' social and linguistic environment. Using electroencephalography (EEG), we compared event-related responses (ERPs) for artificial words previously presented in a continuous speech stream with different distributional statistics. Results revealed an early effect (220-470 ms) of transitional probability and a late component (590-790 ms) modulated by both word frequency and transitional probability. Using fMRI, we searched for brain regions sensitive to statistical regularities in speech. Structured speech elicited lower activity in the basal ganglia, a region involved in sequence learning, and repetition enhancement in the auditory cortex. Speech segmentation in dogs, similar to that of humans, involves complex computations, engaging both domain-general and modality-specific brain areas. VIDEO ABSTRACT.

Modality-general and modality-specific audiovisual valence processing

A fundamental question in affective neuroscience is whether there is a common hedonic system for valence processing independent of modality, or there are distinct neural systems for different modalities. To address this question, we used both region of interest and whole-brain representational similarity analyses on functional magnetic resonance imaging data to identify modality-general and modality-specific brain areas involved in valence processing across visual and auditory modalities. First, region of interest analyses showed that the superior temporal cortex was associated with both modality-general and auditory-specific models, while the primary visual cortex was associated with the visual-specific model. Second, the whole-brain searchlight analyses also identified both modality-general and modality-specific representations. The modality-general regions included the superior temporal, medial superior frontal, inferior frontal, precuneus, precentral, postcentral, supramarginal, paracentral lobule and middle cingulate cortices. The modality-specific regions included both perceptual cortices and higher-order brain areas. The valence representations derived from individualized behavioral valence ratings were consistent with these results. Together, these findings suggest both modality-general and modality-specific representations of valence.

Attention shifting and subliminal cueing under high attentional load: an EEG study using emotional faces.

Emotional expressions and attentional processes interact such that emotion increases perceptual sensitivity in modality-specific brain areas leading to more efficient processing of the stimulus. In the current study, we asked the question of whether the emotional expressions can be processed subliminally when faces are attended but the emotional expression is irrelevant. We presented angry and neutral male and female faces supraliminally and subliminally to participants in an attention-shifting paradigm. Emotional expression was irrelevant to the task, but the gender of the face was predictive of the location of a subsequent target. Analysis of reaction times revealed that, in the supraliminal condition, targets appearing at validly cued locations produced quicker responses than targets at invalidly cued locations. Target-locked event-related potentials (ERPs) confirmed this result and showed that attention selection elicited higher responses at attended locations, as reflected by the N1 component. However, voluntary attention shifting was not triggered when the cue was not consciously perceived. No effects of emotion were observed following target presentation, suggesting that the high attentional load required by the experiment may have prevented the processing of emotion for undetectable faces, as demonstrated by the fact that, following cue onset, the N170 was modulated by emotion in supraliminal trials only. We therefore show that emotional expressions cannot be processed without awareness when faces are attended, but the emotional expression is irrelevant.

Feature activation during word recognition: action, visual, and associative-semantic priming effects

Embodied theories of language postulate that language meaning is stored in modality-specific brain areas generally involved in perception and action in the real world. However, the temporal dynamics of the interaction between modality-specific information and lexical-semantic processing remain unclear. We investigated the relative timing at which two types of modality-specific information (action-based and visual-form information) contribute to lexical-semantic comprehension. To this end, we applied a behavioral priming paradigm in which prime and target words were related with respect to (1) action features, (2) visual features, or (3) semantically associative information. Using a Go/No-Go lexical decision task, priming effects were measured across four different inter-stimulus intervals (ISI = 100, 250, 400, and 1000 ms) to determine the relative time course of the different features. Notably, action priming effects were found in ISIs of 100, 250, and 1000 ms whereas a visual priming effect was seen only in the ISI of 1000 ms. Importantly, our data suggest that features follow different time courses of activation during word recognition. In this regard, feature activation is dynamic, measurable in specific time windows but not in others. Thus the current study (1) demonstrates how multiple ISIs can be used within an experiment to help chart the time course of feature activation and (2) provides new evidence for embodied theories of language.

Electrophysiological correlates of the retention of tones differing in timbre in auditory short-term memory

We examined the electrophysiological correlates of retention in auditory short-term memory (ASTM) for sequences of one, two, or three tones differing in timbre but having the same pitch. We focused on event-related potentials (ERPs) during the retention interval and revealed a sustained fronto-central ERP component (most likely a sustained anterior negativity; SAN) that became more negative as memory load increased. Our results are consistent with recent ERP studies on the retention of pitch and suggest that the SAN reflects brain activity mediating the low-level retention of basic acoustic features in ASTM. The present work shows that the retention of timbre shares common features with the retention of pitch, hence supporting the notion that the retention of basic sensory features is an active process that recruits modality-specific brain areas.

Losing the sound of concepts: Damage to auditory association cortex impairs the processing of sound-related concepts

Conceptual knowledge is classically supposed to be abstract and represented in an amodal unitary system, distinct from the sensory and motor brain systems. A more recent embodiment view of conceptual knowledge, however, proposes that concepts are grounded in distributed modality-specific brain areas which typically process sensory or action-related object information. Recent neuroimaging evidence suggested the significance of left auditory association cortex encompassing posterior superior and middle temporal gyrus in coding conceptual sound features of everyday objects. However, a causal role of this region in processing conceptual sound information has yet to be established. Here we had the unique chance to investigate a patient, JR, with a focal lesion in left posterior superior and middle temporal gyrus. To test the necessity of this region in conceptual and perceptual processing of sound information we administered four different experimental tasks to JR: Visual word recognition, category fluency, sound recognition and voice classification. Compared with a matched control group, patient JR was consistently impaired in conceptual processing of sound-related everyday objects (e.g., “bell”), while performance for non-sound-related everyday objects (e.g., “armchair”), animals, whether they typically produce sounds (e.g., “frog”) or not (e.g., “tortoise”), and musical instruments (e.g., “guitar”) was intact. An analogous deficit pattern in JR was also obtained for perceptual recognition of the corresponding sounds. Hence, damage to left auditory association cortex specifically impairs perceptual and conceptual processing of sounds from everyday objects. In support of modality-specific theories, these findings strongly evidence the necessity of auditory association cortex in coding sound-related conceptual information.

A role for the insula in color-induced nasal thermal sensations

This article is the first step towards understanding the mechanisms underlying the intriguing, recently discovered lateralized color-induced nasal thermal sensations. In the presence of color cues and complete absence of thermal stimulus, larger sensitivity of the left nostril/right hemisphere (RH) for warming sensations and larger right nostril/left hemisphere (LH) for cooling sensations were replicated several times. It was suggested that engagement in a temperature judgment task and the development of specific expectancies due to the presence of color cues could alter and enhance processing in brain areas involved in thermosensory processing, such as the middle/posterior insula. The lateralized patterns could thus intimate hemispheric specialization for thermosensory processing. However, such lateralization may be due to either exclusive specialization of each hemisphere or specialization-through-reciprocal inhibition between the hemispheres. The two hypotheses predict different results following a unilateral insular stroke. Here, we present the results of a sample of healthy volunteers and patient MB, a young woman suffering from unilateral left-side damage of the posterior insula, in a task involving color-induced nasal thermal judgment. The expected lateralized pattern was found in the performance of the controls. In line with our previous suggestions that the LH is more involved in the processing of cooling sensations, patient MB exhibited changes only in the judgment of cooling sensations. Her results also clearly support the specialization-through-reciprocal inhibition account since she exhibited decreased cooling judgments contralaterally but increased cooling judgments ipsilaterally. Accordingly, we conclude that (a) the lateralized patterns arise because of hemispheric specialization; (b) the LH is seemingly more involved in the processing of cooling sensations; (c) this specialization is underlain by reciprocal interhemispheric inhibition; and (d) even in the absence of thermal stimulus, the development of expectancies suffices to activate modality-specific brain areas involved in the current task in such a way that damage to these areas disturbs the corresponding specific processes.

Embodied Language Comprehension Requires an Enactivist Paradigm of Cognition

Two recurrent concerns in discussions on an embodied view of cognition are the “necessity question” (i.e., is activation in modality-specific brain areas necessary for language comprehension?) and the “simulation constraint” (i.e., how do we understand language for which we lack the relevant experiences?). In the present paper we argue that the criticisms encountered by the embodied approach hinge on a cognitivist interpretation of embodiment. We argue that the data relating sensorimotor activation to language comprehension can best be interpreted as supporting a non-representationalist, enactivist model of language comprehension, according to which language comprehension can be described as procedural knowledge – knowledge how, not knowledge that – that enables us to interact with others in a shared physical world. The enactivist view implies that the activation of modality-specific brain areas during language processing reflects the employment of sensorimotor skills and that language comprehension is a context-bound phenomenon. Importantly, an enactivist view provides an embodied approach of language, while avoiding the problems encountered by a cognitivist interpretation of embodiment.

The emergence of semantic categorization in early visual processing: ERP indices of animal vs. artifact recognition.

BackgroundNeuroimaging and neuropsychological literature show functional dissociations in brain activity during processing of stimuli belonging to different semantic categories (e.g., animals, tools, faces, places), but little information is available about the time course of object perceptual categorization. The aim of the study was to provide information about the timing of processing stimuli from different semantic domains, without using verbal or naming paradigms, in order to observe the emergence of non-linguistic conceptual knowledge in the ventral stream visual pathway. Event related potentials (ERPs) were recorded in 18 healthy right-handed individuals as they performed a perceptual categorization task on 672 pairs of images of animals and man-made objects (i.e., artifacts).ResultsBehavioral responses to animal stimuli were ~50 ms faster and more accurate than those to artifacts. At early processing stages (120–180 ms) the right occipital-temporal cortex was more activated in response to animals than to artifacts as indexed by posterior N1 response, while frontal/central N1 (130–160) showed the opposite pattern. In the next processing stage (200–260) the response was stronger to artifacts and usable items at anterior temporal sites. The P300 component was smaller, and the central/parietal N400 component was larger to artifacts than to animals.ConclusionThe effect of animal and artifact categorization emerged at ~150 ms over the right occipital-temporal area as a stronger response of the ventral stream to animate, homomorphic, entities with faces and legs. The larger frontal/central N1 and the subsequent temporal activation for inanimate objects might reflect the prevalence of a functional rather than perceptual representation of manipulable tools compared to animals. Late ERP effects might reflect semantic integration and cognitive updating processes. Overall, the data are compatible with a modality-specific semantic memory account, in which sensory and action-related semantic features are represented in modality-specific brain areas.

Remembrance of Odors Past: Human Olfactory Cortex in Cross-Modal Recognition Memory

Episodic memory is often imbued with multisensory richness, such that the recall of an event can be endowed with the sights, sounds, and smells of its prior occurrence. While hippocampus and related medial temporal structures are implicated in episodic memory retrieval, the participation of sensory-specific cortex in representing the qualities of an episode is less well established. We combined functional magnetic resonance imaging (fMRI) with a cross-modal paradigm, where objects were presented with odors during memory encoding. We then examined the effect of odor context on neural responses at retrieval when these same objects were presented alone. Primary olfactory (piriform) cortex, as well as anterior hippocampus, was activated during the successful retrieval of old (compared to new) objects. Our findings indicate that sensory features of the original engram are preserved in unimodal olfactory cortex. We suggest that reactivation of memory traces distributed across modality-specific brain areas underpins the sensory qualities of episodic memories.

Bimodal and Modality-Specific Brain Areas Activated During a Spatial Selective Attention Task

Bimodal and Modality-Specific Brain Areas Activated During a Spatial Selective Attention Task