Modality-specific Mechanisms Research Articles

Contribution and functional connectivity between cerebrum and cerebellum on sub-lexical and lexical-semantic processing of verbs.

Language comprehension involves both sub-lexical (e.g., phonological) and lexical-semantic processing. We conducted a task using functional magnetic resonance imaging (fMRI) to compare the processing of verbs in these two domains. Additionally, we examined the representation of concrete-motor and abstract-non-motor concepts by including two semantic categories of verbs: motor and mental. The findings indicate that sub-lexical processing during the reading of pseudo-verbs primarily involves the left dorsal stream of the perisylvian network, while lexical-semantic representation during the reading of verbs predominantly engages the ventral stream. According to the embodied or grounded cognition approach, modality-specific mechanisms (such as sensory-motor systems) and the well-established multimodal left perisylvian network contribute to the semantic representation of both concrete and abstract verbs. Our study identified the visual system as a preferential modality-specific system for abstract-mental verbs, which exhibited functional connectivity with the right crus I/lobule VI of the cerebellum. Taken together, these results confirm the dissociation between sub-lexical and lexical-semantic processing and provide neurobiological evidence of functional coupling between specific visual modality regions and the right cerebellum, forming a network that supports the semantic representation of abstract concepts. Further, the results shed light on the underlying mechanisms of semantic processing and contribute to our understanding of how the brain processes abstract concepts.

Cross-modal implicit learning of random time patterns

Perception is sensitive to statistical regularities in the environment, including temporal characteristics of sensory inputs. Interestingly, implicit learning of temporal patterns in one modality can also improve their processing in another modality. However, it is unclear how cross-modal learning transfer affects neural responses to sensory stimuli. Here, we recorded neural activity of human volunteers using electroencephalography (EEG), while participants were exposed to brief sequences of randomly timed auditory or visual pulses. Some trials consisted of a repetition of the temporal pattern within the sequence, and subjects were tasked with detecting these trials. Unknown to the participants, some trials reappeared throughout the experiment across both modalities (Transfer) or only within a modality (Control), enabling implicit learning in one modality and its transfer. Using a novel method of analysis of single-trial EEG responses, we showed that learning temporal structures within and across modalities is reflected in neural learning curves. These putative neural correlates of learning transfer were similar both when temporal information learned in audition was transferred to visual stimuli and vice versa. The modality-specific mechanisms for learning of temporal information and general mechanisms which mediate learning transfer across modalities had distinct physiological signatures: temporal learning within modalities relied on modality-specific brain regions while learning transfer affected beta-band activity in frontal regions.

Increases in sensory noise predict attentional disruptions to audiovisual speech perception.

We receive information about the world around us from multiple senses which combine in a process known as multisensory integration. Multisensory integration has been shown to be dependent on attention; however, the neural mechanisms underlying this effect are poorly understood. The current study investigates whether changes in sensory noise explain the effect of attention on multisensory integration and whether attentional modulations to multisensory integration occur via modality-specific mechanisms. A task based on the McGurk Illusion was used to measure multisensory integration while attention was manipulated via a concurrent auditory or visual task. Sensory noise was measured within modality based on variability in unisensory performance and was used to predict attentional changes to McGurk perception. Consistent with previous studies, reports of the McGurk illusion decreased when accompanied with a secondary task; however, this effect was stronger for the secondary visual (as opposed to auditory) task. While auditory noise was not influenced by either secondary task, visual noise increased with the addition of the secondary visual task specifically. Interestingly, visual noise accounted for significant variability in attentional disruptions to the McGurk illusion. Overall, these results strongly suggest that sensory noise may underlie attentional alterations to multisensory integration in a modality-specific manner. Future studies are needed to determine whether this finding generalizes to other types of multisensory integration and attentional manipulations. This line of research may inform future studies of attentional alterations to sensory processing in neurological disorders, such as Schizophrenia, Autism, and ADHD.

A supramodal and conceptual representation of subsecond time revealed with perceptual learning of temporal interval discrimination

Subsecond time perception has been frequently attributed to modality-specific timing mechanisms that would predict no cross-modal transfer of temporal perceptual learning. In fact, perceptual learning of temporal interval discrimination (TID) reportedly shows either no cross-modal transfer, or asymmetric transfer from audition to vision, but not vice versa. However, here we demonstrate complete cross-modal transfer of auditory and visual TID learning using a double training paradigm. Specifically, visual TID learning transfers to and optimizes auditory TID when the participants also receive exposure to the auditory temporal interval by practicing a functionally orthogonal near-threshold tone frequency discrimination task at the same trained interval. Auditory TID learning also transfers to and optimizes visual TID with additional practice of an orthogonal near-threshold visual contrast discrimination task at the same trained interval. Practicing these functionally orthogonal tasks per se has no impact on TID thresholds. We interpret the transfer results as indications of a supramodal representation of subsecond time. Moreover, because TID learning shows complete transfer between modalities with vastly different temporal precisions, the sub-second time presentation must be conceptual. Double training may refine this supramodal and conceptual subsecond time representation and connect it to a new sense to improve time perception.

The Intricate Interplay of Spatial Attention and Expectation: a Multisensory Perspective.

Attention (i.e., task relevance) and expectation (i.e., signal probability) are two critical top-down mechanisms guiding perceptual inference. Attention prioritizes processing of information that is relevant for observers' current goals. Prior expectations encode the statistical structure of the environment. Research to date has mostly conflated spatial attention and expectation. Most notably, the Posner cueing paradigm manipulates spatial attention using probabilistic cues that indicate where the subsequent stimulus is likely to be presented. Only recently have studies attempted to dissociate the mechanisms of attention and expectation and characterized their interactive (i.e., synergistic) or additive influences on perception. In this review, we will first discuss methodological challenges that are involved in dissociating the mechanisms of attention and expectation. Second, we will review research that was designed to dissociate attention and expectation in the unisensory domain. Third, we will review the broad field of crossmodal endogenous and exogenous spatial attention that investigates the impact of attention across the senses. This raises the critical question of whether attention relies on amodal or modality-specific mechanisms. Fourth, we will discuss recent studies investigating the role of both spatial attention and expectation in multisensory perception, where the brain constructs a representation of the environment based on multiple sensory inputs. We conclude that spatial attention and expectation are closely intertwined in almost all circumstances of everyday life. Yet, despite their intimate relationship, attention and expectation rely on partly distinct neural mechanisms: while attentional resources are mainly shared across the senses, expectations can be formed in a modality-specific fashion.

Shifts of Spatial Attention in Visual and Tactile Working Memory are Controlled by Independent Modality-Specific Mechanisms.

The question whether the attentional control of working memory (WM) is shared across sensory modalities remains controversial. Here, we investigated whether attention shifts in visual and tactile WM are regulated independently. Participants memorized visual and tactile targets in a first memory sample set (S1) before encoding targets in a second sample set (S2). Importantly, visual or tactile S2 targets could appear on the same side as the corresponding S1 targets, or on opposite sides, thus, requiring shifts of spatial attention in visual or tactile WM. The activation of WM representations in modality-specific visual and somatosensory areas was tracked by recording visual and tactile contralateral delay activity (CDA/tCDA). CDA/tCDA components emerged contralateral to the side of visual or tactile S1 targets, and reversed polarity when S2 targets in the same modality appeared on the opposite side. Critically, the visual CDA was unaffected by the presence versus absence of concurrent attention shifts in tactile WM, and the tactile CDA remained insensitive to visual attention shifts. Visual and tactile WM performance was also not modulated by attention shifts in the other modality. These results show that the dynamic control of visual and tactile WM activation processes operates in an independent modality-specific fashion.

Integrative systematic review of psychodrama psychotherapy research: Trends and methodological implications.

BackgroundPsychodrama is an experiential psychotherapy in which guided role-play is used to gain insights and work on personal and interpersonal problems and possible solutions. Despite the wealth of literature describing clinical work, psychodrama intervention research is relatively scarce compared to other psychotherapies and psychological interventions.ObjectiveFor this reason we implemented the integrative approach to systematic review that authorizes the combination of publications with diverse methodologies and all types of participants, interventions, comparisons, and outcomes. Our aim was to produce a comprehensive summary of psychodrama intervention research in the last decade that critically evaluates methodological issues to inform future studies.MethodsWe searched four major electronic databases (PsycINFO, PubMEd, Scopus by Elsevier, and Web of Science) for peer-reviewed articles on psychodrama interventions published in English between 1 January 2007 and 31 December 2017. The quality of qualitative and mixed methods studies was assessed on the basis of pre-established guidelines, and the risk of bias was assessed for all quantitative randomized control studies, consistent with the PRISMA protocol.FindingsThe database search and a hand search resulted in 31 psychodrama intervention publications. Overall, these studies examined the effects of psychodrama on more than 20 different outcomes and most studies had adult clients. The next largest group was adolescents, whereas only two studies involved children. Thus psychodrama intervention research in the last decade suggests there are promising results in all methodologies, and highlights the need to enhance methodological as well as reporting quality and to theorize and examine modality-specific mechanisms that lead to therapeutic change. Recommendations to improve methodology, transparency, and specificity in reporting future psychodrama and other psychotherapy research are discussed.

Functional similarities and differences between the coding of positional information in verbal and spatial short-term order memory

ABSTRACTTemporal grouping effects in verbal and spatial serial recall suggest that the representation of serial order in verbal and spatial short-term memory (STM) incorporates positional information. However, not all effects of grouping are created equal in the verbal and spatial domains. Although grouping a sequence of verbal items engenders an increase in between-group transpositions that maintain their within-group position, grouping a sequence of spatial items does not engender an increase in these so-called interposition errors. Here I present experimental and computational modeling evidence which suggests that positional information is represented in subtly different ways in verbal and spatial STM. Specifically, the findings indicate that in verbal STM, groups are coded for their position in a sequence and items are coded for their position in a group. By contrast, in spatial STM groups are coded for their position in a sequence, but items are coded for their position in a sequence, rather than in a group. Findings support the notion that positional information in verbal and spatial STM is represented by modality-specific mechanisms rather than a domain-general system.

Neural practice effect during cross-modal selective attention: Supra-modal and modality-specific effects

Practice and experiences gradually shape the central nervous system, from the synaptic level to large-scale neural networks. In natural multisensory environment, even when inundated by streams of information from multiple sensory modalities, our brain does not give equal weight to different modalities. Rather, visual information more frequently receives preferential processing and eventually dominates consciousness and behavior, i.e., visual dominance. It remains unknown, however, the supra-modal and modality-specific practice effect during cross-modal selective attention, and moreover whether the practice effect shows similar modality preferences as the visual dominance effect in the multisensory environment. To answer the above two questions, we adopted a cross-modal selective attention paradigm in conjunction with the hybrid fMRI design. Behaviorally, visual performance significantly improved while auditory performance remained constant with practice, indicating that visual attention more flexibly adapted behavior with practice than auditory attention. At the neural level, the practice effect was associated with decreasing neural activity in the frontoparietal executive network and increasing activity in the default mode network, which occurred independently of the modality attended, i.e., the supra-modal mechanisms. On the other hand, functional decoupling between the auditory and the visual system was observed with the progress of practice, which varied as a function of the modality attended. The auditory system was functionally decoupled with both the dorsal and ventral visual stream during auditory attention while was decoupled only with the ventral visual stream during visual attention. To efficiently suppress the irrelevant visual information with practice, auditory attention needs to additionally decouple the auditory system from the dorsal visual stream. The modality-specific mechanisms, together with the behavioral effect, thus support the visual dominance model in terms of the practice effect during cross-modal selective attention.

Bouncing Ball with a Uniformly Varying Velocity in a Metronome Synchronization Task.

Sensorimotor synchronization (SMS), a fundamental human ability to coordinate movements with external rhythms, has long been thought to be modality specific. In the canonical metronome synchronization task that requires tapping a finger along with an isochronous sequence, a well-established finding is that synchronization is much more stable to an auditory sequence consisting of auditory tones than to a visual sequence consisting of visual flashes. However, recent studies have shown that periodically moving visual stimuli can substantially improve synchronization compared with visual flashes. In particular, synchronization of a visual bouncing ball that has a uniformly varying velocity was found to be not less stable than synchronization of auditory tones. Here, the current protocol describes the application of the bouncing ball with a uniformly varying velocity in a metronome synchronization task. The usage of the bouncing ball in sequences with different inter-onset intervals (IOI) is included. The representative results illustrate synchronization performance of the bouncing ball, as compared with the performances of auditory tones and visual flashes. Given its comparable synchronization performance to that of auditory tones, the bouncing ball is of particular importance for addressing the current research topic of whether modality-specific mechanisms underlay SMS.

Behavioral, Modeling, and Electrophysiological Evidence for Supramodality in Human Metacognition.

Human metacognition, or the capacity to introspect on one's own mental states, has been mostly characterized through confidence reports in visual tasks. A pressing question is to what extent results from visual studies generalize to other domains. Answering this question allows determining whether metacognition operates through shared, supramodal mechanisms or through idiosyncratic, modality-specific mechanisms. Here, we report three new lines of evidence for decisional and postdecisional mechanisms arguing for the supramodality of metacognition. First, metacognitive efficiency correlated among auditory, tactile, visual, and audiovisual tasks. Second, confidence in an audiovisual task was best modeled using supramodal formats based on integrated representations of auditory and visual signals. Third, confidence in correct responses involved similar electrophysiological markers for visual and audiovisual tasks that are associated with motor preparation preceding the perceptual judgment. We conclude that the supramodality of metacognition relies on supramodal confidence estimates and decisional signals that are shared across sensory modalities.SIGNIFICANCE STATEMENT Metacognitive monitoring is the capacity to access, report, and regulate one's own mental states. In perception, this allows rating our confidence in what we have seen, heard, or touched. Although metacognitive monitoring can operate on different cognitive domains, we ignore whether it involves a single supramodal mechanism common to multiple cognitive domains or modality-specific mechanisms idiosyncratic to each domain. Here, we bring evidence in favor of the supramodality hypothesis by showing that participants with high metacognitive performance in one modality are likely to perform well in other modalities. Based on computational modeling and electrophysiology, we propose that supramodality can be explained by the existence of supramodal confidence estimates and by the influence of decisional cues on confidence estimates.

Adapting to time: Duration channels do not mediate human time perception.

Accurately encoding the duration and temporal order of events is essential for survival and important to everyday activities, from holding conversations to driving in fast-flowing traffic. Although there is a growing body of evidence that the timing of brief events (< 1 s) is encoded by modality-specific mechanisms, it is not clear how such mechanisms register event duration. One approach gaining traction is a channel-based model; this envisages narrowly-tuned, overlapping timing mechanisms that respond preferentially to different durations. The channel-based model predicts that adapting to a given event duration will result in overestimating and underestimating the duration of longer and shorter events, respectively. We tested the model by having observers judge the duration of a brief (600 ms) visual test stimulus following adaptation to longer (860 ms) and shorter (340 ms) stimulus durations. The channel-based model predicts perceived duration compression of the test stimulus in the former condition and perceived duration expansion in the latter condition. Duration compression occurred in both conditions, suggesting that the channel-based model does not adequately account for perceived duration of visual events.

Modality-specific processing precedes amodal linguistic processing during L2 sign language acquisition: A longitudinal study

The present study tracked activation pattern differences in response to sign language processing by late hearing second language learners of American Sign Language. Learners were scanned before the start of their language courses. They were scanned again after their first semester of instruction and their second, for a total of 10 months of instruction. The study aimed to characterize modality-specific to modality-general processing throughout the acquisition of sign language. Results indicated that before the acquisition of sign language, neural substrates related to modality-specific processing were present. After approximately 45 h of instruction, the learners transitioned into processing signs on a phonological basis (e.g., supramarginal gyrus, putamen). After one more semester of input, learners transitioned once more to a lexico-semantic processing stage (e.g., left inferior frontal gyrus) at which language control mechanisms (e.g., left caudate, cingulate gyrus) were activated. During these transitional steps right hemispheric recruitment was observed, with increasing left-lateralization, which is similar to other native signers and L2 learners of spoken language; however, specialization for sign language processing with activation in the inferior parietal lobule (i.e., angular gyrus), even for late learners, was observed. As such, the present study is the first to track L2 acquisition of sign language learners in order to characterize modality-independent and modality-specific mechanisms for bilingual language processing.

Modality-specific mechanisms of protein kinase C-induced hypersensitivity of TRPV1: S800 is a polymodal sensitization site.

TRPV1 is a nociceptive ion channel activated by polymodal stimuli such as capsaicin, proton, and noxious heat. Multiple inflammatory mediators activate protein kinases, especially protein kinase C (PKC), which phosphorylates TRPV1. Emerging evidence suggests that phosphorylation of TRPV1 constitutes specific signals underpinning pathological nociception. Although the mechanisms of hypersensitivity of TRPV1 to capsaicin are well studied, the phosphorylation residues that contribute to hypersensitivity to heat or acid have not been identified. In this study, we investigated modality-specific mechanisms of PKC-induced hypersensitivity using mutagenic ablation of PKC-associated phosphorylation sites in TRPV1. In heterologous systems, TRPV1 S502 and S800, but not T704, are known to be involved in hypersensitivity to capsaicin after the application of phorbol myristate acetate (PMA), a PKC agonist. Unlike capsaicin, PMA-induced hypersensitivity to heat was attenuated in TRPV1 mutants T704A and S800A, but not in S502A. In contrast, PMA-induced hypersensitivity to acid was attenuated only in S800A. To examine the roles of these phosphorylation sites in more physiologically relevant conditions, TRPV1 and mutants were tested in sensory neurons from TRPV1-null mice. In sensory neurons expressing mutated TRPV1, we found that alanine mutation of S800 commonly attenuates PMA-induced hypersensitivity to capsaicin, heat, and acid. Moreover, bradykinin-induced hypersensitivity to capsaicin was largely attenuated by the S800A mutation. These results suggest that mechanisms of PKC-induced hypersensitivity of TRPV1 are modality specific and that S800 is a polymodal sensitization site integrating multiple inflammatory signals in nociceptors. Our data provide a rationale for a novel approach targeting TRPV1 S800 for antihyperalgesia.

Brain Regions That Represent Amodal Conceptual Knowledge

To what extent do the brain regions implicated in semantic processing contribute to the representation of amodal conceptual content rather than modality-specific mechanisms or mechanisms of semantic access and manipulation? Here, we propose that a brain region can be considered to represent amodal conceptual object knowledge if it is supramodal and plays a role in distinguishing among the conceptual representations of different objects. In an fMRI study, human participants made category typicality judgments about pictured objects or their names drawn from five different categories. Crossmodal multivariate pattern analysis revealed a network of six left-lateralized regions largely outside of category-selective visual cortex that showed a supramodal representation of object categories. These were located in the posterior middle/inferior temporal gyrus (pMTG/ITG), angular gyrus, ventral temporal cortex, posterior cingulate/precuneus (PC), and lateral and dorsomedial prefrontal cortex. Representational similarity analysis within these regions determined that the similarity between category-specific patterns of neural activity in the pMTG/ITG and the PC was consistent with the semantic similarity between these categories. This finding supports the PC and pMTG/ITG as candidate regions for the amodal representation of the conceptual properties of objects.

Modality-specific organization in the representation of sensorimotor sequences

Sensorimotor representations of movement sequences are hierarchically organized. Here we test the effects of different stimulus modalities on such organizations. In the visual group, participants responded to a repeated sequence of visually presented stimuli by depressing spatially compatible keys on a response pad. In the auditory group, learners were required to respond to auditorily presented stimuli, which had no direct spatial correspondence with the response keys: the lowest pitch corresponded to the leftmost key and the highest pitch to the rightmost key. We demonstrate that hierarchically and auto-organized sensorimotor representations are developed through practice, which are specific both to individuals and stimulus modalities. These findings highlight the dynamic and sensory-specific modulation of chunk processing during sensorimotor learning – sensorimotor chunking – and provide evidence that modality-specific mechanisms underlie the hierarchical organization of sequence representations.

Auditory and visual temporal sensitivity: evidence for a hierarchical structure of modality-specific and modality-independent levels of temporal information processing

The present study investigated modality-specific differences in processing of temporal information in the subsecond range. For this purpose, participants performed auditory and visual versions of a rhythm perception and three different duration discrimination tasks to allow for a direct, systematic comparison across both sensory modalities. Our findings clearly indicate higher temporal sensitivity in the auditory than in the visual domain irrespective of type of timing task. To further evaluate whether there is evidence for a common modality-independent timing mechanism or for multiple modality-specific mechanisms, we used structural equation modeling to test three different theoretical models. Neither a single modality-independent timing mechanism, nor two independent modality-specific timing mechanisms fitted the empirical data. Rather, the data are well described by a hierarchical model with modality-specific visual and auditory temporal processing at a first level and a modality-independent processing system at a second level of the hierarchy.

Learning across senses: cross-modal effects in multisensory statistical learning.

It is currently unknown whether statistical learning is supported by modality-general or modality-specific mechanisms. One issue within this debate concerns the independence of learning in one modality from learning in other modalities. In the present study, the authors examined the extent to which statistical learning across modalities is independent by simultaneously presenting learners with auditory and visual streams. After establishing baseline rates of learning for each stream independently, they systematically varied the amount of audiovisual correspondence across 3 experiments. They found that learners were able to segment both streams successfully only when the boundaries of the audio and visual triplets were in alignment. This pattern of results suggests that learners are able to extract multiple statistical regularities across modalities provided that there is some degree of cross-modal coherence. They discuss the implications of their results in light of recent claims that multisensory statistical learning is guided by modality-independent mechanisms.

Processing of disgusted faces is facilitated by odor primes: A functional MRI study

Facilitation of emotional face recognition is an established phenomenon for audiovisual crossmodal stimulation, but not for other sensory modalities. The present study used a crossmodal priming task to identify brain systems controlling olfactory–visual interactions during emotion processing. BOLD fMRI was acquired for 44 healthy subjects during an emotional face discrimination task preceded by an emotionally valenced odorant. Behavioral performance showed that recognition of disgusted faces was improved by the presentation of an olfactory stimulus irrespective of its emotional valence. No such facilitation was seen for other facial expressions. The neuroimaging data showed a selective default network responsivity to emotional faces which was modulated by odor condition. Among disgust faces, hypoactivations during trials preceded by odorants indicated the presence of priming effects. Consistent with studies investigating the brain systems associated with audiovisual emotional integration, activity modulations in clusters in fusiform gyrus, middle frontal and middle cingulate gyrus corresponded to the observed behavioral facilitation. Our study further shows modulation of signal in the anterior insula during trials combining negatively valenced odor and disgusted faces, suggesting a modality-specific mechanism for integration of the disgust response and olfaction. These results indicate the presence of a central network with modality-specific and -unspecific components modulating emotional face recognition.

Caenorhabditis elegans TRPV Channels Function in a Modality-Specific Pathway to Regulate Response to Aberrant Sensory Signaling

Olfaction and some forms of taste (including bitter) are mediated by G protein-coupled signal transduction pathways. Olfactory and gustatory ligands bind to chemosensory G protein-coupled receptors (GPCRs) in specialized sensory cells to activate intracellular signal transduction cascades. G protein-coupled receptor kinases (GRKs) are negative regulators of signaling that specifically phosphorylate activated GPCRs to terminate signaling. Although loss of GRK function usually results in enhanced cellular signaling, Caenorhabditis elegans lacking GRK-2 function are not hypersensitive to chemosensory stimuli. Instead, grk-2 mutant animals do not chemotax toward attractive olfactory stimuli or avoid aversive tastes and smells. We show here that loss-of-function mutations in the transient receptor potential vanilloid (TRPV) channels OSM-9 and OCR-2 selectively restore grk-2 behavioral avoidance of bitter tastants, revealing modality-specific mechanisms for TRPV channel function in the regulation of C. elegans chemosensation. Additionally, a single amino acid point mutation in OCR-2 that disrupts TRPV channel-mediated gene expression, but does not decrease channel function in chemosensory primary signal transduction, also restores grk-2 bitter taste avoidance. Thus, loss of GRK-2 function may lead to changes in gene expression, via OSM-9/OCR-2, to selectively alter the levels of signaling components that transduce or regulate bitter taste responses. Our results suggest a novel mechanism and multiple modality-specific pathways that sensory cells employ in response to aberrant signal transduction.