Multivoxel Pattern Research Articles

Neural Correlates of Retrieval Success and Precision: A Functional Magnetic Resonance Imaging Study

Abstract Prior studies examining the neural mechanisms underlying retrieval success and precision have yielded inconsistent results. Here, the neural correlates of success and precision were examined with a memory task that assessed precision for spatial location. A sample of healthy young adults underwent functional magnetic resonance imaging scanning during a single study–test cycle. At study, participants viewed a series of object images, each placed at a randomly selected location on an imaginary circle. At test, studied images were intermixed with new images and presented to the participants. The requirement was to move a cursor to the location of the studied image, guessing if necessary. Participants then signaled whether the presented image had been studied. Memory precision was quantified as the angular difference between the studied location and the location selected by the participant. A precision effect was evident in the left angular gyrus, where BOLD activity covaried with location accuracy. In addition, multivoxel pattern analysis revealed a significant item-level reinstatement effect in the angular gyrus for high-precision trials. There was no evidence of a retrieval success effect in this region. BOLD activity in the hippocampus was insensitive to both success and precision. These findings are partially consistent with prior evidence that success and precision are dissociable features of memory retrieval.

Syntactic and semantic specialization in 9- to 10-year-old children during auditory sentence processing

Prior literature has debated whether syntax is separable from semantics in the brain. Using functional magnetic resonance imaging and multi-voxel pattern analysis, our previous studies investigated brain activity during morpho-syntactic versus semantic processing. These studies only detected semantic specialization in activation patterns and no syntactic specialization in 5- to 6-year-old and 7- to 8-year-old children. To examine if older children who have mastered morpho-syntactic skills would show specialization for syntax, the current study examined 64 9- to 10-year-old children using the same design and analyses. We observed that only the left IFG pars opercularis was sensitive to syntactic but not semantic information, supporting the hypothesis that this region serves as a core region for syntax. In addition, the left STG which has been implicated in the integration of semantics and syntax, as well as the left MTG and IFG pars triangularis which have been implicated in semantics, were sensitive to both semantic and syntactic information with no evidence of specialization. These findings suggest a lexicalized view of syntax, which argues that semantically sensitive regions are also critical regions for syntactic processing during language comprehension.

A 7T fMRI investigation of hand and tool areas in the lateral and ventral occipitotemporal cortex.

Previous studies demonstrated the existence of hand and tool areas in lateral and ventral occipitotemporal cortex (OTC), as well as an overlap between them. We reinvestigated this organization using 7T fMRI, benefiting from a higher signal-to-noise ratio than 3T. This enabled us to include a wider array of categories to achieve a more holistic perspective, and to omit certain spatial preprocessing steps. Despite these improvements, univariate analysis confirmed the existence of hand-tool overlap across OTC, which is striking given the omission of the spatial preprocessing steps that can influence overlap. There was significantly more overlap between hands and tools, compared to other overlap types in the left hemisphere of OTC. The overlap was also larger in the left lateral OTC as compared to the right lateral OTC. We found in all hand areas a differentiation between tools and other types of objects, although they still responded more to bodies than to tools. Regarding the tool areas, we observed a differentiation between hands and other categories such as faces and non-tool objects. Left hemisphere tool areas also differentiated between hands and bodies. When excluding the overlapping voxels from the hand and tool areas, they still showed a significant response to tools or hands (compared to objects or faces) respectively. Multi-voxel pattern analysis indicated that neural representations in the hand areas showed greater similarity between hands and tools than between hands and other objects. In the tool areas, the neural representations between tools and hands and between tools and other type of objects were all equally similar. To summarize, capitalizing on the benefits of 7T fMRI, we further substantiate the evidence in favor of hand-tool overlap in several regions of occipitotemporal cortex.

The Relationship between Event Boundary Strength and Pattern Shifts across the Cortical Hierarchy during Naturalistic Movie-viewing.

Our continuous experience is spontaneously segmented by the brain into discrete events. However, the beginning of a new event (an event boundary) is not always sharply identifiable: Phenomenologically, event boundaries vary in salience. How are the response profiles of cortical areas at event boundaries modulated by boundary strength during complex, naturalistic movie-viewing? Do cortical responses scale in a graded manner with boundary strength, or do they merely detect boundaries in a binary fashion? We measured "cortical boundary shifts" as transient changes in multivoxel patterns at event boundaries with different strengths (weak, moderate, and strong), determined by across-participant agreement. Cortical regions with different processing timescales were examined. In auditory areas, which have short timescales, cortical boundary shifts exhibited a clearly graded profile in both group-level and individual-level analyses. In cortical areas with long timescales, including the default mode network, boundary strength modulated pattern shift magnitude at the individual participant level. We also observed a positive relationship between boundary strength and the extent of temporal alignment of boundary shifts across different levels of the cortical hierarchy. In addition, hippocampal activity was highest at event boundaries for which cortical boundary shifts were most aligned across hierarchical levels. Overall, we found that event boundary strength modulated cortical pattern shifts strongly in sensory areas and more weakly in higher-level areas and that stronger boundaries were associated with greater alignment of these shifts across the cortical hierarchy.

A common and specialized neural code for social attention triggered by eye gaze and biological motion

Humans appear to be endowed with the ability to readily share attention with interactive partners through the utilization of social direction cues, such as eye gaze and biological motion (BM). Here, we investigated the specialized brain mechanism underlying this fundamental social attention ability by incorporating different types of social (i.e., BM, gaze) and non-social (arrow) cues and combining functional magnetic resonance imaging (fMRI) with a modified central cueing paradigm. Using multi-voxel pattern analysis (MVPA), we found that although gaze- and BM-mediated attentional orienting could be decoded from neural activity in a wide range of brain areas, only the right anterior and posterior superior temporal sulcus (aSTS and pSTS) could specifically decode attentional orienting triggered by social but not non-social cues. Critically, cross-category MVPA further revealed that social attention could be decoded across BM and gaze cues in the right STS and the right superior temporal gyrus (STG). However, these regions could not decode attentional orienting across social and non-social cues. These findings together provide evidence for the existence of a specialized social attention module in the human brain, with the right STS/STG being the critical neural site dedicated to social attention.

Brain activity associated with emotion regulation predicts individual differences in working memory ability.

Previous behavioral research has found that working memory is associated with emotion regulation efficacy. However, there has been mixed evidence as to whether the neural mechanisms between emotion regulation and working memory overlap. The present study tested the prediction that individual differences on the working memory subtest of the Weschler Adult Intelligence Scale (WAIS-IV) could be predicted from the pattern of brain activity produced during emotion regulation in regions typically associated with working memory, such as the dorsal lateral prefrontal cortex (dlPFC). A total of 101 participants completed an emotion regulation fMRI task in which they either viewed or reappraised negative images. Participants also completed working memory test outside the scanner. A whole brain covariate analysis contrasting the reappraise negative and view negative BOLD response found that activity in the right dlPFC positively related to working memory ability. Moreover, a multivoxel pattern analysis approach using tenfold cross-validated support vector regression in regions-of-interest associated with working memory, including bilateral dlPFC, demonstrated that we could predict individual differences in working memory ability from the pattern of activity associated with emotion regulation. These findings support the idea that emotion regulation shares underlying cognitive processes and neural mechanisms with working memory, particularly in the dlPFC.

Neural predictors of fear depend on the situation.

The extent to which neural representations of fear experience depend on or generalize across the situational context has remained unclear. We systematically manipulated variation within and across three distinct fear-evocative situations including fear of heights, spiders, and social threats. Participants (n=21, 10 females and 11 males) viewed ∼20 second clips depicting spiders, heights, or social encounters, and rated fear after each video. Searchlight multivoxel pattern analysis (MVPA) was used to identify whether and which brain regions carry information that predicts fear experience, and the degree to which the fear-predictive neural codes in these areas depend upon or generalize across the situations. The overwhelming majority of brain regions carrying information about fear did so in a situation dependent manner. These findings suggest that local neural representations of fear experience are unlikely to involve a singular pattern, but rather a collection of multiple heterogeneous brain states.Significance Statement Much of the debate on the nature of emotion concerns the uniformity or heterogeneity of representation for particular emotion categories. Here we provide evidence that widely distributed activation patterns characteristic of recent neural signatures of fear reflect an amalgam of functionally heterogeneous brain states. Participants completed a novel fMRI task that parametrically examined subjective fear within and across three content-rich and naturalistic situations: fear of heights, spiders, and social threats. Using searchlight analysis and machine learning methods, we show that the overwhelming majority of brain regions that predict fear only do so for certain situations. These findings carry implications for the generalization of findings on fear across species, translational models of fear and anxiety, and developing neural signatures of fear.

Similar patterns of brain activity for holistic representation of working memory: a study in children and adults.

Working memory is the fundamental function of the various cognitive processes and abilities in the overall trajectory of development. Significant advances in multivariate analysis of human functional magnetic resonance imaging data have converged functional segregation models toward integrated representation-based models. However, due to the inherent limitations of the multi-voxel pattern analysis method, we are unable to determine whether the underlying neural representations are spatially similar in the brain. Our study attempts to answer this question by examining the spatial similarity of brain activity during the working memory task in children and adults. Our results reveal similar patterns of activity between the regions involved in working memory. This functional network of similar spatial patterns was observed in both normally developing children and adults. However, the between-region similarity was more pronounced in adults than in children and associated with better performance. We propose an exchange of similar information flows through the brain at an integrated level of working memory processes, underpinning the holistic nature of working memory representation.

Patterns of language

ABSTRACT Neuroimaging studies have increasingly leveraged the information in multivariate patterns of brain activity, transitioning from voxel-by-voxel activation comparisons to multi-voxel pattern analysis (MVPA) and decoding approaches. Representational Similarity Analysis (RSA) has further advanced this field, enabling comparisons of representational structures across neuroimaging modalities and computational models, in particular in fMRI and EEG/MEG research. Recent applications have extended to brain connectivity estimation, enhancing our understanding of neural representations in language and cognition. This special issue, “Patterns Of Language”, showcases recent applications and methodological developments of multivariate analysis approaches in the neuroscience of language.

Longitudinal stability of individual brain plasticity patterns in blindness

The primary visual cortex (V1) in blindness is engaged in a wide spectrum of tasks and sensory modalities, including audition, touch, language, and memory. This widespread involvement raises questions regarding the constancy of its role and whether it might exhibit flexibility in its function over time, connecting to diverse network functions specific to task demands. This would suggest that reorganized V1 assumes a role like multiple-demand system regions. Alternatively, varying patterns of plasticity in blind V1 may be attributed to individual factors, with different blind individuals recruiting V1 preferentially for different functions. In support of this, we recently showed that V1 functional connectivity (FC) varies greatly across blind individuals. But do these represent stable individual patterns of plasticity, or are they driven more by instantaneous changes, like a multiple-demand system now inhabiting V1? Here, we tested whether individual FC patterns from the V1 of blind individuals are stable over time. We show that over two years, FC from the V1 is unique and highly stable in a small sample of repeatedly sampled congenitally blind individuals. Further, using multivoxel pattern analysis, we demonstrate that the unique reorganization patterns of these individuals allow decoding of participant identity. Together with recent evidence for substantial individual differences in V1 connectivity, this indicates that there may be a consistent role for V1 in blindness, which may differ for each individual. Further, it suggests that the variability in visual reorganization in blindness across individuals could be used to seek stable neuromarkers for sight rehabilitation and assistive approaches.

The role of task on the human brain's responses to, and representation of, visual regularity defined by reflection and rotation

Identifying and segmenting objects in an image is generally achieved effortlessly and is facilitated by the presence of symmetry: a principle of perceptual organisation used to interpret sensory inputs from the retina into meaningful representations. However, while imaging studies show evidence of symmetry selective responses across extrastriate visual areas in the human brain, whether symmetry is processed automatically is still under debate. We used functional Magnetic Resonance Imaging (fMRI) to study the response to and representation of two types of symmetry: reflection and rotation. Dot pattern stimuli were presented to 15 human participants (10 female) under stimulus-relevant (symmetry) and stimulus-irrelevant (luminance) task conditions. Our results show that symmetry-selective responses emerge from area V3 and extend throughout extrastriate visual areas. This response is largely maintained when participants engage in the stimulus irrelevant task, suggesting an automaticity to processing visual symmetry. Our multi-voxel pattern analysis (MVPA) results extend these findings by suggesting that not only spatial organisation of responses to symmetrical patterns can be distinguished from that of non-symmetrical (random) patterns, but also that representation of reflection and rotation symmetry can be differentiated in extrastriate and object-selective visual areas. Moreover, task demands did not affect the neural representation of the symmetry information. Intriguingly, our MVPA results show an interesting dissociation: representation of luminance (stimulus irrelevant feature) is maintained in visual cortex only when task relevant, while information of the spatial configuration of the stimuli is available across task conditions. This speaks in favour of the automaticity for processing perceptual organisation: extrastriate visual areas compute and represent global, spatial properties irrespective of the task at hand.

Fixating targets in visual search: The role of dorsal and ventral attention networks in the processing of relevance and rarity

Abstract The dorsal attention network, often observed to be activated in serial visual search tasks, has been associated with goal-directed attention, responsible for the processing of task relevance. In serial visual search, the moment of target detection constitutes not only a task-relevant event, but also a rare event. In the present fMRI experiment, we disentangled task relevance from item rarity using a fixation-based analysis approach. We used a multiple target search task, and participants had to report the number of targets among distractors in the display. We had also added rare distractors to the displays. We found that rare events (targets and rare distractors) activated the dorsal attention network more strongly than common distractors. More importantly, we observed that the left IPS and the left insula, belonging to the dorsal and ventral attention system, respectively, were more strongly activated for targets compared to rare distractors. Using multi-voxel pattern analysis, we found that activation in the TPJ, bilaterally, an area also associated with the ventral attention system, distinguished between target and rare distractor fixations. These results point to an expanded role of the TPJ that seems to process post-perceptual information which is linked to task relevance.

Interaction between the prefrontal and visual cortices supports subjective fear.

It has been reported that threatening and non-threatening visual stimuli can be distinguished based on the multi-voxel patterns of haemodynamic activity in the human ventral visual stream. Do these findings mean that there may be evolutionarily hardwired mechanisms within early perception, for the fast and automatic detection of threat, and maybe even for the generation of the subjective experience of fear? In this human neuroimaging study, we presented participants ('fear' group: N = 30; 'no fear' group: N = 30) with 2700 images of animals that could trigger subjective fear or not as a function of theindividual's idiosyncratic 'fear profiles' (i.e. fear ratings of animals reported by a given participant). We provide evidence that the ventral visual stream may represent affectively neutral visual features that are statistically associated with fear ratings of participants, without representing the subjective experience of fear itself. More specifically, we show that patterns of haemodynamic activity predictive of a specific 'fear profile' can be observed in the ventral visual stream whether a participant reports being afraid of the stimuli or not. Further, we found that the multivariate information synchronization between ventral visual areas and prefrontal regions distinguished participants who reported being subjectively afraid of the stimuli from those who did not. Together, these findings support the view that the subjective experience of fear may depend on the relevant visual information triggering implicit metacognitive mechanisms in the prefrontal cortex. This article is part of the theme issue 'Sensing and feeling: an integrative approach to sensory processing and emotional experience'.

From long-term to short-term: Distinct neural networks underlying semantic knowledge and its recruitment in working memory

Although numerous studies suggest that working memory (WM) and semantic long-term knowledge interact, the nature and underlying neural mechanisms of this intervention remain poorly understood. Using functional magnetic resonance imaging (fMRI), this study investigated the extent to which neural markers of semantic knowledge in long-term memory (LTM) are activated during the WM maintenance stage in 32 young adults. First, the multivariate neural patterns associated with four semantic categories were determined via an implicit semantic activation task. Next, the participants maintained words – the names of the four semantic categories implicitly activated in the first task – in a verbal WM task. Multi-voxel pattern analyses showed reliable neural decoding of the four semantic categories in the implicit semantic activation and the verbal WM tasks. Critically, however, no between-task classification of semantic categories was observed. Searchlight analyses showed that for the WM task, semantic category information could be decoded in anterior temporal areas associated with abstract semantic category knowledge. In the implicit semantic activation task, semantic category information was decoded in superior temporal, occipital and frontal cortices associated with domain-specific semantic feature representations. These results indicate that item-level semantic activation during verbal WM involves shallow rather than deep semantic information.

A scene with an invisible wall - navigational experience shapes visual scene representation.

Human navigation heavily relies on visual information. Although many previous studies have investigated how navigational information is inferred from visual features of scenes, little is understood about the impact of navigational experience on visual scene representation. In this study, we examined how navigational experience influences both the behavioral and neural responses to a visual scene. During training, participants navigated in the virtual reality (VR) environments which we manipulated navigational experience while holding the visual properties of scenes constant. Half of the environments allowed free navigation (navigable), while the other half featured an 'invisible wall' preventing the participants to continue forward even though the scene was visually navigable (non-navigable). During testing, participants viewed scene images from the VR environment while completing either a behavioral perceptual identification task (Experimentl) or an fMRI scan (Experiment2). Behaviorally, we found that participants judged a scene pair to be significantly more visually different if their prior navigational experience varied, even after accounting for visual similarities between the scene pairs. Neurally, multi-voxel pattern of the parahippocampal place area (PPA) distinguished visual scenes based on prior navigational experience alone. These results suggest that the human visual scene cortex represents information about navigability obtained through prior experience, beyond those computable from the visual properties of the scene. Taken together, these results suggest that scene representation is modulated by prior navigational experience to help us construct a functionally meaningful visual environment.

MRI resting-state signature of the propensity to experience meaningful coincidences: a functional coupling analysis.

The propensity to experience meaningful patterns in random arrangements and unrelated events shows considerable interindividual differences. Reduced inhibitory control (over sensory processes) and decreased working memory capacities are associated with this trait, which implies that the activation of frontal as well as posterior brain regions may be altered during rest and working memory tasks. In addition, people experiencing more meaningful coincidences showed reduced gray matter of the left inferior frontal gyrus (IFG), which is linked to the inhibition of irrelevant information in working memory and the control and integration of multisensory information. To study deviations in the functional connectivity of the IFG with posterior associative areas, the present study investigated the fMRI resting state in a large sample of n = 101 participants. We applied seed-to-voxel analysis and found that people who perceive more meaningful coincidences showed negative functional connectivity of the left IFG (i.e. pars triangularis) with areas of the left posterior associative cortex (e.g. superior parietal cortex). A data-driven multivoxel pattern analysis further indicated that functional connectivity of a cluster located in the right cerebellum with a cluster including parts of the left middle frontal gyrus, left precentral gyrus, and the left IFG (pars opercularis) was associated with meaningful coincidences. These findings add evidence to the neurocognitive foundations of the propensity to experience meaningful coincidences, which strengthens the idea that deviations of working memory functions and inhibition of sensory and motor information explain why people experience more meaning in meaningless noise.

What makes an event significant: an fMRI study on self-defining memories.

Self-defining memories are highly significant personal memories that contribute to an individual's life story and identity. Previous research has identified 4 key subcomponents of self-defining memories: content, affect, specificity, and self-reflection. However, these components were not tested under functional neuroimaging. In this study, we first explored how self-defining memories distinguish themselves from everyday memories (non-self-defining) through their associated brain activity. Next, we evaluated the different self-defining memory subcomponents through their activity in the underlying brain system. Participants recalled both self-defining and non-self-defining memories under functional MRI and evaluated the 4 subcomponents for each memory. Multivoxel pattern analysis uncovered a brain system closely related to the default mode network to discriminate between self-defining and non-self-defining memories. Representational similarity analysis revealed the neural coding of each subcomponent. Self-reflection was coded mainly in the precuneus, middle and inferior frontal gyri, and cingulate, lateral occipital, and insular cortices. To a much lesser extent, content coding was primarily in the left angular gyrus and fusiform gyrus. No region was found to represent information on affect and specificity. Our findings highlight the marked difference in brain processing between significant and non-significant memories, and underscore self-reflection as a predominant factor in the formation and maintenance of self-defining memories, inviting a reassessment of what constitutes significant memories.

Memory traces of duration and location in the right intraparietal sulcus

Time and space form an integral part of every human experience, and for the neuronal representation of these perceptual dimensions, previous studies point to the involvement of the right-hemispheric intraparietal sulcus and structures in the medial temporal lobe. Here we used multi-voxel pattern analysis (MVPA) to investigate long-term memory traces for temporal and spatial stimulus features in those areas. Participants were trained on four images associated with short versus long durations and with left versus right locations. Our results demonstrate stable representations of both temporal and spatial information in the right posterior intraparietal sulcus. Building upon previous findings of stable neuronal codes for directly perceived durations and locations, these results show that the reactivation of long-term memory traces for temporal and spatial features can be decoded from neuronal activation patterns in the right parietal cortex.

Normativity vs. uniqueness: effects of social relationship strength on neural representations of others.

Understanding others involves inferring traits and intentions, a process complicated by our reliance on stereotypes and generalized information when we lack personal information. Yet, as relationships are formed, we shift toward nuanced and individualized perceptions of others. This study addresses how relationship strength influences the creation of unique or normative representations of others in key regions known to be involved in social cognition. Employing a round-robin interpersonal perception paradigm (N = 111, 20 groups of five to six people), we used functional magnetic resonance imaging to examine whether the strength of social relationships modulated the degree to which multivoxel patterns of activity that represented a specific other were similar to a normative average of all others in the study. Behaviorally, stronger social relationships were associated with more normative trait endorsements. Neural findings reveal that closer relationships lead to more unique representations in the medial prefrontal cortex and anterior insula, areas associated with mentalizing and person perception. Conversely, more generalized representations emerge in posterior regions like the posterior cingulate cortex, indicating a complex interplay between individuated and generalized processing of social information in the brain. These findings suggest that cortical regions typically associated with social cognition may compute different kinds of information when representing the distinctiveness of others.

The relationship between event boundary strength and pattern shifts across the cortical hierarchy during naturalistic movie-viewing.

Our continuous experience is spontaneously segmented by the brain into discrete events. However, the beginning of a new event (an event boundary) is not always sharply identifiable: phenomenologically, event boundaries vary in salience. How are the response profiles of cortical areas at event boundaries modulated by boundary strength during complex, naturalistic movie-viewing? Do cortical responses scale in a graded manner with boundary strength, or do they merely detect boundaries in a binary fashion? We measured "cortical boundary shifts" as transient changes in multi-voxel patterns at event boundaries with different strengths (weak, moderate, and strong), determined by across-subject agreement. Cortical regions with different processing timescales were examined. In auditory areas, which have short timescales, cortical boundary shifts exhibited a clearly graded profile both in group-level and individual-level analyses. In cortical areas with long timescales, including the default mode network, boundary strength modulated pattern shift magnitude at the individual subject level. We also observed a positive relationship between boundary strength and the extent of temporal alignment of boundary shifts across different levels of the cortical hierarchy. Additionally, hippocampal activity was highest at event boundaries for which cortical boundary shifts were most aligned across hierarchical levels. Overall, we found that event boundary strength modulated cortical pattern shifts strongly in sensory areas and more weakly in higher-level areas, and that stronger boundaries were associated with greater alignment of these shifts across the cortical hierarchy.