Multivoxel Pattern Classification Research Articles

Real-Time Classification of Motor Imagery Using Dynamic Window-Level Granger Causality Analysis of fMRI Data.

This article presents a method for extracting neural signal features to identify the imagination of left- and right-hand grasping movements. A functional magnetic resonance imaging (fMRI) experiment is employed to identify four brain regions with significant activations during motor imagery (MI) and the effective connections between these regions of interest (ROIs) were calculated using Dynamic Window-level Granger Causality (DWGC). Then, a real-time fMRI (rt-fMRI) classification system for left- and right-hand MI is developed using the Open-NFT platform. We conducted data acquisition and processing on three subjects, and all of whom were recruited from a local college. As a result, the maximum accuracy of using Support Vector Machine (SVM) classifier on real-time three-class classification (rest, left hand, and right hand) with effective connections is 69.3%. And it is 3% higher than that of traditional multivoxel pattern classification analysis on average. Moreover, it significantly improves classification accuracy during the initial stage of MI tasks while reducing the latency effects in real-time decoding. The study suggests that the effective connections obtained through the DWGC method serve as valuable features for real-time decoding of MI using fMRI. Moreover, they exhibit higher sensitivity to changes in brain states. This research offers theoretical support and technical guidance for extracting neural signal features in the context of fMRI-based studies.

Auditory Target Detection Enhances Visual Processing and Hippocampal Functional Connectivity.

Though dividing one’s attention between two input streams typically impairs performance, detecting a behaviorally relevant stimulus can sometimes enhance the encoding of unrelated information presented at the same time. Previous research has shown that selection of this kind boosts visual cortical activity and memory for concurrent items. An important unanswered question is whether such effects are reflected in processing quality and functional connectivity in visual regions and in the hippocampus. In this fMRI study, participants were asked to memorize a stream of naturalistic images and press a button only when they heard a predefined target tone (400 or 1,200 Hz, counterbalanced). Images could be presented with a target tone, with a distractor tone, or without a tone. Auditory target detection increased activity throughout the ventral visual cortex but lowered it in the hippocampus. Enhancements in functional connectivity between the ventral visual cortex and the hippocampus were also observed following auditory targets. Multi-voxel pattern classification of image category was more accurate on target tone trials than on distractor and no tone trials in the fusiform gyrus and parahippocampal gyrus. This effect was stronger in visual cortical clusters whose activity was more correlated with the hippocampus on target tone than on distractor tone trials. In agreement with accounts suggesting that subcortical noradrenergic influences play a role in the attentional boost effect, auditory target detection also caused an increase in locus coeruleus activity and phasic pupil responses. These findings outline a network of cortical and subcortical regions that are involved in the selection and processing of information presented at behaviorally relevant moments.

Overlapping Functional Representations of Self- and Other-Related Thought are Separable Through Multivoxel Pattern Classification.

Self-reflection and thinking about the thoughts and behaviors of others are important skills for humans to function in the social world. These two processes overlap in terms of the component processes involved, and share overlapping functional organizations within the human brain, in particular within the medial prefrontal cortex (MPFC). Several functional models have been proposed to explain these two processes, but none has directly explored the extent to which they are distinctly represented within different parts of the brain. This study used multivoxel pattern classification to quantify the separability of self- and other-related thought in the MPFC and expanded this question to the entire brain. Using a large-scale mega-analytic dataset, spanning three separate studies (n = 142), we find that self- and other-related thought can be reliably distinguished above chance within the MPFC, posterior cingulate cortex and temporal lobes. We highlight subcomponents of the ventral MPFC that are particularly important in representing self-related thought, and subcomponents of the orbitofrontal cortex robustly involved in representing other-related thought. Our findings indicate that representations of self- and other-related thought in the human brain are described best by a distributed pattern rather than stark localization or a purely ventral to dorsal linear gradient in the MPFC.

Neural encoding of task-dependent errors during adaptive learning

Effective learning requires using errors in a task-dependent manner, for example adjusting to errors that result from unpredicted environmental changes but ignoring errors that result from environmental stochasticity. Where and how the brain represents errors in a task-dependent manner and uses them to guide behavior are not well understood. We imaged the brains of human participants performing a predictive-inference task with two conditions that had different sources of errors. Their performance was sensitive to this difference, including more choice switches after fundamental changes versus stochastic fluctuations in reward contingencies. Using multi-voxel pattern classification, we identified task-dependent representations of error magnitude and past errors in posterior parietal cortex. These representations were distinct from representations of the resulting behavioral adjustments in dorsomedial frontal, anterior cingulate, and orbitofrontal cortex. The results provide new insights into how the human brain represents errors in a task-dependent manner and guides subsequent adaptive behavior.

Coding of Navigational Distance and Functional Constraint of Boundaries in the Human Scene-Selective Cortex.

For visually guided navigation, the use of environmental cues is essential. Particularly, detecting local boundaries that impose limits to locomotion and estimating their location is crucial. In a series of three fMRI experiments, we investigated whether there is a neural coding of navigational distance in the human visual cortex (both female and male). We used virtual reality software to systematically manipulate the distance from a viewer perspective to different types of a boundary. Using a multivoxel pattern classification employing a linear support vector machine, we found that the occipital place area (OPA) is sensitive to the navigational distance restricted by the transparent glass wall. Further, the OPA was sensitive to a non-crossable boundary only, suggesting an importance of the functional constraint of a boundary. Together, we propose the OPA as a perceptual source of external environmental features relevant for navigation.SIGNIFICANCE STATEMENT One of major goals in cognitive neuroscience has been to understand the nature of visual scene representation in human ventral visual cortex. An aspect of scene perception that has been overlooked despite its ecological importance is the analysis of space for navigation. One of critical computation necessary for navigation is coding of distance to environmental boundaries that impose limit on navigator's movements. This paper reports the first empirical evidence for coding of navigational distance in the human visual cortex and its striking sensitivity to functional constraint of environmental boundaries. Such finding links the paper to previous neurological and behavioral works that emphasized the distance to boundaries as a crucial geometric property for reorientation behavior of children and other animal species.

Aberrant working memory processing in major depression: evidence from multivoxel pattern classification.

Major depressive disorder (MDD) is often accompanied by severe impairments in working memory (WM). Neuroimaging studies investigating the mechanisms underlying these impairments have produced conflicting results. It remains unclear whether MDD patients show hyper- or hypoactivity in WM-related brain regions and how potential aberrations in WM processing may contribute to the characteristic dysregulation of cognition-emotion interactions implicated in the maintenance of the disorder. In order to shed light on these questions and to overcome limitations of previous studies, we applied a multivoxel pattern classification approach to investigate brain activity in large samples of MDD patients (N = 57) and matched healthy controls (N = 61) during a WM task that incorporated positive, negative, and neutral stimuli. Results showed that patients can be distinguished from healthy controls with good classification accuracy based on functional activation patterns. ROI analyses based on the classification weight maps showed that during WM, patients had higher activity in the left DLPFC and the dorsal ACC. Furthermore, regions of the default-mode network (DMN) were less deactivated in patients. As no performance differences were observed, we conclude that patients required more effort, indexed by more activity in WM-related regions, to successfully perform the task. This increased effort might be related to difficulties in suppressing task-irrelevant information reflected by reduced deactivation of regions within the DMN. Effects were most pronounced for negative and neutral stimuli, thus pointing toward important implications of aberrations in WM processes in cognition-emotion interactions in MDD.

Multi-voxel pattern classification differentiates personally experienced event memories from secondhand event knowledge

Studies of autobiographical memory retrieval often use photographs to probe participants' memories for past events. Recent neuroimaging work has shown that viewing photographs depicting events from one's own life evokes a characteristic pattern of brain activity across a network of frontal, parietal, and medial temporal lobe regions that can be readily distinguished from brain activity associated with viewing photographs from someone else's life (Rissman, Chow, Reggente, and Wagner, 2016). However, it is unclear whether the neural signatures associated with remembering a personally experienced event are distinct from those associated with recognizing previously encountered photographs of an event. The present experiment used a novel functional magnetic resonance imaging (fMRI) paradigm to investigate putative differences in brain activity patterns associated with these distinct expressions of memory retrieval. Eighteen participants wore necklace-mounted digital cameras to capture events from their everyday lives over the course of three weeks. One week later, participants underwent fMRI scanning, where on each trial they viewed a sequence of photographs depicting either an event from their own life or from another participant's life and judged their memory for this event. Importantly, half of the trials featured photographic sequences that had been shown to participants during a laboratory session administered the previous day. Multi-voxel pattern analyses assessed the sensitivity of two brain networks of interest—as identified by a meta-analysis of prior autobiographical and laboratory-based memory retrieval studies—to the original source of the photographs (own life or other's life) and their experiential history as stimuli (previewed or non-previewed). The classification analyses revealed a striking dissociation: activity patterns within the autobiographical memory network were significantly more diagnostic than those within the laboratory-based network as to whether photographs depicted one's own personal experience (regardless of whether they had been previously seen), whereas activity patterns within the laboratory-based memory network were significantly more diagnostic than those within the autobiographical memory network as to whether photographs had been previewed (regardless of whether they were from the participant's own life). These results, also apparent in whole-brain searchlight classifications, provide evidence for dissociable patterns of activation across two putative memory networks as a function of whether real-world photographs trigger the retrieval of firsthand experiences or secondhand event knowledge.

How landmark suitability shapes recognition memory signals for objects in the medial temporal lobes

A role of perirhinal cortex (PrC) in recognition memory for objects has been well established. Contributions of parahippocampal cortex (PhC) to this function, while documented, remain less well understood. Here, we used fMRI to examine whether the organization of item-based recognition memory signals across these two structures is shaped by object category, independent of any difference in representing episodic context. Guided by research suggesting that PhC plays a critical role in processing landmarks, we focused on three categories of objects that differ from each other in their landmark suitability as confirmed with behavioral ratings (buildings > trees > aircraft). Participants made item-based recognition-memory decisions for novel and previously studied objects from these categories, which were matched in accuracy. Multi-voxel pattern classification revealed category-specific item-recognition memory signals along the long axis of PrC and PhC, with no sharp functional boundaries between these structures. Memory signals for buildings were observed in the mid to posterior extent of PhC, signals for trees in anterior to posterior segments of PhC, and signals for aircraft in mid to posterior aspects of PrC and the anterior extent of PhC. Notably, item-based memory signals for the category with highest landmark suitability ratings were observed only in those posterior segments of PhC that also allowed for classification of landmark suitability of objects when memory status was held constant. These findings provide new evidence in support of the notion that item-based memory signals for objects are not limited to PrC, and that the organization of these signals along the longitudinal axis that crosses PrC and PhC can be captured with reference to landmark suitability.

Distributed Neural Processing Predictors of Multi-dimensional Properties of Affect.

Recent evidence suggests that emotions have a distributed neural representation, which has significant implications for our understanding of the mechanisms underlying emotion regulation and dysregulation as well as the potential targets available for neuromodulation-based emotion therapeutics. This work adds to this evidence by testing the distribution of neural representations underlying the affective dimensions of valence and arousal using representational models that vary in both the degree and the nature of their distribution. We used multi-voxel pattern classification (MVPC) to identify whole-brain patterns of functional magnetic resonance imaging (fMRI)-derived neural activations that reliably predicted dimensional properties of affect (valence and arousal) for visual stimuli viewed by a normative sample (n = 32) of demographically diverse, healthy adults. Inter-subject leave-one-out cross-validation showed whole-brain MVPC significantly predicted (p < 0.001) binarized normative ratings of valence (positive vs. negative, 59% accuracy) and arousal (high vs. low, 56% accuracy). We also conducted group-level univariate general linear modeling (GLM) analyses to identify brain regions whose response significantly differed for the contrasts of positive versus negative valence or high versus low arousal. Multivoxel pattern classifiers using voxels drawn from all identified regions of interest (all-ROIs) exhibited mixed performance; arousal was predicted significantly better than chance but worse than the whole-brain classifier, whereas valence was not predicted significantly better than chance. Multivoxel classifiers derived using individual ROIs generally performed no better than chance. Although performance of the all-ROI classifier improved with larger ROIs (generated by relaxing the clustering threshold), performance was still poorer than the whole-brain classifier. These findings support a highly distributed model of neural processing for the affective dimensions of valence and arousal. Finally, joint error analyses of the MVPC hyperplanes encoding valence and arousal identified regions within the dimensional affect space where multivoxel classifiers exhibited the greatest difficulty encoding brain states – specifically, stimuli of moderate arousal and high or low valence. In conclusion, we highlight new directions for characterizing affective processing for mechanistic and therapeutic applications in affective neuroscience.

I can see where you would be: Patterns of fMRI activity reveal imagined landmarks

Visual mental imagery arises when perceptual information is accessed from memory, originating the experience of “seeing with the mind's eye”. Different content-dependent brain areas in the human ventral visual stream are activated during visual mental imagery, similarly to what happens during visual perception. The neural patterns within these regions, but not in the early visual cortex, are similar during imagery and perception, suggesting that, in the absence of perceptual stimulation, content-dependent brain areas are able to re-instantiate specific neural patterns allowing for mental imagery. However, it remains unknown whether these areas contain adequate neural representations that create mental images or need to interact with other regions in the brain, such as the hippocampus (HC), to access the necessary information from memory. To test this hypothesis, we used functional magnetic resonance imaging and both multivoxel pattern classification and psychophysiological interaction analyses. Participants were scanned while viewing or imagining scenes of familiar environments. We found that the identity of familiar places can be decoded from the neural patterns in the parahippocampal place area (PPA), retrosplenial complex/parieto-occipital sulcus (RSC/POS) and HC, during both imagery and perception, and that item-specific information from perceived places was re-instantiated during mental imagery of the same places and vice versa. Furthermore, the right PPA significantly interacted with the right HC and RSC/POS according to the performed task. Specifically, the functional coupling between PPA and HC was higher during mental imagery, whereas the functional coupling between PPA and RSC/POS was higher during perception. Our investigation provides an important contribution to the understanding of how the brain uses previously acquired knowledge to build a mental representation of the world.

Increased activity in frontal motor cortex compensates impaired speech perception in older adults.

Understanding speech in noisy environments is challenging, especially for seniors. Although evidence suggests that older adults increasingly recruit prefrontal cortices to offset reduced periphery and central auditory processing, the brain mechanisms underlying such compensation remain elusive. Here we show that relative to young adults, older adults show higher activation of frontal speech motor areas as measured by functional MRI during a syllable identification task at varying signal-to-noise ratios. This increased activity correlates with improved speech discrimination performance in older adults. Multivoxel pattern classification reveals that despite an overall phoneme dedifferentiation, older adults show greater specificity of phoneme representations in frontal articulatory regions than auditory regions. Moreover, older adults with stronger frontal activity have higher phoneme specificity in frontal and auditory regions. Thus, preserved phoneme specificity and upregulation of activity in speech motor regions provide a means of compensation in older adults for decoding impoverished speech representations in adverse listening conditions.

Age-related differences in the neural basis of the subjective vividness of memories: evidence from multivoxel pattern classification.

Although older adults often show reduced episodic memory accuracy, their ratings of the subjective vividness of their memories often equal or even exceed those of young adults. Such findings suggest that young and older adults may differentially access and/or weight different kinds of information in making vividness judgments. We examined this idea using multivoxel pattern classification of fMRI data to measure category representations while participants saw and remembered pictures of objects and scenes. Consistent with our hypothesis, there were age-related differences in how category representations related to the subjective sense of vividness. During remembering, older adults' vividness ratings were more related, relative to young adults', to category representations in prefrontal cortex. In contrast, young adults' vividness ratings were more related, relative to older adults, to category representations in parietal cortex. In addition, category representations were more correlated among posterior regions in young than in older adults, whereas correlations between PFC and posterior regions did not differ between the 2 groups. Together, these results are consistent with the idea that young and older adults differentially weight different types of information in assessing subjective vividness of their memories.

A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes

Recent multi-voxel pattern classification (MVPC) studies have shown that in early visual cortex patterns of brain activity generated during mental imagery are similar to patterns of activity generated during perception. This finding implies that low-level visual features (e.g., space, spatial frequency, and orientation) are encoded during mental imagery. However, the specific hypothesis that low-level visual features are encoded during mental imagery is difficult to directly test using MVPC. The difficulty is especially acute when considering the representation of complex, multi-object scenes that can evoke multiple sources of variation that are distinct from low-level visual features. Therefore, we used a voxel-wise modeling and decoding approach to directly test the hypothesis that low-level visual features are encoded in activity generated during mental imagery of complex scenes. Using fMRI measurements of cortical activity evoked by viewing photographs, we constructed voxel-wise encoding models of tuning to low-level visual features. We also measured activity as subjects imagined previously memorized works of art. We then used the encoding models to determine if putative low-level visual features encoded in this activity could pick out the imagined artwork from among thousands of other randomly selected images. We show that mental images can be accurately identified in this way; moreover, mental image identification accuracy depends upon the degree of tuning to low-level visual features in the voxels selected for decoding. These results directly confirm the hypothesis that low-level visual features are encoded during mental imagery of complex scenes. Our work also points to novel forms of brain–machine interaction: we provide a proof-of-concept demonstration of an internet image search guided by mental imagery.

Selective diagnostic discrimination of schizophrenia across cognitive control brain regions using multi-voxel pattern classification

Selective diagnostic discrimination of schizophrenia across cognitive control brain regions using multi-voxel pattern classification

Distributed cognitive maps reflecting real distances between places and views in the human brain.

Keeping oriented in the environment is a multifaceted ability that requires knowledge of at least three pieces of information: one’s own location (“place”) and orientation (“heading”) within the environment, and which location in the environment one is looking at (“view”). We used functional magnetic resonance imaging (fMRI) in humans to examine the neural signatures of these information. Participants were scanned while viewing snapshots which varied for place, view and heading within a virtual room. We observed adaptation effects, proportional to the physical distances between consecutive places and views, in scene-responsive (retrosplenial complex and parahippocampal gyrus), fronto-parietal and lateral occipital regions. Multivoxel pattern classification of signals in scene-responsive regions and in the hippocampus allowed supra-chance decoding of place, view and heading, and revealed the existence of map-like representations, where places and views closer in physical space entailed activity patterns more similar in neural representational space. The pattern of hippocampal activity reflected both view- and place-based distances, the pattern of parahippocampal activity preferentially discriminated between views, and the pattern of retrosplenial activity combined place and view information, while the fronto-parietal cortex only showed transient effects of changes in place, view, and heading. Our findings provide evidence for the presence of map-like spatial representations which reflect metric distances in terms of both one’s own and landmark locations.

Encoding of Prospective Tasks in the Human Prefrontal Cortex under Varying Task Loads

Successful realization of planned actions requires the brain to encode intentions over delays. Previous research has indicated that several regions in the rostral or anterior prefrontal cortex (PFC) encode delayed intentions. However, different processes may encode the same future task depending on task load during the delay. This difference may depend on the computational resources available when the delay is occupied with an ongoing task and when it is task-free. Here we directly investigated and compared the representation of delayed intentions in the human brain in the presence and absence of ongoing task load during the delay. We acquired fMRI data in combination with an event-based prospective memory design where human subjects remembered to perform the same future tasks over occupied and task-free delays. We used time-resolved multivoxel pattern classification and found that: (1) rostrolateral PFC (BA 46) encoded the delayed intention during both delay types; (2) rostromedial PFC (BA 10) encoded the intentions during occupied delays; whereas (3) a variety of more posterior regions, including the anterior cingulate cortex (BA 24), the supplementary motor area (BA 6), and the precuneus, encoded intentions during task-free delays. Overall, the medial PFC encoded delayed intentions more rostrally in the presence of an ongoing delay task and more caudally in its absence. Thus, rostromedial PFC may play a specialized role in the encoding of prospective memory that depends on higher computational demands (e.g., given higher task load during the delay). In contrast, the rostrolateral PFC is a more general area that encodes future intentions regardless of task load.

Choosing the rules: distinct and overlapping frontoparietal representations of task rules for perceptual decisions.

Behavior is governed by rules that associate stimuli with responses and outcomes. Human and monkey studies have shown that rule-specific information is widely represented in the frontoparietal cortex. However, it is not known how establishing a rule under different contexts affects its neural representation. Here, we use event-related functional MRI (fMRI) and multivoxel pattern classification methods to investigate the human brain's mechanisms of establishing and maintaining rules for multiple perceptual decision tasks. Rules were either chosen by participants or specifically instructed to them, and the fMRI activation patterns representing rule-specific information were compared between these contexts. We show that frontoparietal regions differ in the properties of their rule representations during active maintenance before execution. First, rule-specific information maintained in the dorsolateral and medial frontal cortex depends on the context in which it was established (chosen vs specified). Second, rule representations maintained in the ventrolateral frontal and parietal cortex are independent of the context in which they were established. Furthermore, we found that the rule-specific coding maintained in anticipation of stimuli may change with execution of the rule: representations in context-independent regions remain invariant from maintenance to execution stages, whereas rule representations in context-dependent regions do not generalize to execution stage. The identification of distinct frontoparietal systems with context-independent and context-dependent task rule representations, and the distinction between anticipatory and executive rule representations, provide new insights into the functional architecture of goal-directed behavior.

Neural correlates of object-based attentional selection in human cortex

Humans can attend to different objects independent of their spatial locations. While selecting an object has been shown to modulate object processing in high-level visual areas in occipitotemporal cortex, where/how behavioral importance (i.e., priority) for objects is represented is unknown. Here we examined the patterns of distributed neural activity during an object-based selection task. We measured brain activity with functional magnetic resonance imaging (fMRI), while participants viewed two superimposed, dynamic objects (left- and right-pointing triangles) and were cued to attend to one of the triangle objects. Enhanced fMRI response was observed for the attention conditions compared to a neutral condition, but no significant difference was found in overall response amplitude between two attention conditions. By using multi-voxel pattern classification (MVPC), however, we were able to distinguish the neural patterns associated with attention to different objects in early visual cortex (V1 to hMT+) and lateral occipital complex (LOC). Furthermore, distinct multi-voxel patterns were also observed in frontal and parietal areas. Our results demonstrate that object-based attention has a wide-spread modulation effect along the visual hierarchy and suggest that object-specific priority information is represented by patterned neural activity in the dorsal frontoparietal network.

The Neural Basis of Visual Word Form Processing: A Multivariate Investigation

Current research on the neurobiological bases of reading points to the privileged role of a ventral cortical network in visual word processing. However, the properties of this network and, in particular, its selectivity for orthographic stimuli such as words and pseudowords remain topics of significant debate. Here, we approached this issue from a novel perspective by applying pattern-based analyses to functional magnetic resonance imaging data. Specifically, we examined whether, where and how, orthographic stimuli elicit distinct patterns of activation in the human cortex. First, at the category level, multivariate mapping found extensive sensitivity throughout the ventral cortex for words relative to false-font strings. Secondly, at the identity level, the multi-voxel pattern classification provided direct evidence that different pseudowords are encoded by distinct neural patterns. Thirdly, a comparison of pseudoword and face identification revealed that both stimulus types exploit common neural resources within the ventral cortical network. These results provide novel evidence regarding the involvement of the left ventral cortex in orthographic stimulus processing and shed light on its selectivity and discriminability profile. In particular, our findings support the existence of sublexical orthographic representations within the left ventral cortex while arguing for the continuity of reading with other visual recognition skills.

Pattern classification predicts individuals’ responses to affective stimuli

AbstractSince the successful demonstration of “brain reading” of fMRI BOLD signals using multivoxel pattern classification (MVPA) techniques, the neuroimaging community has made vigorous attempts to exploit the technique in order to identify the signature patterns of brain activities associated with different cognitive processes or mental states. In the current study, we tested whether the valence and arousal dimensions of the affective information could be used to successfully predict individual’s active affective states. Using a whole-brain MVPA approach, together with feature elimination procedures, we are able to discriminate between brain activation patterns associated with the processing of positive or negative valence and cross validate the discriminant function with an independent data set. Arousal information, on the other hand, failed to provide such discriminating power. With an independent sample, we test further whether the MVPA identified brain network could be used for inter-individual classification. Although the inter-subject classification success was only marginal, we found correlations with individual differences in affective processing. We discuss the implications of our findings for future attempts to classify patients based on their responses to affective stimuli.