fMRI Datasets Research Articles

Challenges in Multi-Task Learning for fMRI-Based Diagnosis: Benefits for Psychiatric Conditions and CNVs Would Likely Require Thousands of Patients

Abstract There is a growing interest in using machine learning (ML) models to perform automatic diagnosis of psychiatric conditions; however, generalising the prediction of ML models to completely independent data can lead to sharp decrease in performance. Patients with different psychiatric diagnoses have traditionally been studied independently, yet there is a growing recognition of neuroimaging signatures shared across them as well as rare genetic copy number variants (CNVs). In this work, we assess the potential of multi-task learning (MTL) to improve accuracy by characterising multiple related conditions with a single model, making use of information shared across diagnostic categories and exposing the model to a larger and more diverse dataset. As a proof of concept, we first established the efficacy of MTL in a context where there is clearly information shared across tasks: the same target (age or sex) is predicted at different sites of data collection in a large fMRI dataset compiled from multiple studies. MTL generally led to substantial gains relative to independent prediction at each site. Performing scaling experiments on the UK Biobank, we observed that performance was highly dependent on sample size: for large sample sizes (N>6000) sex prediction was better using MTL across three sites (N=K per site) than prediction at a single site (N=3K), but for small samples (N<500) MTL was actually detrimental for age prediction. We then used established machine learning methods to benchmark the diagnostic accuracy of each of the 7 CNVs (N=19-103) and 4 psychiatric conditions (N=44-472) independently, replicating the accuracy previously reported in the literature on psychiatric conditions. We observed that MTL hurt performance when applied across the full set of diagnoses, and complementary analyses failed to identify pairs of conditions which would benefit from MTL. Taken together, our results show that if a successful multi-task diagnostic model of psychiatric conditions were to be developed with resting-state fMRI, it would likely require datasets with thousands of patients across different diagnoses.

Probabilistic Presurgical Language fMRI Atlas of Patients with Brain Tumors.

Patients with brain tumors have high intersubject variation in putative language regions, which may limit the utility of straightforward application of healthy subject brain atlases in clinical scenarios. The purpose of this study was to develop a probabilistic functional brain atlas that consolidates language functional activations of sentence completion and Silent Word Generation language paradigms using a large sample of patients with brain tumors. The atlas was developed using retrospectively collected fMRI data from patients with brain tumors who underwent their first standard-of-care presurgical language fMRI scan at our institution between July 18, 2015, and May 13, 2022. Three hundred seventeen patients (861 fMRI scans) were used to develop the language functional atlas. An independent presurgical language fMRI data set of 39 patients with brain tumors from a previous study was used to evaluate our atlas. Family-wise error-corrected binary functional activation maps from sentence completion, letter fluency, and category fluency presurgical fMRI were used to create probability overlap maps and pooled probabilistic overlap maps in Montreal Neurological Institute standard space. The Wilcoxon signed-rank test was used to determine a significant difference in the maximum Dice coefficient for our atlas compared with a meta-analysis-based template with respect to expert-delineated primary language area activations. Probabilities of activating the left anterior primary language area and left posterior primary language area in the temporal lobe were 87.9% and 91.5%, respectively, for sentence completion, 88.5% and 74.2%, respectively, for letter fluency, and 83.6% and 67.6%, respectively, for category fluency. Maximum Dice coefficients for templates derived from our language atlas were significantly higher than the meta-analysis-based template in the left anterior primary language area (0.351 and 0.326, respectively, P < .05) and the left posterior primary language area in the temporal lobe (0.274 and 0.244, respectively, P < .005). Brain tumor patient- and paradigm-specific probabilistic language atlases were developed. These atlases had superior spatial agreement with fMRI activations in individual patients compared with the meta-analysis-based template.

Test-retest reliability of the attention network test from the perspective of intrinsic network organization.

The attention network test (ANT), developed based on the triple-network taxonomy by Posner and colleagues, has been widely used to examine the efficacy of alerting, orienting and executive control in clinical and developmental neuroscience studies. Recent research suggests the imperfect reliability of the behavioural ANT and its variants. However, the classical ANT fMRI task's test-retest reliability has received little attention. Moreover, it remains ambiguous whether the attention-related intrinsic network components, especially the dorsal attention, ventral attention and frontoparietal network, manifest acceptable reliability. The present study approaches these issues by utilizing an openly available ANT fMRI dataset for participants with Parkinson's disease and healthy elderly. The reproducibility of group-level activations across sessions and participant groups and the test-retest reliability at the individual level were examined at the voxel, region and network levels. The intrinsic network was defined using the Yeo-Schaefer atlas. Our results reveal three critical facets: (1) the overlapping of the group-level contrast map between sessions and between participant groups was unsatisfactory; (2) the reliability of alerting, orienting and executive, defined as a contrast between conditions, was worse than estimates of specific conditions. (3) Dorsal attention, ventral attention, visual and somatomotor networks showed acceptable reliability for the congruent and incongruent conditions. Our results suggest that specific condition estimates might be used instead of the contrast map for individual or group-difference studies.

A deep learning paradigm for medical imaging data

We present a novel deep learning framework to address several challenges in medical imaging analysis. Especially for functional magnetic resonance imaging (fMRI) data, we develop a statistical sampling algorithm to augment small data for deep learning modeling. Also, the ComBat harmonization is utilized for multi-source batch effect correction. Finally, we implement the prediction uncertainty in deep learning through a dropout approximation of the deep Gaussian process. The proposed approach offers effective statistical tools to solve persistent problems in deep learning modeling for medical imaging data. To assess the performance of the proposed approach and demonstrate its effectiveness for various types of data, we apply it to autism fMRI and Pneumonia imaging datasets.

Individual Brain Charting dataset extension, third release for movie watching and retinotopy data

The Individual Brain Charting (IBC) is a multi-task functional Magnetic Resonance Imaging dataset acquired at high spatial-resolution and dedicated to the cognitive mapping of the human brain. It consists in the deep phenotyping of twelve individuals, covering a broad range of psychological domains suitable for functional-atlasing applications. Here, we present the inclusion of task data from both naturalistic stimuli and trial-based designs, to uncover structures of brain activation. We rely on the Fast Shared Response Model (FastSRM) to provide a data-driven solution for modelling naturalistic stimuli, typically containing many features. We show that data from left-out runs can be reconstructed using FastSRM, enabling the extraction of networks from the visual, auditory and language systems. We also present the topographic organization of the visual system through retinotopy. In total, six new tasks were added to IBC, wherein four trial-based retinotopic tasks contributed with a mapping of the visual field to the cortex. IBC is open access: source plus derivatives imaging data and meta-data are available in public repositories.

Verbal semantic expertise is associated with reduced functional connectivity between left and right anterior temporal lobes.

The left and right anterior temporal lobes (ATLs) encode semantic representations. They show graded hemispheric specialization in function, with the left ATL contributing preferentially to verbal semantic processing. We investigated the cognitive correlates of this organization, using resting-state functional connectivity as a measure of functional segregation between ATLs. We analyzed two independent resting-state fMRI datasets (n= 86 and n= 642) in which participants' verbal semantic expertise was measured using vocabulary tests. In both datasets, people with more advanced verbal semantic knowledge showed weaker functional connectivity between left and right ventral ATLs. This effect was highly specific. It was not observed for within-hemisphere connections between semantic regions (ventral ATL and inferior frontal gyrus (IFG), though it was found for left-right IFG connectivity in one dataset). Effects were not found for tasks probing semantic control, nonsemantic cognition, or face recognition. Our results suggest that hemispheric specialization in the ATLs is not an innate property but rather emerges as people develop highly detailed verbal semantic representations. We speculate that this effect is a consequence of the left ATL's greater connectivity with left-lateralized written word recognition regions, which causes it to preferentially represent meaning for advanced vocabulary acquired primarily through reading.

Hypergraph-Based Multitask Feature Selection with Temporally Constrained Group Sparsity Learning on fMRI

Localizing the brain regions affected by tasks is crucial to understanding the mechanisms of brain function. However, traditional statistical analysis does not accurately identify the brain regions of interest due to factors such as sample size, task design, and statistical effects. Here, we propose a hypergraph-based multitask feature selection framework, referred to as HMTFS, which we apply to a functional magnetic resonance imaging (fMRI) dataset to extract task-related brain regions. HMTFS is characterized by its ability to construct a hypergraph through correlations between subjects, treating each subject as a node to preserve high-order information of time-varying signals. Additionally, it manages feature selection across different time windows in fMRI data as multiple tasks, facilitating time-constrained group sparse learning with a smoothness constraint. We utilize a large fMRI dataset from the Human Connectome Project (HCP) to validate the performance of HMTFS in feature selection. Experimental results demonstrate that brain regions selected by HMTFS can provide higher accuracy for downstream classification tasks compared to other competing feature selection methods and align with findings from previous neuroscience studies.

Anatomy-Guided Spatio-Temporal Graph Convolutional Networks (AG-STGCNs) for Modeling Functional Connectivity Between Gyri and Sulci Across Multiple Task Domains.

The cerebral cortex is folded as gyri and sulci, which provide the foundation to unveil anatomo-functional relationship of brain. Previous studies have extensively demonstrated that gyri and sulci exhibit intrinsic functional difference, which is further supported by morphological, genetic, and structural evidences. Therefore, systematically investigating the gyro-sulcal (G-S) functional difference can help deeply understand the functional mechanism of brain. By integrating functional magnetic resonance imaging (fMRI) with advanced deep learning models, recent studies have unveiled the temporal difference in functional activity between gyri and sulci. However, the potential difference of functional connectivity, which represents functional dependency between gyri and sulci, is much unknown. Moreover, the regularity and variability of the G-S functional connectivity difference across multiple task domains remains to be explored. To address the two concerns, this study developed new anatomy-guided spatio-temporal graph convolutional networks (AG-STGCNs) to investigate the regularity and variability of functional connectivity differences between gyri and sulci across multiple task domains. Based on 830 subjects with seven different task-based and one resting state fMRI (rs-fMRI) datasets from the public Human Connectome Project (HCP), we consistently found that there are significant differences of functional connectivity between gyral and sulcal regions within task domains compared with resting state (RS). Furthermore, there is considerable variability of such functional connectivity and information flow between gyri and sulci across different task domains, which are correlated with individual cognitive behaviors. Our study helps better understand the functional segregation of gyri and sulci within task domains as well as the anatomo-functional-behavioral relationship of the human brain.

A hierarchical atlas of the human cerebellum for functional precision mapping.

The human cerebellum is activated by a wide variety of cognitive and motor tasks. Previous functional atlases have relied on single task-based or resting-state fMRI datasets. Here, we present a functional atlas that integrates information from 7 large-scale datasets, outperforming existing group atlasses. The new atlas has three further advantages: First, the atlas allows for precision mapping in individuals: The integration of the probabilistic group atlas with an individual localizer scan results in a marked improvement in prediction of individual boundaries. Second, we provide both asymmetric and symmetric versions of the atlas. The symmetric version, which is obtained by constraining the boundaries to be the same across hemispheres, is especially useful in studying functional lateralization. Finally, the regions are hierarchically organized across 3 levels, allowing analyses at the appropriate level of granularity. Overall, the new atlas is an important resource for the study of the interdigitated functional organization of the human cerebellum in health and disease.

Coactivation pattern analysis reveals altered whole-brain functional transient dynamics in autism spectrum disorder.

Recent studies on autism spectrum disorder (ASD) have identified recurring states dominated by similar coactivation pattern (CAP) and revealed associations between dysfunction in seed-based large-scale brain networks and clinical symptoms. However, the presence of abnormalities in moment-to-moment whole-brain dynamics in ASD remains uncertain. In this study, we employed seed-free CAP analysis to identify transient brain activity configurations and investigate dynamic abnormalities in ASD. We utilized a substantial multisite resting-state fMRI dataset consisting of 354 individuals with ASD and 446 healthy controls (HCs, from HC groups and 2). CAP were generated from a subgroup of all HC subjects (HC group 1) through temporal K-means clustering, identifying four CAPs. These four CAPs exhibited either the activation or inhibition of the default mode network (DMN) and were grouped into two pairs with opposing spatial CAPs. CAPs for HC group 2 and ASD were identified by their spatial similarity to those for HC group 1. Compared with individuals in HC group 2, those with ASD spent more time in CAPs involving the ventral attention network but less time in CAPs related to executive control and the dorsal attention network. Support vector machine analysis demonstrated that the aberrant dynamic characteristics of CAPs achieved an accuracy of 74.87% in multisite classification. In addition, we used whole-brain dynamics to predict symptom severity in ASD. Our findings revealed whole-brain dynamic functional abnormalities in ASD from a single transient perspective, emphasizing the importance of the DMN in abnormal dynamic functional activity in ASD and suggesting that temporally dynamic techniques offer novel insights into time-varying neural processes.

RECONSTRUCTING RETINAL VISUAL IMAGES FROM 3T FMRI DATA ENHANCED BY UNSUPERVISED LEARNING.

The reconstruction of human visual inputs from brain activity, particularly through functional Magnetic Resonance Imaging (fMRI), holds promising avenues for unraveling the mechanisms of the human visual system. Despite the significant strides made by deep learning methods in improving the quality and interpretability of visual reconstruction, there remains a substantial demand for high-quality, long-duration, subject-specific 7-Tesla fMRI experiments. The challenge arises in integrating diverse smaller 3-Tesla datasets or accommodating new subjects with brief and low-quality fMRI scans. In response to these constraints, we propose a novel framework that generates enhanced 3T fMRI data through an unsupervised Generative Adversarial Network (GAN), leveraging unpaired training across two distinct fMRI datasets in 7T and 3T, respectively. This approach aims to overcome the limitations of the scarcity of high-quality 7-Tesla data and the challenges associated with brief and low-quality scans in 3-Tesla experiments. In this paper, we demonstrate the reconstruction capabilities of the enhanced 3T fMRI data, highlighting its proficiency in generating superior input visual images compared to data-intensive methods trained and tested on a single subject.

Explainable Multimodal Graph Isomorphism Network for Interpreting Sex Differences in Adolescent Neurodevelopment

Explainable Multimodal Graph Isomorphism Network for Interpreting Sex Differences in Adolescent Neurodevelopment

Functionality of arousal-regulating brain circuitry at rest predicts human cognitive abilities.

Arousal state is regulated by subcortical neuromodulatory nuclei, such as locus coeruleus, which send wide-reaching projections to cortex. Whether higher-order cortical regions have the capacity to recruit neuromodulatory systems to aid cognition is unclear. Here, we hypothesized that select cortical regions activate the arousal system, which, in turn, modulates large-scale brain activity, creating a functional circuit predicting cognitive ability. We utilized the Human Connectome Project 7T functional magnetic resonance imaging dataset (n = 149), acquired at rest with simultaneous eye tracking, along with extensive cognitive assessment for each subject. First, we discovered select frontoparietal cortical regions that drive large-scale spontaneous brain activity specifically via engaging the arousal system. Second, we show that the functionality of the arousal circuit driven by bilateral posterior cingulate cortex (associated with the default mode network) predicts subjects' cognitive abilities. This suggests that a cortical region that is typically associated with self-referential processing supports cognition by regulating the arousal system.

An inclusive multivariate approach to neural localization of language components

To determine how language is implemented in the brain, it is important to know which brain areas are primarily engaged in language processing and which are not. Existing protocols for localizing language are typically univariate, treating each small unit of brain volume as independent. One prominent example that focuses on the overall language network in functional magnetic resonance imaging (fMRI) uses a contrast between neural responses to sentences and sets of pseudowords (pronounceable nonwords). This contrast reliably activates peri-sylvian language areas but is less sensitive to extra-sylvian areas that are also known to support aspects of language such as word meanings (semantics). In this study, we assess areas where a multivariate, pattern-based approach shows high reproducibility across multiple measurements and participants, identifying these areas as multivariate regions of interest (mROI). We then perform a representational similarity analysis (RSA) of an fMRI dataset where participants made familiarity judgments on written words. We also compare those results to univariate regions of interest (uROI) taken from previous sentences > pseudowords contrasts. RSA with word stimuli defined in terms of their semantic distance showed greater correspondence with neural patterns in mROI than uROI. This was confirmed in two independent datasets, one involving single-word recognition, and the other focused on the meaning of noun-noun phrases by contrasting meaningful phrases > pseudowords. In all cases, areas of spatial overlap between mROI and uROI showed the greatest neural association. This suggests that ROIs defined in terms of multivariate reproducibility can help localize components of language such as semantics. The multivariate approach can also be extended to focus on other aspects of language such as phonology, and can be used along with the univariate approach for inclusively mapping language cortex.

Spectral Graph Neural Network-Based Multi-Atlas Brain Network Fusion for Major Depressive Disorder Diagnosis.

Major Depressive Disorder (MDD) imposes a substantial burden within the healthcare domain, impacting millions of individuals worldwide. Functional Magnetic Resonance Imaging (fMRI) has emerged as a promising tool for the objective diagnosis of MDD, enabling the investigation of functional connectivity patterns in the brain associated with this disorder. However, most existing methods focus on a single brain atlas, which limits their ability to capture the complex, multi-scale nature of functional brain networks. To address these limitations, we propose a novel multi-atlas fusion method that incorporates early and late fusion in a unified framework. Our method introduces the concept of the holistic Functional Connectivity Network (FCN), which captures both intra-atlas relationships within individual atlases and inter-regional relationships between atlases with different brain parcellation scales. This comprehensive representation enables the identification of potential disease-related patterns associated with MDD in the early stage of our framework. Moreover, by decoding the holistic FCN from various perspectives through multiple spectral Graph Convolutional Neural Networks and fusing their results with decision-level ensembles, we further improve the performance of MDD diagnosis. Our approach is easily implemented with minimal modifications to existing model structures and demonstrates a robust performance across different baseline models. Our method, evaluated on public resting-state fMRI datasets, surpasses the current multi-atlas fusion methods, enhancing the accuracy of MDD diagnosis. The proposed novel multi-atlas fusion framework provides a more reliable MDD diagnostic technique. Experimental results show our approach outperforms both single- and multi-atlas-based methods, demonstrating its effectiveness in advancing MDD diagnosis.

Representing Context and Priority in Working Memory.

The ability to prioritize among contents in working memory (WM) is critical for successful control of thought and behavior. Recent work has demonstrated that prioritization in WM can be implemented by representing different states of priority in different representational formats. Here, we explored the mechanisms underlying WM prioritization by simulating the double serial retrocuing task with recurrent neural networks. Visualization of stimulus representational dynamics using principal component analysis revealed that the network represented trial context (order of presentation) and priority via different mechanisms. Ordinal context, a stable property lasting the duration of the trial, was accomplished by segregating representations into orthogonal subspaces. Priority, which changed multiple times during a trial, was accomplished by separating representations into different strata within each subspace. We assessed the generality of these mechanisms by applying dimensionality reduction and multiclass decoding to fMRI and EEG data sets and found that priority and context are represented differently along the dorsal visual stream and that behavioral performance is sensitive to trial-by-trial variability of priority coding, but not context coding.

High performers demonstrate greater neural synchrony than low performers across behavioral domains

Abstract Heterogeneity in brain activity can give rise to heterogeneity in behavior, which in turn comprises our distinctive characteristics as individuals. Studying the path from brain to behavior, however, often requires making assumptions about how similarity in behavior scales with similarity in brain activity. Here, we expand upon recent work (Finn et al., 2020) which proposes a theoretical framework for testing the validity of such assumptions. Using intersubject representational similarity analysis in two independent movie-watching functional MRI (fMRI) datasets, we probe how brain-behavior relationships vary as a function of behavioral domain and participant sample. We find evidence that, in some cases, the neural similarity of two individuals is not correlated with behavioral similarity. Rather, individuals with higher behavioral scores are more similar to other high scorers whereas individuals with lower behavioral scores are dissimilar from everyone else. Ultimately, our findings motivate a more extensive investigation of both the structure of brain-behavior relationships and the tacit assumption that people who behave similarly will demonstrate shared patterns of brain activity.

Distinctive and Complementary Roles of Default Mode Network Subsystems in Semantic Cognition.

The default mode network (DMN) typically deactivates to external tasks, yet supports semantic cognition. It comprises medial temporal (MT), core, and frontotemporal (FT) subsystems, but its functional organization is unclear: the requirement for perceptual coupling versus decoupling, input modality (visual/verbal), type of information (social/spatial), and control demands all potentially affect its recruitment. We examined the effect of these factors on activation and deactivation of DMN subsystems during semantic cognition, across four task-based human functional magnetic resonance imaging (fMRI) datasets, and localized these responses in whole-brain state space defined by gradients of intrinsic connectivity. FT showed activation consistent with a central role across domains, tasks, and modalities, although it was most responsive to abstract, verbal tasks; this subsystem uniquely showed more "tuned" states characterized by increases in both activation and deactivation when semantic retrieval demands were higher. MT also activated to both perceptually coupled (scenes) and decoupled (autobiographical memory) tasks and showed stronger responses to picture associations, consistent with a role in scene construction. Core DMN consistently showed deactivation, especially to externally oriented tasks. These diverse contributions of DMN subsystems to semantic cognition were related to their location on intrinsic connectivity gradients: activation was closer to the sensory-motor cortex than deactivation, particularly for FT and MT, while activation for core DMN was distant from both visual cortex and cognitive control. These results reveal distinctive yet complementary DMN responses: MT and FT support different memory-based representations that are accessed externally and internally, while deactivation in core DMN is associated with demanding, external semantic tasks.

A Stepwise Multivariate Granger Causality Method for Constructing Hierarchical Directed Brain Functional Network.

The directed brain functional network construction gives us the new insights into the relationships between brain regions from the causality point of view. The Granger causality analysis is one of the powerful methods to model the directed network. The complex brain network is also hierarchically constructed, which is particularly suited to facilitate segregated functions and the global integration of the segregated functions. Therefore, it is of great interest to explore new approach to model the hierarchical architecture of the directed network. In the present study, we proposed a new approach, namely, stepwise multivariate Granger causality (SMGC), considering both the directed and hierarchical features of brain functional network to explore the stepwise causal relationship in the network. The simulation study demonstrated that the diverse and complex hierarchical organization could be embedded in the apparently simple directed network. The proposed SMGC method could capture the multiple hierarchy of the directed network. When applying to the real functional magnetic resonance imaging (fMRI) datasets, the core triple resting-state networks in human brain showed within-network directed connections in the first-level directed network and rich and diverse between-network pathways in the second-level hierarchical network. The default mode network (DMN) had a prominent role in the resting-state acting as both the causal source and the important relay station. Further exploratory research on the adaption of directed hierarchical network in athletes suggested the enhanced bidirectional communication between the DMN and the central executive network (CEN) and the enhanced directed connections from the salience network (SN) to the CEN in the athlete group. The SMGC approach is capable of capturing the hierarchical architecture of the brain directed functional network, which refreshes the new stepwise causal relationship in the directed network. This might shed light on the potential application for exploring the altered hierarchical organization of brain directed network in neuropsychiatric disorders.

A Learnable Counter-Condition Analysis Framework for Functional Connectivity-Based Neurological Disorder Diagnosis.

To understand the biological characteristics of neurological disorders with functional connectivity (FC), recent studies have widely utilized deep learning-based models to identify the disease and conducted post-hoc analyses via explainable models to discover disease-related biomarkers. Most existing frameworks consist of three stages, namely, feature selection, feature extraction for classification, and analysis, where each stage is implemented separately. However, if the results at each stage lack reliability, it can cause misdiagnosis and incorrect analysis in afterward stages. In this study, we propose a novel unified framework that systemically integrates diagnoses (i.e., feature selection and feature extraction) and explanations. Notably, we devised an adaptive attention network as a feature selection approach to identify individual-specific disease-related connections. We also propose a functional network relational encoder that summarizes the global topological properties of FC by learning the inter-network relations without pre-defined edges between functional networks. Last but not least, our framework provides a novel explanatory power for neuroscientific interpretation, also termed counter-condition analysis. We simulated the FC that reverses the diagnostic information (i.e., counter-condition FC): converting a normal brain to be abnormal and vice versa. We validated the effectiveness of our framework by using two large resting-state functional magnetic resonance imaging (fMRI) datasets, Autism Brain Imaging Data Exchange (ABIDE) and REST-meta-MDD, and demonstrated that our framework outperforms other competing methods for disease identification. Furthermore, we analyzed the disease-related neurological patterns based on counter-condition analysis.