Naturalistic fMRI Research Articles

Le Petit Prince Hong Kong (LPPHK): Naturalistic fMRI and EEG data from older Cantonese speakers

Currently, the field of neurobiology of language is based on data from only a few Indo-European languages. The majority of this data comes from younger adults neglecting other age groups. Here we present a multimodal database which consists of task-based and resting state fMRI, structural MRI, and EEG data while participants over 65 years old listened to sections of the story The Little Prince in Cantonese. We also provide data on participants’ language history, lifetime experiences, linguistic and cognitive skills. Audio and text annotations, including time-aligned speech segmentation and prosodic information, as well as word-by-word predictors such as frequency and part-of-speech tagging derived from natural language processing (NLP) tools are included in this database. Both MRI and EEG data diagnostics revealed that the data has good quality. This multimodal database could advance our understanding of spatiotemporal dynamics of language comprehension in the older population and help us study the effects of healthy aging on the relationship between brain and behaviour.

Psychological well-being modulates neural synchrony during naturalistic fMRI

Psychological well-being (PWB) is a combination of feeling good and functioning efficiently, and has a significant relationship with physical and mental health. Previous research has shown that PWB is associated with improvements in selective attention, mindfulness, semantic self-images, and adaptive decision making, however, it is unclear how these differences manifest in the brain. Naturalistic stimuli better encapsulate everyday experiences and can elicit more “true-to-life” neural responses. The current study seeks to identify how differing levels of PWB modulate neural synchrony in response to an audiovisual film. With consideration of the inherent variability of the literature, we aim to ascertain the validity of the regions previously associated with PWB. We identified that higher levels of PWB were associated with heightened stimulus driven neural synchrony in the bilateral superior parietal lobule, right planum temporale, and left superior temporal gyrus, and that lower levels of PWB were associated with heightened neural synchrony in the bilateral lateral occipital cortex and precuneus. Taken together, this research suggests that there is an association between differing levels of PWB and differential neural synchrony during movie-watching. PWB may therefore have an effect on complex, multimodal processing.

Resting state functional brain connectivity in child and adolescent psychiatry: where are we now?

Approaching the 30th anniversary of the discovery of resting state functional magnetic resonance imaging (rsfMRI) functional connectivity, we reflect on the impact of this neuroimaging breakthrough on the field of child and adolescent psychiatry. The study of intrinsic functional brain architecture that rsfMRI affords across a wide range of ages and abilities has yielded numerous key insights. For example, we now know that many neurodevelopmental conditions are associated with more widespread circuit alterations across multiple large-scale brain networks than previously suspected. The emergence of population neuroscience and effective data-sharing initiatives have made large rsfMRI datasets publicly available, providing sufficient power to begin to identify brain-based subtypes within heterogeneous clinical conditions. Nevertheless, several methodological and theoretical challenges must still be addressed to fulfill the promises of personalized child and adolescent psychiatry. In particular, incomplete understanding of the physiological mechanisms driving developmental changes in intrinsic functional connectivity remains an obstacle to further progress. Future directions include cross-species and multimodal neuroimaging investigations to illuminate such mechanisms. Data collection and harmonization efforts that span multiple countries and diverse cohorts are urgently needed. Finally, incorporating naturalistic fMRI paradigms such as movie watching should be a priority for future research efforts.

Localizing Syntactic Composition with Left-Corner Recurrent Neural Network Grammars.

In computational neurolinguistics, it has been demonstrated that hierarchical models such as recurrent neural network grammars (RNNGs), which jointly generate word sequences and their syntactic structures via the syntactic composition, better explained human brain activity than sequential models such as long short-term memory networks (LSTMs). However, the vanilla RNNG has employed the top-down parsing strategy, which has been pointed out in the psycholinguistics literature as suboptimal especially for head-final/left-branching languages, and alternatively the left-corner parsing strategy has been proposed as the psychologically plausible parsing strategy. In this article, building on this line of inquiry, we investigate not only whether hierarchical models like RNNGs better explain human brain activity than sequential models like LSTMs, but also which parsing strategy is more neurobiologically plausible, by developing a novel fMRI corpus where participants read newspaper articles in a head-final/left-branching language, namely Japanese, through the naturalistic fMRI experiment. The results revealed that left-corner RNNGs outperformed both LSTMs and top-down RNNGs in the left inferior frontal and temporal-parietal regions, suggesting that there are certain brain regions that localize the syntactic composition with the left-corner parsing strategy.

Will you read how I will read? Naturalistic fMRI predictors of emergent reading

Despite reading being an essential and almost universal skill in the developed world, reading proficiency varies substantially from person to person. To study why, the fMRI field is beginning to turn from single-word or nonword reading tasks to naturalistic stimuli like connected text and listening to stories. To study reading development in children just beginning to read, listening to stories is an appropriate paradigm because speech perception and phonological processing are important for, and are predictors of, reading proficiency. Our study examined the relationship between behavioral reading-related skills and the neural response to listening to stories in the fMRI environment. Functional MRI were gathered in a 3T TIM-Trio scanner. During the fMRI scan, children aged approximately 7 years listened to professionally narrated common short stories and answered comprehension questions following the narration. Analyses of the data used inter-subject correlation (ISC), and representational similarity analysis (RSA). Our primary finding is that ISC reveals areas of increased synchrony in both high- and low-performing emergent readers previously implicated in reading ability/disability. Of particular interest are that several previously identified brain regions (medial temporal gyrus (MTG), inferior frontal gyrus (IFG), inferior temporal gyrus (ITG)) were found to “synchronize” across higher reading ability participants, while lower reading ability participants had idiosyncratic activation patterns in these regions. Additionally, two regions (superior frontal gyrus (SFG) and another portion of ITG) were recruited by all participants, but their specific timecourse of activation depended on reading performance. These analyses support the idea that different brain regions involved in reading follow different developmental trajectories that correlate with reading proficiency on a spectrum rather than the usual dichotomy of poor readers versus strong readers.

Exploring static and dynamic functional brain networks in adolescent depression using a co-produced novel irritability paradigm

Background Irritability is a core symptom of depression in adolescence and a risk factor for emotion regulation problems. However, its neural correlates are not well understood. Existing functional magnetic resonance imaging (fMRI) research on irritability typically overlooks its social context. Methods Here, we pilot a novel naturalistic fMRI paradigm targeting the social nature of irritability that was co-produced with young people (N = 88) and apply it in an independent sample of youth (N = 29, mean age 18.9 years, 77% female) with self-reported low mood that were aged 16 to 20 years. Participants were also fluent English speakers, free from MRI contraindications, and did not report a diagnosis of a neurological or neurodevelopmental condition. Our aim was to investigate whether graph theoretic and dynamical properties of functional brain networks differed between a resting state scan and our irritability paradigm. We also examined whether these brain features were associated with depressive symptoms and trait irritability. Results Using Leading Eigenvector Dynamics Analysis (LEiDA), we found that the dynamic properties of brain networks comprising default-mode and fronto-parietal regions differed significantly during the irritability paradigm compared to the rest condition. While no gross static topological differences were found between these two conditions, we found that some dynamic and topological features of emotion-related brain networks were related to trait irritability and depressive symptoms in our sample. Conclusions Although the current findings are preliminary due to the pilot nature of this study, this work showcases the feasibility of co-produced research in neuroimaging and lays a strong foundation for further study.

A large-scale fMRI dataset for the visual processing of naturalistic scenes

One ultimate goal of visual neuroscience is to understand how the brain processes visual stimuli encountered in the natural environment. Achieving this goal requires records of brain responses under massive amounts of naturalistic stimuli. Although the scientific community has put a lot of effort into collecting large-scale functional magnetic resonance imaging (fMRI) data under naturalistic stimuli, more naturalistic fMRI datasets are still urgently needed. We present here the Natural Object Dataset (NOD), a large-scale fMRI dataset containing responses to 57,120 naturalistic images from 30 participants. NOD strives for a balance between sampling variation between individuals and sampling variation between stimuli. This enables NOD to be utilized not only for determining whether an observation is generalizable across many individuals, but also for testing whether a response pattern is generalized to a variety of naturalistic stimuli. We anticipate that the NOD together with existing naturalistic neuroimaging datasets will serve as a new impetus for our understanding of the visual processing of naturalistic stimuli.

Neuroscout, a unified platform for generalizable and reproducible fMRI research.

Functional magnetic resonance imaging (fMRI) has revolutionized cognitive neuroscience, but methodological barriers limit the generalizability of findings from the lab to the real world. Here, we present Neuroscout, an end-to-end platform for analysis of naturalistic fMRI data designed to facilitate the adoption of robust and generalizable research practices. Neuroscout leverages state-of-the-art machine learning models to automatically annotate stimuli from dozens of fMRI studies using naturalistic stimuli-such as movies and narratives-allowing researchers to easily test neuroscientific hypotheses across multiple ecologically-valid datasets. In addition, Neuroscout builds on a robust ecosystem of open tools and standards to provide an easy-to-use analysis builder and a fully automated execution engine that reduce the burden of reproducible research. Through a series of meta-analytic case studies, we validate the automatic feature extraction approach and demonstrate its potential to support more robust fMRI research. Owing to its ease of use and a high degree of automation, Neuroscout makes it possible to overcome modeling challenges commonly arising in naturalistic analysis and to easily scale analyses within and across datasets, democratizing generalizable fMRI research.

Separating Stimulus-Induced and Background Components of Dynamic Functional Connectivity in Naturalistic fMRI.

We consider the challenges in extracting stimulus-related neural dynamics from other intrinsic processes and noise in naturalistic functional magnetic resonance imaging (fMRI). Most studies rely on inter-subject correlations (ISC) of low-level regional activity and neglect varying responses in individuals. We propose a novel, data-driven approach based on low-rank plus sparse ( [Formula: see text]) decomposition to isolate stimulus-driven dynamic changes in brain functional connectivity (FC) from the background noise, by exploiting shared network structure among subjects receiving the same naturalistic stimuli. The time-resolved multi-subject FC matrices are modeled as a sum of a low-rank component of correlated FC patterns across subjects, and a sparse component of subject-specific, idiosyncratic background activities. To recover the shared low-rank subspace, we introduce a fused version of principal component pursuit (PCP) by adding a fusion-type penalty on the differences between the columns of the low-rank matrix. The method improves the detection of stimulus-induced group-level homogeneity in the FC profile while capturing inter-subject variability. We develop an efficient algorithm via a linearized alternating direction method of multipliers to solve the fused-PCP. Simulations show accurate recovery by the fused-PCP even when a large fraction of FC edges are severely corrupted. When applied to natural fMRI data, our method reveals FC changes that were time-locked to auditory processing during movie watching, with dynamic engagement of sensorimotor systems for speech-in-noise. It also provides a better mapping to auditory content in the movie than ISC.

Comparing the Effectiveness of Brain Structural Imaging, Resting-state fMRI, and Naturalistic fMRI in Recognizing Social Anxiety Disorder in Children and Adolescents

Social anxiety disorder (SAD) is a common anxiety disorder in childhood and adolescence. Studies on SAD in adults have reported both structural and functional aberrancies of the brain at the group level. However, evidence has shown differences in anxiety-related brain abnormalities between adolescents and adults. Since children and adolescents can afford limited scan time, optimizing the scan tasks is essential for SAD research in children and adolescents. Thus, we need to address whether brain structure, resting-state fMRI, and naturalistic imaging enable individualized identification of SAD in children and adolescents, which measurement is more effective, and whether pooling multi-modal features can improve the identification of SAD. We comprehensively addressed these questions by building machine learning models based on parcel-wise brain features. We found that naturalistic fMRI yielded higher classification accuracy (69.17%) than the other modalities and the classification performance showed dependence on the contents of the movie. The classification models also identified contributing brain regions, some of which exhibited correlations with the symptoms scores of SAD. However, pooling brain features from the three modalities did not help enhance the classification accuracy. These results support the application of carefully designed naturalistic imaging in recognizing children and adolescents at risk of SAD.

Distinct neural substrates of individual differences in components of reading comprehension in adults with or without dyslexia

Reading comprehension is a complex task that depends on multiple cognitive and linguistic processes. According to the updated Simple View of Reading framework, in adults, individual variation in reading comprehension can be largely explained by combined variance in three component abilities: (1) decoding accuracy, (2) fluency, and (3) language comprehension. Here we asked whether the neural correlates of the three components are different in adults with dyslexia as compared to typically-reading adults and whether the relative contribution of these correlates to reading comprehension is similar in the two groups. We employed a novel naturalistic fMRI reading task to identify the neural correlates of individual differences in the three components using whole-brain and literature-driven regions-of-interest approaches. Across all participants, as predicted by the Simple View framework, we found distinct patterns of associations with linguistic and domain-general regions for the three components, and that the left-hemispheric neural correlates of language comprehension in the angular and posterior temporal gyri made the largest contributions to explaining out-of-scanner reading comprehension performance. These patterns differed between the two groups. In typical adult readers, better fluency was associated with greater activation of left occipitotemporal regions, better comprehension with lesser activation in prefrontal and posterior parietal regions, and there were no significant associations with decoding. In adults with dyslexia, better fluency was associated with greater activation of bilateral inferior parietal regions, better comprehension was associated with greater activation in some prefrontal clusters and lower in others, and better decoding skills were associated with lesser activation of bilateral prefrontal and posterior parietal regions. Extending the behavioral findings of skill-level differences in the relative contribution of the three components to reading comprehension, the relative contributions of the neural correlates to reading comprehension differed based on dyslexia status. These findings reveal some of the neural correlates of individual differences in the three components and the underlying mechanisms of reading comprehension deficits in adults with dyslexia.

Semantics-weighted lexical surprisal modeling of naturalistic functional MRI time-series during spoken narrative listening

Probabilistic language models are increasingly used to provide neural representations of linguistic features under naturalistic settings. Word surprisal models can be applied to continuous fMRI recordings during task-free listening of narratives, to detect regions linked to language prediction and comprehension. Here, to this purpose, a novel semantics-weighted lexical surprisal is applied to naturalistic fMRI data.FMRI was performed at 3 Tesla in 31 subjects during task-free listening to a 12-minute audiobook played in both original and word-reversed (control) version. Lexical-only and semantics-weighted lexical surprisal models were estimated for the original and control word series. The two series were alternatively chosen to build the predictor of interest in the first-level general linear model and were compared in the second-level (group) analysis.The addition of the surprisal predictor to the stimulus-related predictors significantly improved the fitting of the neural signal. In average, the semantics-weighted model yielded lower surprisal values and, in some areas, better fitting of the fMRI data compared to the lexical-only model. The two models produced both overlapping and distinct activations: while lexical-only surprisal activated secondary auditory areas in the superior temporal gyri and the cerebellum, semantics-weighted surprisal additionally activated the left inferior frontal gyrus.These results confirm the usefulness of surprisal models in the naturalistic fMRI analysis of linguistic processes and suggest that the use of semantic information may increase the sensitivity of a probabilistic language model in higher-order language-related areas, with possible implications for future naturalistic fMRI studies of language under normal and (clinically or pharmacologically) modified conditions.

Custom-molded headcases have limited efficacy in reducing head motion during naturalistic fMRI experiments

Effectively minimizing head motion continues to be a challenge for the collection of functional magnetic resonance imaging (fMRI) data. The use of individual-specific custom molded headcases is a promising solution to this issue, but there has been limited work to date. In the present work, we examine the efficacy of headcases in a larger group of participants engaged in naturalistic scanning paradigms including: long movie-watching scans (~20 to 45min) and a recall task that involved talking aloud inside the MRI. Unlike previous work, we find that headcases do not reliably reduce motion during movie viewing compared to alternative methods such as foam pillows or foam pillows plus medical tape. Surprisingly, we also find that motion is worse when participants talk aloud while wearing headcases. These differences appear to be driven by large, brief rotations of the head as well as translations in the z-plane as participants speak. Smaller, constant head movements appear equivalent with or without headcases. The largest reductions in head motion are observable when participants were situated with both foam pillows and medical tape. Altogether, this work suggests that in a healthy adult population, custom-molded headcases may provide limited efficacy in reducing head motion beyond existing tools available to researchers. We hope this work can help improve the quality of custom headcases, motivate the investigation of additional solutions, and provide additional information about head motion in naturalistic contexts.

Informed feature regularization in voxelwise modeling for naturalistic fMRI experiments.

Voxelwise modeling is a powerful framework to predict single-voxel functional selectivity for the stimulus features that exist in complex natural stimuli. Yet, because VM disregards potential correlations across stimulus features or neighboring voxels, it may yield suboptimal sensitivity in measuring functional selectivity in the presence of high levels of measurement noise. Here, we introduce a novel voxelwise modeling approach that simultaneously utilizes stimulus correlations in model features and response correlations among voxel neighborhoods. The proposed method performs feature and spatial regularization while still generating single-voxel response predictions. We demonstrated the performance of our approach on a functional magnetic resonance imaging dataset from a natural vision experiment. Compared to VM, the proposed method yields clear improvements in prediction performance, together with increased feature coherence and spatial coherence of voxelwise models. Overall, the proposed method can offer improved sensitivity in modeling of single voxels in naturalistic functional magnetic resonance imaging experiments.

Leveraging shared connectivity to aggregate heterogeneous datasets into a common response space

Connectivity hyperalignment can be used to estimate a single shared response space across disjoint datasets. We develop a connectivity-based shared response model that factorizes aggregated fMRI datasets into a single reduced-dimension shared connectivity space and subject-specific topographic transformations. These transformations resolve idiosyncratic functional topographies and can be used to project response time series into shared space. We evaluate this algorithm on a large collection of heterogeneous, naturalistic fMRI datasets acquired while subjects listened to spoken stories. Projecting subject data into shared space dramatically improves between-subject story time-segment classification and increases the dimensionality of shared information across subjects. This improvement generalizes to subjects and stories excluded when estimating the shared space. We demonstrate that estimating a simple semantic encoding model in shared space improves between-subject forward encoding and inverted encoding model performance. The shared space estimated across all datasets is distinct from the shared space derived from any particular constituent dataset; the algorithm leverages shared connectivity to yield a consensus shared space conjoining diverse story stimuli.

Analysis of stimulus-induced brain dynamics during naturalistic paradigms

Naturalistic stimuli offer promising avenues for investigating brain function across the rich, realistic spectrum of human experiences. Functional magnetic resonance imaging (fMRI) studies of brain activity during naturalistic paradigms have provided new information about dynamic neural processing in ecologically valid contexts. Yet, the complex, uncontrolled nature of such stimuli -- and the resulting mixture of neuronal and physiological responses embedded within the fMRI signals -- present challenges with respect to data analysis and interpretation. In this brief commentary, we discuss methods and open challenges in naturalistic fMRI investigations, with a focus on extracting and interpreting stimulus-induced fMRI signals.

A pseudo-population-based approach for the analysis of pharmacological and naturalistic fMRI experiments

A pseudo-population-based approach for the analysis of pharmacological and naturalistic fMRI experiments

Language beyond the language system: Dorsal visuospatial pathways support processing of demonstratives and spatial language during naturalistic fast fMRI

Spatial demonstratives are powerful linguistic tools used to establish joint attention. Identifying the meaning of semantically underspecified expressions like “this one” hinges on the integration of linguistic and visual cues, attentional orienting and pragmatic inference. This synergy between language and extralinguistic cognition is pivotal to language comprehension in general, but especially prominent in demonstratives.In this study, we aimed to elucidate which neural architectures enable this intertwining between language and extralinguistic cognition using a naturalistic fMRI paradigm. In our experiment, 28 participants listened to a specially crafted dialogical narrative with a controlled number of spatial demonstratives. A fast multiband-EPI acquisition sequence (TR = 388 m s) combined with finite impulse response (FIR) modelling of the hemodynamic response was used to capture signal changes at word-level resolution.We found that spatial demonstratives bilaterally engage a network of parietal areas, including the supramarginal gyrus, the angular gyrus, and precuneus, implicated in information integration and visuospatial processing. Moreover, demonstratives recruit frontal regions, including the right FEF, implicated in attentional orienting and reference frames shifts. Finally, using multivariate similarity analyses, we provide evidence for a general involvement of the dorsal (“where”) stream in the processing of spatial expressions, as opposed to ventral pathways encoding object semantics.Overall, our results suggest that language processing relies on a distributed architecture, recruiting neural resources for perception, attention, and extra-linguistic aspects of cognition in a dynamic and context-dependent fashion.

Spatially informed voxelwise modeling for naturalistic fMRI experiments

Voxelwise modeling (VM) is a powerful framework to predict single voxel responses evoked by a rich set of stimulus features present in complex natural stimuli. However, because VM disregards correlations across neighboring voxels, its sensitivity in detecting functional selectivity can be diminished in the presence of high levels of measurement noise. Here, we introduce spatially-informed voxelwise modeling (SPIN-VM) to take advantage of response correlations in spatial neighborhoods of voxels. To optimally utilize shared information, SPIN-VM performs regularization across spatial neighborhoods in addition to model features, while still generating single-voxel response predictions. We demonstrated the performance of SPIN-VM on a rich dataset from a natural vision experiment. Compared to VM, SPIN-VM yields higher prediction accuracies and better capture locally congruent information representations across cortex. These results suggest that SPIN-VM offers improved performance in predicting single-voxel responses and recovering coherent information representations.

Cooperating yet distinct brain networks engaged during naturalistic paradigms: A meta-analysis of functional MRI results.

Cognitive processes do not occur by pure insertion and instead depend on the full complement of co-occurring mental processes, including perceptual and motor functions. As such, there is limited ecological validity to human neuroimaging experiments that use highly controlled tasks to isolate mental processes of interest. However, a growing literature shows how dynamic, interactive tasks have allowed researchers to study cognition as it more naturally occurs. Collective analysis across such neuroimaging experiments may answer broader questions regarding how naturalistic cognition is biologically distributed throughout the brain. We applied an unbiased, data-driven, meta-analytic approach that uses k-means clustering to identify core brain networks engaged across the naturalistic functional neuroimaging literature. Functional decoding allowed us to, then, delineate how information is distributed between these networks throughout the execution of dynamical cognition in realistic settings. This analysis revealed six recurrent patterns of brain activation, representing sensory, domain-specific, and attentional neural networks that support the cognitive demands of naturalistic paradigms. Although gaps in the literature remain, these results suggest that naturalistic fMRI paradigms recruit a common set of networks that allow both separate processing of different streams of information and integration of relevant information to enable flexible cognition and complex behavior.