Naturalistic Imaging Research Articles

A normative model ofbrain responses to social scenarios reflects the maturity ofchildren andadolescents' social-emotional abilities.

The rapid brain maturation in childhood and adolescence accompanies the development of socio-emotional functioning. However, it is unclear how the maturation of the neural activity drives the development of socio-emotional functioning and individual differences. This study aimed to reflect the age dependence of inter-individual differences in brain responses to socio-emotional scenarios and to develop naturalistic imaging indicators to assess the maturity of socio-emotional ability at the individual level. Using three independent naturalistic imaging datasets containing healthy participants (n = 111, 21 and 122), we found and validated that age-modulated inter-individual concordance of brain responses to socio-emotional movies in specific brain regions. The similarity of an individual's brain response to the average response of older participants was defined as response typicality, which predicted an individual's emotion regulation strategies in adolescence and theory of mind (ToM) in childhood. Its predictive power was not superseded by age, sex, cognitive performance or executive function. We further showed that the movie's valence and arousal ratings grounded the response typicality. The findings highlight that forming typical brain response patterns may be a neural phenotype underlying the maturation of socio-emotional ability. The proposed response typicality represents a neuroimaging approach to measure individuals' maturity of cognitive reappraisal and ToM.

Comparing the Effectiveness of Brain Structural Imaging, Resting-state fMRI, and Naturalistic fMRI in Recognizing Social Anxiety Disorder in Children and Adolescents

Social anxiety disorder (SAD) is a common anxiety disorder in childhood and adolescence. Studies on SAD in adults have reported both structural and functional aberrancies of the brain at the group level. However, evidence has shown differences in anxiety-related brain abnormalities between adolescents and adults. Since children and adolescents can afford limited scan time, optimizing the scan tasks is essential for SAD research in children and adolescents. Thus, we need to address whether brain structure, resting-state fMRI, and naturalistic imaging enable individualized identification of SAD in children and adolescents, which measurement is more effective, and whether pooling multi-modal features can improve the identification of SAD. We comprehensively addressed these questions by building machine learning models based on parcel-wise brain features. We found that naturalistic fMRI yielded higher classification accuracy (69.17%) than the other modalities and the classification performance showed dependence on the contents of the movie. The classification models also identified contributing brain regions, some of which exhibited correlations with the symptoms scores of SAD. However, pooling brain features from the three modalities did not help enhance the classification accuracy. These results support the application of carefully designed naturalistic imaging in recognizing children and adolescents at risk of SAD.

Video-evoked fMRI BOLD responses are highly consistent across different data acquisition sites.

Naturalistic imaging paradigms, in which participants view complex videos in the scanner, are increasingly used in human cognitive neuroscience. Videos evoke temporally synchronized brain responses that are similar across subjects as well as within subjects, but the reproducibility of these brain responses across different data acquisition sites has not yet been quantified. Here, we characterize the consistency of brain responses across independent samples of participants viewing the same videos in functional magnetic resonance imaging (fMRI) scanners at different sites (Indiana University and Caltech). We compared brain responses collected at these different sites for two carefully matched datasets with identical scanner models, acquisition, and preprocessing details, along with a third unmatched dataset in which these details varied. Our overall conclusion is that for matched and unmatched datasets alike, video‐evoked brain responses have high consistency across these different sites, both when compared across groups and across pairs of individuals. As one might expect, differences between sites were larger for unmatched datasets than matched datasets. Residual differences between datasets could in part reflect participant‐level variability rather than scanner‐ or data‐ related effects. Altogether our results indicate promise for the development and, critically, generalization of video fMRI studies of individual differences in healthy and clinical populations alike.

Functional Subdivisions of the Cerebellum in Naturalistic Paradigm Functional Magnetic Resonance Imaging.

Compelling evidence has suggested that the human cerebellum is engaged in a wide range of cognitive tasks besides traditional opinions of motor control, and it is organized into a set of distinct functional subregions. The existing model-driven cerebellum parcellations through resting-state functional MRI (rsfMRI) and task-fMRI are relatively coarse, introducing challenges in resolving the functions of the cerebellum especially when the brain is exposed to naturalistic environments. The current study took the advantages of the naturalistic paradigm (i.e., movie viewing) fMRI (nfMRI) to derive fine parcellations via a data-driven dual-regression-like sparse representation framework. The parcellations were quantitatively evaluated by functional homogeneity, and global and local boundary confidence. In addition, the differences of cerebellum–cerebrum functional connectivities between rsfMRI and nfMRI for some exemplar parcellations were compared to provide qualitatively functional validations. Our experimental results demonstrated that the proposed study successfully identified distinct subregions of the cerebellum. This fine parcellation may serve as a complementary solution to existing cerebellum parcellations, providing an alternative template for exploring neural activities of the cerebellum in naturalistic environments.

Reliability map of individual differences reflected in inter-subject correlation in naturalistic imaging

Understanding individual differences in brain function is an essential aim of neuroscience. Naturalistic imaging links neural activity to real-life contexts and reflects individual differences in brain response. These unique features make it a promising tool for individualized psychiatry. An essential prerequisite for the extensive use of this paradigm is the reliable representation of inter-individual relationships. We used a test–retest approach to examine whether the naturalistic paradigm reliably represents inter-individual differences, which brain regions have the superior capability, and whether the ability alters with the contents of the stimuli. We quantified the reliability of the inter-subject relationships in repeated scans of two movie clips: a natural sight view and an emotion-evoking story. Besides statistical inference, we included resting-state scans, behavioral tests, and questionnaires as references for the comparison. The results showed that over one-third area of the brain could reliably characterize the inter-individual relationship, and the superior temporal lobe demonstrated comparable reliability representation with the State and Trait Anxiety Inventory. Furthermore, the temporal lobe regions could retain this capability across emotional movies with different contents. This study provides a base for pushing the naturalistic imaging paradigm towards clinical applications and proposes reliable target brain regions for future studies.

Neural dynamics of sentiment processing during naturalistic sentence reading

When we read, our eyes move through the text in a series of fixations and high-velocity saccades to extract visual information. This process allows the brain to obtain meaning, e.g., about sentiment, or the emotional valence, expressed in the written text. How exactly the brain extracts the sentiment of single words during naturalistic reading is largely unknown. This is due to the challenges of naturalistic imaging, which has previously led researchers to employ highly controlled, timed word-by-word presentations of custom reading materials that lack ecological validity. Here, we aimed to assess the electrical neural correlates of word sentiment processing during naturalistic reading of English sentences. We used a publicly available dataset of simultaneous electroencephalography (EEG), eye-tracking recordings, and word-level semantic annotations from 7129 words in 400 sentences (Zurich Cognitive Language Processing Corpus; Hollenstein et al., 2018). We computed fixation-related potentials (FRPs), which are evoked electrical responses time-locked to the onset of fixations. A general linear mixed model analysis of FRPs cleaned from visual- and motor-evoked activity showed a topographical difference between the positive and negative sentiment condition in the 224–304 ​ms interval after fixation onset in left-central and right-posterior electrode clusters. An additional analysis that included word-, phrase-, and sentence-level sentiment predictors showed the same FRP differences for the word-level sentiment, but no additional FRP differences for phrase- and sentence-level sentiment. Furthermore, decoding analysis that classified word sentiment (positive or negative) from sentiment-matched 40-trial average FRPs showed a 0.60 average accuracy (95% confidence interval: [0.58, 0.61]). Control analyses ruled out that these results were based on differences in eye movements or linguistic features other than word sentiment. Our results extend previous research by showing that the emotional valence of lexico-semantic stimuli evoke a fast electrical neural response upon word fixation during naturalistic reading. These results provide an important step to identify the neural processes of lexico-semantic processing in ecologically valid conditions and can serve to improve computer algorithms for natural language processing.

First-Person Virtual Embodiment Modulates the Cortical Network that Encodes the Bodily Self and Its Surrounding Space during the Experience of Domestic Violence.

Social aggression, such as domestic violence, has been associated with a reduced ability to take on others’ perspectives. In this naturalistic imaging study, we investigated whether training human participants to take on a first-person embodied perspective during the experience of domestic violence enhances the identification with the victim and elicits brain activity associated with the monitoring of the body and surrounding space and the experience of threat. We combined fMRI measurements with preceding virtual reality exposure from either first-person perspective (1PP) or third-person perspective (3PP) to manipulate whether the domestic abuse stimulus was perceived as directed to oneself or another. We found that 1PP exposure increased body ownership and identification with the virtual victim. Furthermore, when the stimulus was perceived as directed toward oneself, the brain network that encodes the bodily self and its surrounding space was more strongly synchronized across participants and connectivity increased from premotor cortex (PM) and intraparietal sulcus towards superior parietal lobe. Additionally, when the stimulus came near the body, brain activity in the amygdala (AMG) strongly synchronized across participants. Exposure to 3PP reduced synchronization of brain activity in the personal space network, increased modulation of visual areas and strengthened functional connectivity between PM, supramarginal gyrus and primary visual cortex. In conclusion, our results suggest that 1PP embodiment training enhances experience from the viewpoint of the virtual victim, which is accompanied by synchronization in the fronto-parietal network to predict actions toward the body and in the AMG to signal the proximity of the stimulus.

Idiosynchrony: From shared responses to individual differences during naturalistic neuroimaging

Two ongoing movements in human cognitive neuroscience have researchers shifting focus from group-level inferences to characterizing single subjects, and complementing tightly controlled tasks with rich, dynamic paradigms such as movies and stories. Yet relatively little work combines these two, perhaps because traditional analysis approaches for naturalistic imaging data are geared toward detecting shared responses rather than between-subject variability. Here, we review recent work using naturalistic stimuli to study individual differences, and advance a framework for detecting structure in idiosyncratic patterns of brain activity, or “idiosynchrony”. Specifically, we outline the emerging technique of inter-subject representational similarity analysis (IS-RSA), including its theoretical motivation and an empirical demonstration of how it recovers brain-behavior relationships during movie watching using data from the Human Connectome Project. We also consider how stimulus choice may affect the individual signal and discuss areas for future research. We argue that naturalistic neuroimaging paradigms have the potential to reveal meaningful individual differences above and beyond those observed during traditional tasks or at rest.

Social cognition in context: A naturalistic imaging approach

Social processing occurs within dynamic, complex, and multimodal contexts, but the study of social cognition typically involves static, artificial stimuli. Naturalistic approaches (e.g., movie viewing) can recapture the richness and complexity of real-world interactions. Novel analytic approaches allow for the investigation of functional brain organization in response to contextually embedded and extended events with a complex temporal structure during movie viewing or narrative processing. In addition to these within-brain measures, movies afford between-brain analyses such as inter-subject correlation, which allows for identification of stimulus-specific brain response through the correlation of brain activity between participants’ brains. Research using these approaches offers both practical and theoretical advantages in understanding how we navigate our social world. Practically, movies are engaging stimuli that allow for more rapid presentation of multiple event types and improve compliance even in very young populations. Theoretically, studies have validated the use of these measures by demonstrating functional selectivity to contextually embedded stimuli. Naturalistic approaches also allow for novel insights. For example, regions associated with social cognition have longer temporal receptive windows, making them well suited to social-cognitive processes that require integration of information over longer timescales. Furthermore, the similarity in the temporal and spatial brain response between individuals during naturalistic viewing is related to age, predictive of friendships, and reduced in autism spectrum disorder. These findings offer first glimpses into the power of using these naturalistic, dynamic approaches to understand how we perceive, reason about, and interact with others.

Individualized psychiatric imaging based on inter-subject neural synchronization in movie watching

The individual heterogeneity is a challenge to the prosperous promises of cutting-edge neuroimaging techniques for better diagnosis and early detection of psychiatric disorders. Individuals with similar clinical manifestations may result from very different pathophysiology. Conventional approaches based on comparing group-averages provide insufficient information to support the individualized diagnosis. Here we present an individualized imaging methodology that combines naturalistic imaging and the normative model. This paradigm adopts video clips with rich cognitive, social, and emotional contents to evoke synchronized brain dynamics of healthy participants and builds a spatiotemporal response norm. By comparing individual brain responses with the response norm, we could recognize patients using machine learning techniques. We applied this methodology to recognize first-episode drug-naïve schizophrenia patients in a dataset containing 72 patients and 54 healthy controls. Some segments of the video evoked more synchronized brain activity in the healthy controls than in the schizophrenia patients. We built a spatiotemporal response norm by averaging the brain responses of the healthy controls in a training set, and trained a classifier to recognize patients based on the differences between individual brain responses and the norm. The performance of the classifier was then evaluated using an independent test set. The mean accuracies from a 5-fold cross-validation were 0.71–0.78 depending on the parameters such as the number of features and the width of the sliding windows. These findings reflected the potential of this methodology towards a clinical tool for individualized diagnosis.

Segregated precuneus network and default mode network in naturalistic imaging.

A resting-state network centered at the precuneus has been recently proposed as a precuneus network (PCUN) or "parietal memory network". Due to its spatial adjacency and overlapping with the default mode network (DMN), it is still not consensus to consider PCUN and DMN separately. Whether consideringPCUN and DMN as different networks is a critical question thatinfluencesour understanding of brain functions and impairments. Previous resting-state studies using multiple methodologies have demonstrated a robust separation of the two networks. However, since there is no gold standard in justifying the functional difference between the networksin resting-state, we still lack of biological evidence to directly support the separation of the two networks. This study compared the responses and functional couplings of PCUN and DMN when participants were watching a movie and examined how the continuity of the movie context modulated the response of the networks. We identified PCUN and DMN in resting-state fMRI of 48 healthy subjects. The networks' response to a context-rich video and its context-shuffled version was characterized using the variance of temporal fluctuations and functional connectivity metrics. The results showed that (1) scrambling the contextual information altered the fluctuation level of DMN and PCUN in reversed ways; (2) compared to DMN, the FC within PCUN showed significantly higher sensitivity to the contextual continuity; (3) PCUN exhibited a significantly stronger functional network connectivity with the primary visual regions than DMN. These findings provide evidence for the distinct functional roles of PCUN and DMN in processing context-rich information and call for separately considering the functions and impairments of these networks in resting-state studies.

Functional Imaging of Visuospatial Attention in Complex and Naturalistic Conditions.

One of the ultimate goals of cognitive neuroscience is to understand how the brain works in the real world. Functional imaging with naturalistic stimuli provides us with the opportunity to study the brain in situations similar to the everyday life. This includes the processing of complex stimuli that can trigger many types of signals related both to the physical characteristics of the external input and to the internal knowledge that we have about natural objects and environments. In this chapter, I will first outline different types of stimuli that have been used in naturalistic imaging studies. These include static pictures, short video clips, full-length movies, and virtual reality, each comprising specific advantages and disadvantages. Next, I will turn to the main issue of visual-spatial orienting in naturalistic conditions and its neural substrates. I will discuss different classes of internal signals, related to objects, scene structure, and long-term memory. All of these, together with external signals about stimulus salience, have been found to modulate the activity and the connectivity of the frontoparietal attention networks. I will conclude by pointing out some promising future directions for functional imaging with naturalistic stimuli. Despite this field of research is still in its early days, I consider that it will play a major role in bridging the gap between standard laboratory paradigms and mechanisms of brain functioning in the real world.