Selective processing of social interactions during naturalistic movie viewing

Abstract

The ability to recognize interactions between two or more people in a complex visual scene is a crucial skill that helps us navigate the social world. Previous studies have revealed that the brain selectively represents others’ social interactions in the posterior superior temporal sulcus (pSTS). These studies, however, have all relied on simple artificial stimuli. It is unclear to what extent social interaction selectivity exists in the real world where social interactions co-vary with many other sensory and social features (such as faces, voices, theory of mind). To address this, a few studies have adopted a naturalist movie viewing paradigm, but none have looked at the unique variance explained by social interactions beyond that explained by other covarying features in natural movies. The current study utilizes machine learning-based fMRI analyses and computer vision techniques to uncover the brain mechanisms uniquely underlying naturalistic social interaction perception. We analyzed two publicly available fMRI datasets, collected while participants watched two different commercial movies in the MRI scanner. By performing voxel-wise encoding and variance partitioning analyses, we demonstrate that a socio-affective model (a linear combination of an agent speaking, social interactions, theory of mind, perceived valence, and arousal) independently contribute to predicting brain responses in the STS and the dorsal medial prefrontal cortex. Importantly, the STS and the precuneus show unique selectivity for scenes that contain social interactions, even after the effects of all other visual and social features, including the presence of faces and theory of mind, have been controlled for. This selectivity generalized across both sets of movie data despite the fact that data came from different genres, subjects, and labs. Together, these findings suggest that social interaction perception recruits dedicated neural circuits during natural viewing and is an essential dimension of social understanding.