Representations of modality-general valence for videos and music derived from fMRI data

Abstract

This study tested for neural representations of valence that are shared across visual and auditory modalities referred to as modality-general representations. On a given trial participants made either affective or semantic judgments of short silent videos or music samples. For each modality valence was manipulated at three levels, positive, neutral, and negative, while controlling for the level of arousal. Whole-brain crossmodal identification of affect indicated the presence of modality-general valence representations that distinguished 1) positive from negative trials (signed valence) and 2) valenced from non-valenced trials (unsigned valence). These results generalized across the two tasks. Brain regions that were sensitive to valence states in the same way for both modalities were identified by searchlight analysis of fMRI data by comparing the correlation of voxel responses to the same and different valence conditions across the two modalities. These analyses identified seven clusters that distinguished signed valence, unsigned valence or both. Signed valence was represented in the precuneus, unsigned valence in the bilateral medial prefrontal cortex, superior temporal sulcus (STS)/postcentral, and middle frontal gyrus (MFG) and both types were represented in the STS/MFG and thalamus. These results support the idea that modality general valence is represented in a network of several locations throughout the brain.