The representational dynamics of perceived voice emotions evolve from categories to dimensions.

Bruno L Giordano,Pascal Belin,Joachim Gross,Nikolaus Kriegeskorte,Sonja A Kotz,Caroline Whiting

doi:10.1038/s41562-021-01073-0

Abstract

Long-standing affective science theories conceive the perception of emotional stimuli either as discrete categories (e.g., an angry voice) or continuous dimensional attributes (e.g., an intense and negative vocal emotion). Which position provides a better account is still widely debated. Here, we contrast them to account for acoustics-independent perceptual and cerebral representational geometry of perceived voice emotions. We combined multimodal imaging of the cerebral response to heard vocal stimuli (functional magnetic resonance imaging – fMRI – and magneto-encephalography – MEG) with post-scanning behavioral assessment of voice emotion perception. By using representational similarity analysis (RSA), we find that categories prevail in perceptual and early (< 200ms) fronto-temporal cerebral representational geometries and that dimensions impinge predominantly on a later limbic-temporal network (240ms and > 500ms). These results reconcile the two long-opposing views by reframing the perception of emotions as the interplay of cerebral networks with different representational dynamics that emphasize either categories or dimensions.

Highlights

Sampled densely the space of perceived emotions (Fig. 1a)
With representational similarity analysis (RSA), we examine the representational geometry of the cerebral response to emotional voices as a window onto the cerebral representation of vocal emotion
Data from each of the emotion rating tasks were selectively modulated by at least two of the morphing weights (p ≤ 0.05 family-wise error (FWE)-corrected across rating tasks and morphing weights (absolute significant T(9) ≥ 4.343, permutation-based two-tailed p ≤ 0.039 corrected for multiple comparisons across pairs of rating scales with morphing weights, absolute Fisher Z scale chance-corrected s.p.r ≥ 0.042, SEM ≤ 0.077, widest percentile bootstrap 95% CI = -0.522/-0.394; absolute non-significant T(9) ≤ 3.801, p ≥ 0.078, absolute s.p.r

Summary

Introduction

Sampled densely the space of perceived emotions (Fig. 1a). Morphing was performed independently on recordings from two different actors (one male, one female) to dissociate general emotional from identity- and gender-related acoustics, resulting in 78 stimuli overall. With RSA, we examine the representational geometry of the cerebral response to emotional voices as a window onto the cerebral representation of vocal emotion This is performed by abstracting across participants over the specific structure of the cerebral response pattern that carries information about perceived emotions (e.g., potential lower latency evoked response for fear stimuli). As low-level acoustical properties could plausibly influence both perceptual and cerebral responses to the stimuli, we considered their acoustic structure as reflected by their spectro-temporal modulations. For this we analyzed stimuli using banks of spectro-temporal modulation filters that can intuitively be described as the auditory analogue of Gabor-like representations in the early visual system 18. We built acoustic 39x39 representational dissimilarity matrices (RDM) capturing overall acoustic differences for each post-onset time window (Fig. 1d) and used them in subsequent analyses to remove acoustic confounds

Objectives

Methods

Results

Conclusion