Facial Expressions of Emotion Transmit Multiplexed Signals of Categorical and Dimensional Information

Abstract

Human facial expressions are complex dynamic signals that are used to communicate emotions 1. As a highly evolved social species, human facial expressions likely signal complex information to inform adaptive response 2. However, a long-standing debate is what information such facial expressions transmit. One influential theory posits that facial expressions transmit discrete categorical information such as happy or anger 3–6, while another argues they transmit broad dimensional information such as valence and arousal 7–10 (see 11 for a review). Here, we tested an alternative that reconciles these contrasting theories – that facial expressions are multiplexed signals that jointly transmit categorical and dimensional information. Using a data-driven approach 12,13 and subjective affective perception, we first reconstructed the facial expression signals that transmit emotion category information (i.e., the six classic emotions, plus 19 complex emotions) and, separately, those that transmit affective dimensional information (i.e., valence and arousal). Cross-comparison showed that facial expressions of emotion categories are systematically embedded as discrete clusters within those of affective dimensions, suggesting that certain facial movements jointly transmit emotion categories and affective dimensions as multiplexed signals. Next, using a novel information-theoretic analysis 14,15, we disentangled the specific information that each facial movement transmits – emotion categories, affective dimensions, or both as multiplex signals. Results showed that facial expressions of emotion primarily comprise complexed multiplex signals that jointly transmit emotion category and affective dimensional information. Our results unite long-standing contrasted theories of emotion communication and provide a novel framework that can better characterize the complexities of human facial expression signalling.