Emotional Body Expressions Research Articles

Facial feedback manipulation influences the automatic detection of unexpected emotional body expressions

Unexpected or changing facial expressions are known to be able to engage more automatic processing than frequently occurring facial expressions, thereby inducing a neural differential wave response known as expression mismatch negativity (EMMN). Recent studies have shown that EMMN can be modulated by the observer's facial feedback (i.e., feedback from their own facial movements). A similar EMMN activity has been discovered for body expressions, but thus far only a few emotion types have been investigated. It is unknown whether the EMMNs evoked by body expressions can be influenced by facial feedback. To explore this question, we recorded EEG activity of 29 participants in the reverse oddball paradigm. Here two unexamined categories of body expressions were presented, happy and sad, placed in two paired stimulus sequences: in one the happy body was presented with a probability of 80% (standards) while the sad body was presented with a probability of 20% (deviants), and in the other the probabilities were reversed. The facial feedback was manipulated by different pen holding conditions (i.e., participants holding the pen with the teeth, lips, or nondominant hand). The nonparametric cluster permutation test revealed significant happy and sad body-related EMMN (bEMMN) activities. The happy-bEMMN were more negative than sad-bEMMN within the range of 100–150 ms. Additionally, the bEMMN amplitude of both emotions is modulated by the facial feedback conditions. These results expand the range of emotional types applicable to bEMMN and provide evidence for the validity of the facial feedback hypothesis across emotional carriers.

Preschoolers Can Match the Facial and Bodily Emotional Expressions: A Behavior and Eye-Tracking Study

The preschool age is an important period in the development of children’s emotional abilities. The face and body are both vital carriers of emotional information and adults readily match the emotional cues coming from these two modalities. However, it is unknown whether preschoolers have this ability or not. In the current study, 36 preschoolers (22 boys, aged 44–73 months old) participated in an emotional facial and bodily matching test including angry, happy, and neutral expressions (Experiment 1) and an eye-tracking experiment (Experiment 2). The results show that: (1) preschoolers can accurately match three categories of facial and bodily expressions of emotion (all matching rates were above 93% in Experiment 1); (2) robust emotional matching was observed across conditions in Experiment 2. That is, the eye-tracking indexes including the total fixation duration, the first fixation duration, and the fixation count, all indicated that the preschoolers more thoroughly processed bodily expressions of emotion which were congruent to the facial expressions compared to incongruent bodily expressions of emotion. This study shows that the ability to match facial and bodily expressions of emotion develops at an early age. Preschool children have already reached a high level of this ability.

Impairment in reading negative social cues extends beyond the face in autism

Nonverbal expressions are essential to regulating social communication and interaction. Impaired emotion recognition from facial expressions has been linked to various psychiatric conditions characterized by severe social deficits such as autism. As body expressions as an additional source of social-emotional information have attracted little research attention, little is known about whether emotion recognition impairments are specific to faces, or extend to body expressions. This study explored and compared emotion recognition from face versus body expressions in autism spectrum disorder. We compared 30 men with autism spectrum disorder to 30 male age- and IQ-matched control participants in their ability to recognize angry, happy, and neutral expressions from dynamic face and body expressions. Participants with autism spectrum disorder showed impaired recognition of angry expressions from both faces and bodies, while there were no group differences in recognizing happy and neutral expressions. In autism spectrum disorder, recognizing angry face expressions was inversely predicted by gaze avoidance, while recognizing angry body expressions was inversely predicted by impairments in social interaction and autistic traits. These findings suggest that distinct mechanisms may underlie the impaired emotion recognition from face and body expressions in autism spectrum disorder, respectively. Overall, our study demonstrates that emotion-specific recognition difficulties in autism spectrum disorder are not limited to face expressions but extend to emotional body expressions.

Turn that music down! Affective musical bursts cause an auditory dominance in children recognizing bodily emotions

Previous work has shown that different sensory channels are prioritized across the life course, with children preferentially responding to auditory information. The aim of the current study was to investigate whether the mechanism that drives this auditory dominance in children occurs at the level of encoding (overshadowing) or when the information is integrated to form a response (response competition). Given that response competition is dependent on a modality integration attempt, a combination of stimuli that could not be integrated was used so that if children’s auditory dominance persisted, this would provide evidence for the overshadowing over the response competition mechanism. Younger children (≤7 years), older children (8–11 years), and adults (18+ years) were asked to recognize the emotion (happy or fearful) in either nonvocal auditory musical emotional bursts or human visual bodily expressions of emotion in three conditions: unimodal, congruent bimodal, and incongruent bimodal. We found that children performed significantly worse at recognizing emotional bodies when they heard (and were told to ignore) musical emotional bursts. This provides the first evidence for auditory dominance in both younger and older children when presented with modally incongruent emotional stimuli. The continued presence of auditory dominance, despite the lack of modality integration, was taken as supportive evidence for the overshadowing explanation. These findings are discussed in relation to educational considerations, and future sensory dominance investigations and models are proposed.

Emotional body expressions facilitate working memory: Evidence from the n-back task.

In daily life, individuals need to recognize and update emotional information from others' changing body expressions. However, whether emotional bodies can enhance working memory (WM) remains unknown. In the present study, participants completed a modified n-back task, in which they were required to indicate whether a presented image of an emotional body matched that of an item displayed before each block (0-back) or two positions previously (2-back). Each block comprised only fear, happiness, or neutral. We found that in the 0-back trials, when compared with neutral body expressions, the participants took less time and showed comparable ceiling effects for accuracy in happy bodies followed by fearful bodies. When WM load increased to 2-back, both fearful and happy bodies significantly facilitated WM performance (i.e., faster reaction time and higher accuracy) relative to neutral conditions. In summary, the current findings reveal the enhancement effect of emotional body expressions on WM and highlight the importance of emotional action information in WM.

Emotional body postures affect inhibitory control only when task-relevant.

A classical theoretical frame to interpret motor reactions to emotional stimuli is that such stimuli, particularly those threat-related, are processed preferentially, i.e., they are capable of capturing and grabbing attention automatically. Research has recently challenged this view, showing that the task relevance of emotional stimuli is crucial to having a reliable behavioral effect. Such evidence indicated that emotional facial expressions do not automatically influence motor responses in healthy young adults, but they do so only when intrinsically pertinent to the ongoing subject's goals. Given the theoretical relevance of these findings, it is essential to assess their generalizability to different, socially relevant emotional stimuli such as emotional body postures. To address this issue, we compared the performance of 36 right-handed participants in two different versions of a Go/No-go task. In the Emotional Discrimination task, participants were required to withhold their responses at the display of emotional body postures (fearful or happy) and to move at the presentation of neutral postures. Differently, in the control task, the same images were shown, but participants had to respond according to the color of the actor/actress' t-shirt, disregarding the emotional content. Results showed that participants made more commission errors (instances in which they moved even though the No-go signal was presented) for happy than fearful body postures in the Emotional Discrimination task. However, this difference disappeared in the control task. Such evidence indicates that, like facial emotion, emotional body expressions do not influence motor control automatically, but only when they are task-relevant.

The shared signal hypothesis: Facial and bodily expressions of emotion mutually inform one another.

Decades of research show that contextual information from the body, visual scene, and voices can facilitate judgments of facial expressions of emotion. To date, most research suggests that bodily expressions of emotion offer context for interpreting facial expressions, but not vice versa. The present research aimed to investigate the conditions under which mutual processing of facial and bodily displays of emotion facilitate and/or interfere with emotion recognition. In the current two studies, we examined whether body and face emotion recognition are enhanced through integration of shared emotion cues, and/or hindered through mixed signals (i.e., interference). We tested whether faces and bodies facilitate or interfere with emotion processing by pairing briefly presented (33 ms), backward-masked presentations of faces with supraliminally presented bodies (Experiment 1) and vice versa (Experiment 2). Both studies revealed strong support for integration effects, but not interference. Integration effects are most pronounced for low-emotional clarity facial and bodily expressions, suggesting that when more information is needed in one channel, the other channel is recruited to disentangle any ambiguity. That this occurs for briefly presented, backward-masked presentations reveals low-level visual integration of shared emotional signal value.

EmBody/EmFace as a new open tool to assess emotion recognition from body and face expressions

Nonverbal expressions contribute substantially to social interaction by providing information on another person’s intentions and feelings. While emotion recognition from dynamic facial expressions has been widely studied, dynamic body expressions and the interplay of emotion recognition from facial and body expressions have attracted less attention, as suitable diagnostic tools are scarce. Here, we provide validation data on a new open source paradigm enabling the assessment of emotion recognition from both 3D-animated emotional body expressions (Task 1: EmBody) and emotionally corresponding dynamic faces (Task 2: EmFace). Both tasks use visually standardized items depicting three emotional states (angry, happy, neutral), and can be used alone or together. We here demonstrate successful psychometric matching of the EmBody/EmFace items in a sample of 217 healthy subjects with excellent retest reliability and validity (correlations with the Reading-the-Mind-in-the-Eyes-Test and Autism-Spectrum Quotient, no correlations with intelligence, and given factorial validity). Taken together, the EmBody/EmFace is a novel, effective (< 5 min per task), highly standardized and reliably precise tool to sensitively assess and compare emotion recognition from body and face stimuli. The EmBody/EmFace has a wide range of potential applications in affective, cognitive and social neuroscience, and in clinical research studying face- and body-specific emotion recognition in patient populations suffering from social interaction deficits such as autism, schizophrenia, or social anxiety.

Duchenne smiles are actions not mere happenings: lessons from the debate on expressive action

In this paper, I will argue that, contrary to what is generally assumed in the debate on expressive action, we do not have good reasons to exclude facial and bodily expressions of emotion such as smiling or frowning from the category of actions. For this purpose, I will compare facial and bodily expressions of emotion with simple expressive actions, such as jumping for joy or covering one’s face in shame. I will try to show that simple expressive actions cannot be presented as actions while excluding facial and bodily expressions of emotion from this condition. My contention will then be that either both sorts of behaviour are to be identified as actions or neither is. The latter sounds rather implausible, though, as we would have to assimilate jumping for joy or covering one’s face in shame to spasms, which conflicts with the way we relate to such behaviours. My conclusion will then be that both simple expressive actions and facial and bodily expressions of emotion should be included within the category of actions, at least on the basis of the main assumptions in the current debate on expressive action.

Learning Unseen Emotions from Gestures via Semantically-Conditioned Zero-Shot Perception with Adversarial Autoencoders

We present a novel generalized zero-shot algorithm to recognize perceived emotions from gestures. Our task is to map gestures to novel emotion categories not encountered in training. We introduce an adversarial autoencoder-based representation learning that correlates 3D motion-captured gesture sequences with the vectorized representation of the natural-language perceived emotion terms using word2vec embeddings. The language-semantic embedding provides a representation of the emotion label space, and we leverage this underlying distribution to map the gesture sequences to the appropriate categorical emotion labels. We train our method using a combination of gestures annotated with known emotion terms and gestures not annotated with any emotions. We evaluate our method on the MPI Emotional Body Expressions Database (EBEDB) and obtain an accuracy of 58.43%. We see an improvement in performance compared to current state-of-the-art algorithms for generalized zero-shot learning by an absolute 25-27%. We also demonstrate our approach on publicly available online videos and movie scenes, where the actors' pose has been extracted and map to their respective emotive states.

Differential beta desynchronisation responses to dynamic emotional facial expressions are attenuated in higher trait anxiety and autism

Daily life demands that we differentiate between a multitude of emotional facial expressions (EFEs). The mirror neuron system (MNS) is becoming increasingly implicated as a neural network involved with understanding emotional body expressions. However, the specificity of the MNS’s involvement in emotion recognition has remained largely unexplored. This study investigated whether six basic dynamic EFEs (anger, disgust, fear, happiness, sadness, and surprise) would be differentiated through event-related desynchronisation (ERD) of sensorimotor alpha and beta oscillatory activity, which indexes sensorimotor MNS activity. We found that beta ERD differentiated happy, fearful, and sad dynamic EFEs at the central region of interest, but not at occipital regions. Happy EFEs elicited significantly greater central beta ERD relative to fearful and sad EFEs within 800 - 2,000 ms after EFE onset. These differences were source-localised to the primary somatosensory cortex, which suggests they are likely to reflect differential sensorimotor simulation rather than differential attentional engagement. Furthermore, individuals with higher trait anxiety showed less beta ERD differentiation between happy and sad faces. Similarly, individuals with higher trait autism showed less beta ERD differentiation between happy and fearful faces. These findings suggest that the differential simulation of specific affective states is attenuated in individuals with higher trait anxiety and autism. In summary, the MNS appears to support the skills needed for emotion processing in daily life, which may be influenced by certain individual differences. This provides novel evidence for the notion that simulation-based emotional skills may underlie the emotional difficulties that accompany affective disorders, such as anxiety.

Automatic facial reactions to emotional body expressions are not driven by emotional experience.

Perceiving emotional expressions automatically triggers a tendency to react with a matching facial expression. Although it is considered fundamental for healthy social interactions, the mechanism behind it is unclear. One prevalent explanation suggests that perceiving emotional expressions induces emotions in the observer and that it is these emotions that elicit the facial reactions. This study directly tested this hypothesis, investigating whether emotion elicitation is what drives the effect. Two experiments used a facial stimulus-response compatibility (SRC) paradigm-a widely used measure of the tendency to facially match emotional expressions-in which the irrelevant stimuli were happy and angry body postures. Reaction times were measured using facial electromyography. Experiment 1 replicated the known SRC effect to body postures using a simpler task with only one, prespecified, response. This established a novel variant of the paradigm in which the facial effects cannot be attributed to motor matching or response selection and which focuses specifically on the automatic components of the effect. Experiment 2 then added to this paradigm a habituation protocol and self-report ratings of affective valence. Results indicated that emotional body postures elicited limited emotional reactions, which were further habituated following repeated presentations. However, the facial SRC effect did not undergo such habituation, suggesting that reducing emotional reaction to observed expressions does not reduce the tendency to match those expressions. Our findings do not support the emotion elicitation hypothesis and suggest that automatic facial reactions to emotional body postures are not driven by emotional reactions to the stimuli. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The influence of body expression, group affiliation and threat proximity on interactions in virtual reality

• Immersive virtual reality was used to create ecologically valid threat scenarios. • Participants embodied either a white-skinned or a black-skinned virtual body (VB). • Heart rate and body sway reduction were taken as indexes of freezing reactivity. • Freezing was observed when participants faced aggressive-looking and proximal avatars. • Freezing was observed for ingroup aggression when embodying a black-skinned VB. Social threat requires fast adaptive reactions. One prominent threat-coping behavior present in both humans and other species is freezing, of which heart rate deceleration and reduced postural mobility are two key components. Previous studies mostly focused on freezing reactions in rodents, but now virtual reality offers unique possibilities for controlled and ecologically valid lab-based experiments in humans. This immersive virtual reality study examined how several understudied aspects of social threat, i.e., emotional body expressions, group affiliation, and physical distance from the potential threat, affect freezing behavior in humans. We hypothesized that freezing would be observed for approaching aggressive-looking virtual characters and for virtual characters situated in close proximity. Furthermore, we predicted an enhanced freezing response for approaching, aggressive outgroup members. As expected, reduced heart rate and postural mobility were observed in participants when they faced aggressive-looking and proximal virtual characters. Freezing was also observed for ingroup aggression, specifically when participants were embodied in a black-skinned virtual body and faced black-skinned aggressive and proximal virtual characters. Our results provide novel evidence on the social factors that elicit freezing behavior in humans. Importantly, this evidence is based on a highly ecological virtual reality paradigm that enables people to experience a threatening scenario “as if” it was actually happening to them.

Cerebellar Contribution to Emotional Body Language Perception.

Body language is a powerful form of non-verbal communication providing important information about the emotions and intentions of others. The ability to infer other's emotions from their bodily movements and postures recruits an extended network in the brain that encompasses both cortical and subcortical regions. In this chapter, we review recent evidence suggesting that the cerebellum is a critical node of this network. Specifically, we present convergent findings from patients', neuroimaging and non-invasive brain stimulation studies that have shown that the cerebellum is involved in both biological motion perception and in discrimination of bodily emotional expressions. We discuss the potential underlying mechanisms that drive the recruitment of the sensorimotor (anterior) and cognitive (posterior) cerebellum in inferring others' emotions through their bodily movements and postures and how the cerebellum may exert these functions within different cortico-cerebellar and limbic-cerebellar networks dedicated to body language perception.

Beyond Face Value: Evidence for the Universality of Bodily Expressions of Emotion.

The online version contains supplementary material available at 10.1007/s42761-021-00052-y.

Recognition Characteristics of Facial and Bodily Expressions: Evidence From ERPs.

In the natural environment, facial and bodily expressions influence each other. Previous research has shown that bodily expressions significantly influence the perception of facial expressions. However, little is known about the cognitive processing of facial and bodily emotional expressions and its temporal characteristics. Therefore, this study presented facial and bodily expressions, both separately and together, to examine the electrophysiological mechanism of emotional recognition using event-related potential (ERP). Participants assessed the emotions of facial and bodily expressions that varied by valence (positive/negative) and consistency (matching/non-matching emotions). The results showed that bodily expressions induced a more positive P1 component and a shortened latency, whereas facial expressions triggered a more negative N170 and prolonged latency. Among N2 and P3, N2 was more sensitive to inconsistent emotional information and P3 was more sensitive to consistent emotional information. The cognitive processing of facial and bodily expressions had distinctive integrating features, with the interaction occurring in the early stage (N170). The results of the study highlight the importance of facial and bodily expressions in the cognitive processing of emotion recognition.

Can Children Recognize Bodily Expressions of Emotion?

Can Children Recognize Bodily Expressions of Emotion?

Difficulties in Recognising Dynamic but not Static Emotional Body Movements in Autism Spectrum Disorder

In this study, we investigated whether the difficulties in body motion (BM) perception may led to deficit in emotion recognition in Autism spectrum disorder (ASD). To this aim, individuals with high-functioning ASD were asked to recognise fearful, happy, and neutral BM depicted as static images or dynamic point-light and full-light displays. Results showed slower response times in participants with ASD only in recognising dynamic stimuli, but no group differences in accuracy. This suggests that i) a deficit in action chaining mechanism in ASD may prevent the recognition of dynamic BM automatically and rapidly, ii) individuals with ASD and high cognitive resources can develop alternative—but equally successful—strategies to recognise emotional body expressions. Implications for treatment are discussed

Gait Patterns and Mood in Everyday Life: A Comparison Between Depressed Patients and Non-depressed Controls

BackgroundPrevious laboratory findings suggest deviant gait characteristics in depressed individuals (i.e., reduced walking speed and vertical up-and-down movements, larger lateral swaying movements, slumped posture). However, since most studies to date assessed gait in the laboratory, it is largely an open question whether this association also holds in more naturalistic, everyday life settings. Thus, within the current study we (1) aimed at replicating these results in an everyday life and (2) investigated whether gait characteristics could predict change in current mood.MethodsWe recruited a sample of patients (n = 35) suffering from major depressive disorder and a sample of age and gender matched non-depressed controls (n = 36). During a 2-day assessment we continuously recorded gait patterns, general movement intensity and repetitively assessed the participant’s current mood.ResultsWe replicated previous laboratory results and found that patients as compared to non-depressed controls showed reduced walking speed and reduced vertical up-and-down movements, as well as a slumped posture during everyday life episodes of walking. Moreover, independent of clinical diagnoses, higher walking speed, and more vertical up-and-down movements significantly predicted more subsequent positive mood, while changes in mood did not predict subsequent changes in gait patterns.ConclusionIn sum, our results support expectations that embodiment (i.e., the relationship between bodily expression of emotion and emotion processing itself) in depression is also observable in naturalistic settings, and that depression is bodily manifested in the way people walk. The data further suggest that motor displays affect mood in everyday life.

Bodily emotional expressions are a primary source of information for dogs, but not for humans

Dogs have remarkable abilities to synergise their behaviour with that of people, but how dogs read facial and bodily emotional cues in comparison to humans remains unclear. Both species share the same ecological niche, are highly social and expressive, making them an ideal comparative model for intra- and inter-species emotion perception. We compared eye-tracking data from unrestrained humans and dogs when viewing dynamic and naturalistic emotional expressions in humans and dogs. Dogs attended more to the body than the head of human and dog figures, unlike humans who focused more on the head of both species. Dogs and humans also showed a clear age effect that reduced head gaze. Our results indicate a species-specific evolutionary adaptation for emotion perception, which is only partly modified for heterospecific cues. These results have important implications for managing the risk associated with human–dog interactions, where expressive and perceptual differences are crucial.