Recognition Of Body Expressions Research Articles

Affective body expression recognition framework based on temporal and spatial fusion features

Affective body expression recognition technology enables machines to interpret non-verbal emotional signals from human movements, which is crucial for facilitating natural and empathetic human–machine interaction (HCI). This work proposes a new framework for emotion recognition from body movements, providing a universal and effective solution for decoding the temporal–spatial mapping between emotions and body expressions. Compared with previous studies, our approach extracted interpretable temporal and spatial features by constructing a body expression energy model (BEEM) and a multi-input symmetric positive definite matrix network (MSPDnet). In particular, the temporal features extracted from the BEEM reveal the energy distribution, dynamical complexity, and frequency activity of the body expression under different emotions, while the spatial features obtained by MSPDnet capture the spatial Riemannian properties between body joints. Furthermore, this paper introduces an attentional temporal–spatial feature fusion (ATSFF) algorithm to adaptively fuse temporal and spatial features with different semantics and scales, significantly improving the discriminability and generalizability of the fused features. The proposed method achieves recognition accuracies over 90% across four public datasets, outperforming most state-of-the-art approaches.

The priming effect of emotional words on body expressions: Two ERP studies

The impact of emotional words on the recognition of body expression and the underlying neurodynamic mechanisms remain poorly understood. This study used a classic supraliminal priming paradigm and event related potential (ERP) to investigate the effect of emotion-label words (experiment 1) and emotional verbs (experiment 2) on the recognition of body expressions. The behavioral results revealed that individuals exhibited a higher accuracy in recognizing happy expressions when presented with a happy-label word condition, in contrast to neutral expressions. Furthermore, it was observed that the accuracy of recognizing happy body expressions was reduced when preceded by angry verb priming, compared to happy and neutral priming conditions. Conversely, the accuracy of recognizing angry body expressions was higher in response to angry verb priming than happy and neutral primings. The ERP results showed that, in the recognition of happy body expressions, the P300 amplitude elicited by angry-label words was more positive, while a congruent verb-expression condition elicited more positive P300 amplitude than an incongruent condition in the left hemisphere and midline. However, in the recognition of angry body expressions, the N400 amplitude elicited by a congruent verb-expression condition was smaller than that elicited by an incongruent condition. These results suggest that both abstract emotion-label words and specific emotional verbs influence the recognition of body expressions. In addition, integrating happy semantic context and body expression might occur at the P300 stage, whereas integrating angry semantic context and body expression might occur at the N400 stage. These findings present novel evidence regarding the criticality of emotional context in the recognition of emotions.

Age-of-actor effects in body expression recognition of children

Investigations of developmental trajectories for emotion recognition suggest that both face- and body expression recognition increases rapidly in early childhood and reaches adult levels of performance near the age of ten. So far, little is known about whether children's ability to recognise body expressions is influenced by the age of the person they are observing. This question is investigated here by presenting 119 children and 42 young adults with videos of children, young adults and older adults expressing emotions with their whole body. The results revealed an own-age advantage for children, reflected in adult-level accuracy for videos of children for most expressions but reduced accuracy for videos of older adults. Children's recognition of older adults' expressions was not correlated with children's estimated amount of contact with older adults. Support for potential influences of social biases on performance measures was minimal. The own-age advantage was explained in terms of children's reduced familiarity with body expressions of older adults due to aging related changes in the kinematics characteristics of movements and potentially due to stronger embodiment of other children's bodily movements.

Generic Body Expression Recognition Based on Synthesis of Realistic Neutral Motion

Most automatic expression analysis systems attempt to recognize a conventional set of expressions such as happiness, sadness, anger, surprise and fear, etc. Although this set of expressions is the most typical of the face, it is not the most representative/relevant for what the body expressions tell us. This paper presents a novel and generic approach for the recognition of body expressions using human postures. Our method is based on the notion of neutral motion generated from a given expressive one. In a second time, we estimate a residue function, as the difference between the two associated motions, namely the expressive and the neutral motion. More precisely, this function that is inspired by studies from psychology domain, gives a “neutrality” score of a motion. Using this “neutrality score”, we propose a cost function which enables to synthesis the neutral motion from any input expressive motion. The synthesis of neutral motion process is based on two nested Principal Component Analysis providing a space where moving and selecting realistic human animations become possible. Proposed approach is evaluated on four databases with heterogeneous movements and body expressions and it achieved recognition results for body expression recognition that exceed state of the art.

Gaze patterns in viewing static and dynamic body expressions

Evidence for the importance of bodily cues for emotion recognition has grown over the last two decades. Despite this growing literature, it is underspecified how observers view whole bodies for body expression recognition. Here we investigate to which extent body-viewing is face- and context-specific when participants are categorizing whole body expressions in static (Experiment 1) and dynamic displays (Experiment 2). Eye-movement recordings showed that observers viewed the face exclusively when visible in dynamic displays, whereas viewing was distributed over head, torso and arms in static displays and in dynamic displays with faces not visible. The strong face bias in dynamic face-visible expressions suggests that viewing of the body responds flexibly to the informativeness of facial cues for emotion categorisation. However, when facial expressions are static or not visible, observers adopt a viewing strategy that includes all upper body regions. This viewing strategy is further influenced by subtle viewing biases directed towards emotion-specific body postures and movements to optimise recruitment of diagnostic information for emotion categorisation.

Anterior Temporal Lobectomy Impairs Neural Classification of Body Emotions in Right Superior Temporal Sulcus and Reduces Emotional Enhancement in Distributed Brain Areas without Affecting Behavioral Classification.

Humans with amygdalar lesions show proportional reductions of the emotional response to facial expressions in the fusiform face area as well as deficits in emotion recognition from facial expressions. While processing of bodily expressions shares many similarities with facial expressions, there is no substantial evidence that lesions of the amygdala result in similar behavioral and neural sequelae. We combined behavioral assessment with functional neuroimaging in a group of male and female humans with unilateral anterior temporal lobe (ATL) resections, including the amygdala (right: n = 10; left: n = 10) and 12 matched controls. The objective was to assess whether the amygdala is crucial for the recognition of body expressions and for modulatory effects on distant areas during perception of body expressions. The behavioral results revealed normal performance in both patient groups on emotion categorization of body expressions. The neuroimaging results showed that ATL patients displayed no enhanced activations in right fusiform body area and left extrastriate body area and that left ATL patients additionally displayed no enhanced activations in right posterior superior temporal sulcus and right extrastriate body area, respectively. Multivoxel pattern analysis revealed altered categorization capacity between emotional and neutral stimuli in right posterior superior temporal sulcus in right ATL patients. In addition, we also found emotional enhancement in frontal, parietal, occipital, and cingulate regions in controls. Together, our data show that the amygdala and ATLs are not necessary for recognition of dynamic body expressions, but suggest that amygdala lesions affect body emotion processing in distant brain areas.SIGNIFICANCE STATEMENT For humans, information from emotional expressions of others is crucial to support social interactions. The majority of emotion studies has focused on facial expressions; however, in daily life, we also use information from body postures and body movement. Visual processing of body expressions relies on a brain network, including body-specific visual areas and visuomotor areas. Even though the importance of the amygdala and its modulatory effects on distant brain regions have been documented, it remains unclear whether the amygdala plays a crucial role in emotional body processing. By combining behavioral and neuroimaging data in patients with amygdalar lesions, we provide further evidence for its modulatory effect on distant areas during the perception of body expressions.

Validación de una batería para evaluar el reconocimiento de emociones a través del rostro y del cuerpo utilizando estímulos dinámicos

This paper presents validation studies of a new neuropsychological assessment battery, locally developed, designed to assess facial and body expression recognition of basic and complex emotions, with video stimuli. We analyzed item difficulty and item discrimination power of the items (Study 1), convergent validity and reliability (Study 2), and the effect of demographic variables, gender, age and educational level, on the battery tests (Study 3). Results showed that the four emotional tests of the battery had adequate psychometric properties, with appropriate difficulty and discrimination indexes, and satisfactory indicators of validity and reliability. Age and educational level affected performance in all battery tests, but there weren’t significative differences between men and women. In future works it is expected test the performance of the battery in pathological populations in order to obtain clinical validation data.

Validación de una batería para evaluar el reconocimiento de emociones a través del rostro y del cuerpo utilizando estímulos dinámicos

This paper presents validation studies of a new neuropsychological assessment battery, locally developed, designed to assess facial and body expression recognition of basic and complex emotions, with video stimuli. We analyzed item difficulty and item discrimination power of the items (Study 1), convergent validity and reliability (Study 2), and the effect of demographic variables, gender, age and educational level, on the battery tests (Study 3). Results showed that the four emotional tests of the battery had adequate psychometric properties, with appropriate difficulty and discrimination indexes, and satisfactory indicators of validity and reliability. Age and educational level affected performance in all battery tests, but there weren’t significative differences between men and women. In future works it is expected test the performance of the battery in pathological populations in order to obtain clinical validation data.

Dissociation between facial and bodily expressions in emotion recognition: A case study

Objective: Existing single-case studies have reported deficit in recognizing basic emotions through facial expression and unaffected performance with body expressions, but not the opposite pattern. The aim of this paper is to present a case study with impaired emotion recognition through body expressions and intact performance with facial expressions. Methods: In this single-case study we assessed a 30-year-old patient with autism spectrum disorder, without intellectual disability, and a healthy control group (n = 30) with four tasks of basic and complex emotion recognition through face and body movements, and two non-emotional control tasks. To analyze the dissociation between facial and body expressions, we used Crawford and Garthwaite’s operational criteria, and we compared the patient and the control group performance with a modified one-tailed t-test designed specifically for single-case studies. Results: There were no statistically significant differences between the patient’s and the control group’s performances on the non-emotional body movement task or the facial perception task. For both kinds of emotions (basic and complex) when the patient’s performance was compared to the control group’s, statistically significant differences were only observed for the recognition of body expressions. There were no significant differences between the patient’s and the control group’s correct answers for emotional facial stimuli. Conclusions: Our results showed a profile of impaired emotion recognition through body expressions and intact performance with facial expressions. This is the first case study that describes the existence of this kind of dissociation pattern between facial and body expressions of basic and complex emotions.

Age-congruency and contact effects in body expression recognition from point-light displays (PLD).

Recognition of older people’s body expressions is a crucial social skill. We here investigate how age, not just of the observer, but also of the observed individual, affects this skill. Age may influence the ability to recognize other people’s body expressions by changes in one’s own ability to perform certain action over the life-span (i.e., an own-age bias may occur, with best recognition for one’s own age). Whole body point light displays of children, young adults and older adults (>70 years) expressing six different emotions were presented to observers of the same three age-groups. Across two variations of the paradigm, no evidence for the predicted own-age bias (a cross-over interaction between one’s own age and the observed person’s age) was found. Instead, experience effects were found with children better recognizing older actors’ expressions of ‘active emotions,’ such as anger and happiness with greater exposure in daily life. Together, the findings suggest that age-related changes in one own’s mobility only influences body expression categorization in young children who interact frequently with older adults.

Impaired recognition of body expressions in the behavioral variant of frontotemporal dementia

Progressive deterioration of social cognition and emotion processing are core symptoms of the behavioral variant of frontotemporal dementia (bvFTD). Here we investigate whether bvFTD is also associated with impaired recognition of static (Experiment 1) and dynamic (Experiment 2) bodily expressions. In addition, we compared body expression processing with processing of static (Experiment 3) and dynamic (Experiment 4) facial expressions, as well as with face identity processing (Experiment 5). The results reveal that bvFTD is associated with impaired recognition of static and dynamic bodily and facial expressions, while identity processing was intact. No differential impairments were observed regarding motion (static vs. dynamic) or category (body vs. face). Within the bvFTD group, we observed a significant partial correlation between body and face expression recognition, when controlling for performance on the identity task. Voxel-Based Morphometry (VBM) analysis revealed that body emotion recognition was positively associated with gray matter volume in a region of the inferior frontal gyrus (pars orbitalis/triangularis). The results are in line with a supramodal emotion recognition deficit in bvFTD.

Social context influences recognition of bodily expressions

Previous studies have shown that recognition of facial expressions is influenced by the affective information provided by the surrounding scene. The goal of this study was to investigate whether similar effects could be obtained for bodily expressions. Images of emotional body postures were briefly presented as part of social scenes showing either neutral or emotional group actions. In Experiment 1, fearful and happy bodies were presented in fearful, happy, neutral and scrambled contexts. In Experiment 2, we compared happy with angry body expressions. In Experiment 3 and 4, we blurred the facial expressions of all people in the scene. This way, we were able to ascribe possible scene effects to the presence of body expressions visible in the scene and we were able to measure the contribution of facial expressions to the body expression recognition. In all experiments, we observed an effect of social scene context. Bodily expressions were better recognized when the actions in the scenes expressed an emotion congruent with the bodily expression of the target figure. The specific influence of facial expressions in the scene was dependent on the emotional expression but did not necessarily increase the congruency effect. Taken together, the results show that the social context influences our recognition of a person’s bodily expression.