Emotion Recognition From Body Movement

Ferdous Ahmed,A S M Hossain Bari,Marina L Gavrilova

doi:10.1109/access.2019.2963113

Ferdous Ahmed, A S M Hossain Bari + Show 1 more

Open Access

https://doi.org/10.1109/access.2019.2963113

Copy DOI

Abstract

Automatic emotion recognition from the analysis of body movement has tremendous potential to revolutionize virtual reality, robotics, behavior modeling, and biometric identity recognition domains. A computer system capable of recognizing human emotion from the body can also significantly change the way we interact with the computers. One of the significant challenges is to identify emotion-specific features from a vast number of descriptors of human body movements. In this paper, we introduce a novel two-layer feature selection framework for emotion classification from a comprehensive list of body movement features. We used the feature selection framework to accurately recognize five basic emotions: happiness, sadness, fear, anger, and neutral. In the first layer, a unique combination of Analysis of Variance (ANOVA) and Multivariate Analysis of Variance (MANOVA) was utilized to eliminate irrelevant features. In the second layer, a binary chromosome-based genetic algorithm was proposed to select a feature subset from the relevant list of features that maximizes the emotion recognition rate. Score and rank-level fusion were applied to further improve the accuracy of the system. The proposed system was validated on proprietary and public datasets, containing 30 subjects. Different action scenarios, such as walking and sitting actions, as well as an action-independent case, were considered. Based on the experimental results, the proposed emotion recognition system achieved a very high emotion recognition rate outperforming all of the state-of-the-art methods. The proposed system achieved recognition accuracy of 90.0% during walking, 96.0% during sitting, and 86.66% in an action-independent scenario, demonstrating high accuracy and robustness of the developed method.

Highlights

Emotion recognition based on human body movement is an emerging area of research
The number of relevant features selected from each group was based on the normalized Multivariate Analysis of Variance (MANOVA) score computed for each motion feature group
Significant benefits can be achieved for biometric security, patient behavior monitoring, gaming, and robotics with the creation of a movement-based emotionaware computer system

Summary

INTRODUCTION

Emotion recognition based on human body movement is an emerging area of research. Most of the recent research on emotion recognition is focusing on developing a system that can recognize emotions based on nonverbal cues expressed through body movements [5]. Based on the above discussion, an increasing number of applications, that use body movement information for emotion recognition, has emerged. The number of relevant features selected from each group was based on the normalized MANOVA score computed for each motion feature group. The method outperfomed all of the state-of-the-art approaches tested on our proprietary dataset Information fusion techniques such as score and rank-level fusion further improved the emotion recognition accuracy of the proposed system. Proposing a unique combination of score and rank-level fusion with two-layer feature selection algorithm to maximize the emotion recognition accuracy. Preliminary work on this subject was carried out and published in [23]

PREVIOUS WORK

MOTION FEATURE GROUPS

TEMPORAL PROFILE

Findings

CONCLUSION AND FUTURE WORK