Recognition of human emotion transition from video sequence using triangulation induced various centre pairs distance signatures

Abstract

Human emotion recognition manifests the concept of contextual understanding of human communication in nonverbal form. This fascinates us to monitor the dynamic information of an emotion. In this article, we present an automatic recognition system for observing gradual changes in human emotion available in the form of a video sequence. In our proposed system, we introduce a triangulation mechanism for geometric feature generation from salient landmarks on the face image. Here, well known active appearance model (AAM) is applied to consecutive image frames in a sequence to track all locations of landmark points. In this work, we explore four centre points (Incentre, Circumcentre, Barycentre, and Orthocentre) from all the triangles formed by salient facial points selected in a triplet. Six different geometric distance signatures viz. Incentre–Circumcentre, Incentre–Barycentre, Incentre–Orthocentre, Circumcentre–Barycentre, Circumcentre–Orthocentre, and Barycentre–Orthocentre are extracted accordingly by taking all possible pairs of those four centres into account to signify several ways of the geometric representation of an emotional image sequence. The discriminative merit of each distance signature is analysed independently by employing them separately as input feature sets into a multilayer perceptron (MLP) classifier for recognition of six basic emotions(anger, disgust, fear, happiness, sadness, and surprise). We verify the performance of our proposed system through the experimentation on several publically available benchmark video datasets such as Extended Cohn–Kanade (CK＋), M&M Initiative (MMI), and Multimedia Understanding Group (MUG). Experimental results indicate impressive accuracy of 98.47% for CK＋, 92.57% for MMI, and 98.62% for the MUG dataset on Incentre–Circumcentre pair distance signature. Moreover, to justify the effectiveness of our proposed method we compare our experimental results with other existing states of the art available in the literature.