Micro-expression spotting: A new benchmark

Thuong-Khanh Tran,Quang-Nhat Vo,Xiaopeng Hong,Xiaobai Li,Guoying Zhao

doi:10.1016/j.neucom.2021.02.022

Thuong-Khanh Tran, Quang-Nhat Vo + Show 3 more

Open Access

https://doi.org/10.1016/j.neucom.2021.02.022

Copy DOI

Journal: Neurocomputing	Publication Date: Mar 2, 2021
Citations: 29	License type: cc-by

Affiliation: University of Oulu, Xi'an Jiaotong University

Abstract

Micro-expressions (MEs) are brief and involuntary facial expressions that occur when people are trying to hide their true feelings or conceal their emotions. Based on psychology research, MEs play an important role in understanding genuine emotions, which leads to many potential applications. Therefore, ME analysis has become an attractive topic for various research areas, such as psychology, law enforcement, and psychotherapy. In the computer vision field, the study of MEs can be divided into two main tasks, spotting and recognition, which are used to identify positions of MEs in videos and determine the emotion category of the detected MEs, respectively. Recently, although much research has been done, no fully automatic system for analyzing MEs has yet been constructed on a practical level for two main reasons: most of the research on MEs only focuses on the recognition part, while abandoning the spotting task; current public datasets for ME spotting are not challenging enough to support developing a robust spotting algorithm. The contributions of this paper are threefold: (1) we introduce an extension of the SMIC-E database, namely the SMIC-E-Long database, which is a new challenging benchmark for ME spotting; (2) we suggest a new evaluation protocol that standardizes the comparison of various ME spotting techniques; (3) extensive experiments with handcrafted and deep learning-based approaches on the SMIC-E-Long database are performed for baseline evaluation.

Highlights

Affective computing is a research field that processes, recognizes, interprets, and simulates human emotions, which play an important role in human-machine interaction analysis
For the frames with large head movements, we reduced the value of M and conducted face alignment of the processing video once again to ensure the quality of the face alignment
To provide a more informative comparison, we present the results of several methods following the Detection Error Tradeoff (DET) curve evaluation protocol, utilizing the research of Tran et al [22]

Summary

Introduction

Affective computing is a research field that processes, recognizes, interprets, and simulates human emotions, which play an important role in human-machine interaction analysis. Affective computing can be related to voices, facial expressions, gestures, and bio-signals [1]. Facial expressions (FEs) are certainly one of the most important channels used by people to convey internal emotions. There has been much research on the topic of FE recognition. Several state-of-the-art FE recognition methods reported an accuracy rate of more than 90% [1]. Aside from ordinary FEs, under certain cases emotions can manifest themselves in a special form called “micro-expressions” (MEs). Aside from ordinary FEs, under certain cases emotions can manifest themselves in a special form called “micro-expressions” (MEs). [2, 3, 4]

Objectives

Methods

Results

Conclusion