Spatio-temporal texture-based feature extraction for spontaneous facial expression recognition

Abstract

Recently, recognition of naturalistic expressions known as spontaneous facial expressions has attracted attention from researchers due to its significant application in behavioral and clinical research. Currently, most of the work consider recognition of posed expressions. In this paper, we propose a spatio-temporal feature extraction method, Spatio-Temporal Texture Map (STTM), for recognition of spontaneous expressions and compare its performance against that of state-of-the-art feature extraction methods. Both appearance-based and geometry-based feature extraction approaches are considered for comparisons against STTM. The appearance-based techniques considered are Volume Local Binary Pattern (VLBP) and Local Binary Pattern from Three Orthogonal Planes (LBP-TOP) whereas a multi-view tree-based face detector is considered as a geometry-based technique. Support Vector Machine (SVM) is used as the classifier where the extracted features are classified into classes of naturalistic expressions. The feature extraction methods are evaluated over the spontaneous facial expression data from CASME II database. Experimental results show that STTM is capable of recognizing spontaneous expressions and outperforming the other methods in terms of recognition rate, accuracy and computational cost.