LCBP-STGCN: A local cube binary pattern spatial temporal graph convolutional network for micro-expression recognition

Abstract

Automated micro-expression recognition has become a research highlight in the emotion recognition field. Recent works proposed an LCBP (Local Cube Binary Pattern) method for micro-expression recognition and made full use of spatiotemporal features to represent micro-expressions. Nevertheless, LCBP misses the features while ignoring the underlying discriminative information. In this paper, we present an LCBP-STGCN (Local Cube Binary Pattern Spatial-Temporal Graph Convolutional Network) to resolve the problems of LCBP. A new STGCN with the ability to handle non-Euclidean structure data is proposed to extract high-level features of the micro-expression. STGCN is composed of Spatial Graph Convolutional Network (SGCN) to obtain spatial information and Temporal Convolutional Network (TCN) to capture temporal information of micro-expression. To validly establish the spatiotemporal graph structure of SGCN, we apply ROI (Region of Interest) as node position, LCBP features as node information. By the alternating convolution of SGCN and TCN, high-level spatiotemporal features can be obtained. The extensive experiments on four spontaneous micro-expression datasets of SMIC, CASME I, CASME II, and SAMM demonstrate the proposed LCBP-STGCN can effectively recognize micro-expressions and achieve better performance than some state-of-the-arts.