Expression Recognition Based on Multi-feature Fusion of Peak-neutral Differences

Abstract

Facial Expression Recognition (FER) is an important research direction of pattern recognition. This paper proposes a multi-feature fusion network based on peak-neutral difference (MFPND). The network consists of two branches: One branch extracts the local features of the expression changes from the partial image, and the other branch extracts the global features of the expression changes from the entire image. In this model, Local features describe the details of expression changes, which are fused by geometric spatial features and Local Binary Pattern (LBP) features. The global features describe high-level semantic information, which is extracted by the Siamese network composed of Convolutional Neural Networks (CNN). Before classifying, we combined local features with global features. Compared with most current methods of single feature type, these two features represent expression information of different scales, and this model can more comprehensively and effectively represent expression changes. In addition, in the training phase, a new pooling strategy is proposed to deal with disturbances such as illumination and noise during expression changes, which is called intensity transformation-invariant pooling (ITI-pooling). Experiments were performed on the extended Cohn±Kanade (CK+) face recognition database and the Oulu-CASIA data set.