3D facial expression recognition algorithm using local threshold binary pattern and histogram of oriented gradient

Abstract

Facial expression which carries rich information of body behavior is the leading carrier of human affective and the symbol of intelligence. The main purpose of this paper is to recognize 3D human facial expression. The research in this paper includes the expression feature extraction algorithm and fusion with different kinds of feature. To contain more local texture feature information, we proposed a new feature of 3D facial expression named Local Threshold Binary Pattern (LTBP) which based on Local Binary Pattern (LBP). We calculate the difference of gray value standard between neighboring pixels and the center pixel as a threshold to binary instead of the traditional LBP operation which only comparison of size between neighboring pixels and the center pixel. After we get the LTBP feature, we fuse the LTBP and HOG (Histogram of Oriented Gradient) features to get multi-feature fusion for 3D facial expression recognition. Our algorithm of 3D facial expression recognition comprises three steps: (1) extracting two sets of feature vectors and establishing the correlation criterion function between the two sets of feature vectors; (2) solving the two sets canonical projective vectors and extracting their canonical correlation features by the framework of canonical correlation analysis algorithm; (3) doing feature fusion for classification by using proposed strategy. We have performed comprehensive experiments on the BU-3DFE database which is presently the largest available 3D face database. We have achieved verification rates of more than 90% for the 3D facial expression recognition.