FgbCNN: A unified bilinear architecture for learning a fine-grained feature representation in facial expression recognition

Abstract

A facial expression recognition system has been proposed in this paper. The challenges of the facial expression recognition system lie due to low intra-class variance within a class of negative emotions, such as anger, disgust, and fear. A conventional Convolution Neural Network (CNN) model may extract discriminative features and has shown outstanding performance in various computer vision tasks. However, it is unable to extract the second-order feature information that demonstrates the interaction between features in the image of wild-environment datasets. This paper presents a novel method to solve the facial expression recognition problem by addressing several limitations concerning emotion recognition problems. A deep learning-based bilinear convolutional neural network framework has been proposed, termed an Fine-Grained Bilinear CNN (FgbCNN) model that consists of two branches with optimized CNN along with a normalization layer composed of batch-normalization, square-root normalization, L2-normalization, and drop-out layers. Here, local and holistic features have been aggregated using a dot-product layer to extract more discriminant features. Finally, experimenting with two wild-environments (SFEW 1.0 and SFEW 2.0) and two lab-controlled datasets (KDEF and CK+), it has been observed that the proposed model can minimize the intra-class variances and has attained outstanding performance compared to other state-of-the-art methods.