A hybrid framework for smile detection in class imbalance scenarios

Abstract

In this study, we consider the problem of smile detection in both an imbalanced data scenario, in which the number of smile images is in the minority compared with the number of neutral images, and a balanced data scenario. We first propose a smile detection model using a convolutional neural network (SD-CNN) to improve the performance in the balanced data scenario, and then a hybrid deep learning framework (HF-SD) that uses a modification of the SD-CNN model to learn and then extracts the features from dataset. These extracted features are then used to train an extreme gradient boosting approach to handle the imbalanced problem. An experiment shows that the proposed model has impressive discriminative ability for smile detection, in both balanced and imbalanced data scenarios, compared with existing approaches. HF-SD yields an accuracy of 93.6% and outperforms the state-of-the-art approaches for the original GENK14K database in the balanced data scenario. The results of the second experiment show that HF-SD also achieves better AUCs (area under the receiver operating characteristic curve) compared with the state-of-the-art methods for smile detection in an imbalanced data scenario with different balancing ratios.