A novel deep learning system for facial feature extraction by fusing CNN and MB-LBP and using enhanced loss function

Abstract

Extensive use of surveillance cameras for human tracking and observation have been fostering the research on face recognition technique for individual identification in an unconstrained environment. However, face recognition is a challenging task in an unconstrained environment, where the captured images are affected by illumination effect, varying poses, noise and occlusion. The main objective of this research is to improve the accuracy and processing time in extracting facial features by using the fusion of deep learning and handcrafted architecture for recognizing individuals in unconstrained conditions, thereby providing accurate information about the individuals to security systems. The proposed system consists of Multi-Block Local Binary Pattern (MB-LBP) modules for extracting the handcrafted features and Convolutional Neural Network (CNN) for extracting the high-level distinctive features. The features from both modules are fused and passed through fully connected layer with Softmax classifier to identify individuals. The results show that the enhanced algorithm based on Softmax loss function aided classifier with regularization improves the accuracy and processing time for face recognition. The proposed model improves accuracy by 94.37% against 90.01% for the state-of-the-art solution. In addition to that, it improves the processing time of 307 ms against 357 ms. The proposed system focuses on fusing hand-crafted and deep learned features to extract face features accurately and thus improving the accuracy and overall performance of the proposed system in an unconstrained environment.