Face recognition in multi-sensor images based on a novel modular feature selection technique

Abstract

In this paper we propose a novel multi-sensor information fusion based methodology for human face recognition. Recent studies have indicated that the fusion of complementary information obtained from multiple sensors provide better recognition accuracy compared to that from individual sensors. Images captured by visible and long wave infrared sensors are used in this research. The face recognition technique presented in this paper is based on a regional feature selection strategy. Phase congruency feature maps are used instead of intensity images to make the recognition procedure invariant to illumination and contrast in an image. A novel modularization procedure which takes the importance of the relationship among the pixels within the images into consideration is also presented. Smaller sub-regions from a predefined neighborhood within the phase congruency images of the training samples are merged to obtain a large set of features. As variations in face images are confined to local regions, it is helpful to consider additional pixel dependencies across various sub-regions to improve the classification accuracy. Experiments are conducted on the Aleix Martinez and Robert Benavente (AR) face database which contain visible spectrum images, to prove the efficiency of the proposed recognition technique. The proposed modular feature selection strategy outperformed the modular Gabor features based technique in terms of recognition accuracy. The recognition technique is implemented on both thermal and visible images obtained from the Equinox face database. The thermal face recognition accuracy is also boosted by the feature selection policy. A novel decision level fusion technique which incorporates the proposed feature selection strategy is developed for improved face recognition. In terms of recognition accuracy this technique outperforms all the individual image modalities as well as the data level fused images obtained using the Discrete Wavelet Transform (DWT) based fusion technique. We also discuss the effects of image registration, which is a critical aspect for image fusion techniques. Experiments are conducted on several image sets to study the effect of discrepancies in the image registration process. We observed that the performance of our proposed technique proves to be less affected by registration errors.