Deep independent component network for thermal-to-visible face recognition

Abstract

Cross-domain face matching between the thermal and visible spectrum is a desired technology to recognize the visible face image with probe images captured in the thermal spectrum for the night-time surveillance and security applications. However, the large modality gap between faces captured in different spectrum makes thermal-to-visible face recognition quite a challenging problem. Classical Independent Components Analysis (ICA) and its variants have been successfully used for feature learning, and the projection matrix can be used for forming the basis of images. In this paper, we present a deep joint independent component analysis network (DJICAN), a deep architecture based on multilayer independent component analysis, which learns the mutual mappings between visible and thermal face images. Joint ICA assumes the sources are the same for a visible image and its corresponding thermal image. The goal of DJICAN is to obtain the basis matrices of visible and thermal face images, which represent the individual imaging systems for the two domains. First, a forward multilayer ICA is performed. Then, we use a novel backpropagation algorithm based on a reconstruction loss function to optimize the ICA basis and the sources. Extensive experiments are performed on the ARL polarimetric thermal facial datasets that contain face images that have been taken at three different ranges and with different face expressions. The results show that the proposed method performs better than the state-of-the-art methods in terms of both synthetic image quality and thermal-to-visible face recognition accuracy.