Matching thermal to visible face images using hidden factor analysis in a cascaded subspace learning framework

Abstract

Matching thermal (THM) face images against visible (VIS) face images poses a significant challenge to automated face recognition systems. In this work, we introduce a Heterogeneous Face Recognition (HFR) matching framework, which uses multiple sets of subspaces generated by sampling patches from VIS and THM face images and subjecting them to a sequence of transformations. In the training phase of the proposed scheme, face images from VIS and THM are subjected to three different filters separately and then tessellated into patches. Each patch is represented by either a Pyramid Scale Invariant Feature Transform (PSIFT) or Histograms of Principal Oriented Gradients (HPOG). Then, a cascaded subspace learning process consisting of whitening transformation, factor analysis, and common discriminant analysis is used to construct multiple common subspaces between VIS and THM facial images. During the testing phase, the projected feature vectors from individual subspaces are concatenated to form a final feature vector. Nearest Neighbor (NN) classifier is used to compare feature vectors and the resulting scores corresponding to three filtered images are combined via the sum-rule. The proposed face matching algorithm is evaluated on two multispectral face datasets and is shown to achieve very promising results.