Non-rigid visible and infrared face registration via regularized Gaussian fields criterion

Abstract

Registration of multi-sensor data (particularly visible color sensors and infrared sensors) is a prerequisite for multimodal image analysis such as image fusion. Typically, the relationships between image pairs are modeled by rigid or affine transformations. However, this cannot produce accurate alignments when the scenes are not planar, for example, face images. In this paper, we propose a regularized Gaussian fields criterion for non-rigid registration of visible and infrared face images. The key idea is to represent an image by its edge map and align the edge maps by a robust criterion with a non-rigid model. We model the transformation between images in a reproducing kernel Hilbert space and a sparse approximation is applied to the transformation to avoid high computational complexity. Moreover, a coarse-to-fine strategy by applying deterministic annealing is used to overcome local convergence problems. The qualitative and quantitative comparisons on two publicly available databases demonstrate that our method significantly outperforms the state-of-the-art method with an affine model. As a result, our method will be beneficial for fusion-based face recognition.