A registration strategy from scale adjustment to fine for thermal-visible face images

Abstract

The analysis of multispectral facial features is rising as one of the new emerging tools in medical diagnosis. The dynamic infrared thermography and visible image data fusion of a human face can manifest several physiological behaviours used in medical diagnosis. However, an essential but challenging step is the registration of thermal and visible images. This paper presents a two-step strategy to register visible face image into the same coordinate of the thermal image. First, to adjust the large-scale difference, a pre-transformation is performed, in which the thermal and visible images are coarsely aligned by automatic selection of common keypoints in the image pair using a calibration rig. Then, the fine registration is performed by extracting phase-congruent feature points in the image pair for more precise registration. The particle swarm optimization (PSO) based fast sample consensus (PSOFSC) algorithm is also introduced for selecting correct inlier correspondence pairs. The proposed method was validated using several thermal-visible face image pairs with various resolutions from different cameras, acquired in different scenarios. The results are encouraging when compared to many state-of-the-art registration methods for thermal-visible image pairs.