Highly robust thermal infrared and visible image registration with canny and phase congruence detection

Abstract

When a dual-optical vehicle-mounted camera is photographed, visible and thermal infrared (VI-T) images cannot be registered, and realising fine matching of the images to solve for the optimal affine parameters is thus key to this research. Visible and thermal image matching is of great interest owing to the presence of challenging scale transformations, nonlinear radial aberrations, and robustness under illumination conditions. Thus, this paper presents a novel algorithm, namely, Highly Robust Thermal Infrared and Visible Image Registration with Canny and Phase Congruence Detection (HRCP). Firstly, the image is pre-processed, and feature point detection is performed by combining adaptive thresholding GW-Canny edge detection and phase coherence (PC), which increases the edge information. Secondly, SURF and FAST feature detection at the maximum distance of PC maps can obtain stable and reliable feature points, which are robust to scale invariance and rotation invariance. Finally, the optimal affine parameter is solved using the least square method with several iterations. Compared to existing matching methods, we find that VI-T image matching is particularly sensitive to both lighting factors and scale-transformed nonlinear radial distortion and that HRCP is robust in performing matching. For the randomly selected dataset, the average image matching time is reduced by 3.265 s, the number of correct image matches increases by 44 pairs on average, the root mean square error improves to 1.830, and the average error improves to 2.815. The proposed method not only improves the inaccuracy of radiation-insensitive feature transform matching in both low and high light, but also achieves a higher accuracy and an increase in the number of feature matching points, which facilitates robust VI-T image registration.