Edge-guided two-stage feature matching for infrared and visible image registration in electric power scenes

Abstract

Collaborative processing of infrared and visible (IR–VS) images is essential in identifying potential equipment failures during power inspections. However, due to significant differences in spectrum, resolution, and intensity between IR–VS images, it is still challenging to obtain an accurate spatial registration for them. To address these issues, we propose the edge-guided two-stage registration strategy for various kinds of real-world applications. We first develop a heated, hollow calibration board tailored to IR–VS cameras, thus conducting offline calibration to estimate the scale and resolution relationships. The IR–VS images are pretransformed into a similar scale to improve registration accuracy. Furthermore, we propose the edge-guided feature matching network (EGFMNet) to extract feature points and build associated descriptors end-to-end. Specifically, benefiting from the excellent description ability of the network for image intensity changes, we directly adopt multimodal edge maps to learn more consistent features for accurate registration. Meanwhile, the local–global consistent features from edge maps are extracted to eliminate the nonlinear radiation distortion. By jointly learning the repeatability and reliability information, robust feature points with distinctive descriptors can be extracted and matched for homography estimation. Comparative and ablation experiments demonstrate the effectiveness and practicality of the proposed registration strategy in real-world power inspection scenarios. Extended experiments on several public datasets further demonstrate the superiority of the proposed method.