Robust IR–VIS image registration with different FOVs in railway intrusion detection

Abstract

Perimeter intrusion is a fatal threat to high-speed railway operation safety. IR (Infrared) and VIS (Visible light) image fusion can significantly enhance detection performance. Image registration is the premise of fusion, but classical registration algorithms are not suitable for railway owing to different image modalities and resolutions, large FOV (Field of View) differences, and repetitive texture features. To overcome these challenges, a coarse-to-fine area-based image registration method with feature-based constraints is proposed. An image transform model just considering translation and scaling is presented to reduce registration parameters. An edge-map-based SSIM (Structural Similarity Index) is designed to measure transform model precision of multi-modal images. Corresponding rail vanishing point pair is firstly used for coarse registration to accelerate registration, then corresponding rail line pairs are applied for fine registration to obtain accurate registration. Sufficient experiments demonstrate that the proposed method can achieve accurate and robust image registration in railway with poor lighting and large FOV differences.