Multi-modal Image Stitching with Nonlinear Optimization

Abstract

Despite significant advances in recent years, the problem of image stitching still lacks a robust solution. Most of the feature based image stitching algorithms perform image alignment based on either homography-based transformation or content-preserving warping. Pairwise homography-based approach miserably fails to handle parallax whereas content-preserving warping approach does not preserve the structural property of the images. In this paper, we propose a nonlinear optimization to find out the global homographies using pairwise homography estimates and point correspondences. We further compute local warping based alignment to mitigate the aberration caused by noises in the global homography estimation. To this end, we incorporate geometric as well as photometric constraints to design our cost function which is minimized to obtain better alignment after the global registration, thus producing accurate image stitching. Experimental results on various open datasets demonstrate that our proposed method outperforms state-of-the-art image stitching algorithms.