Structure Preservation and Seam Optimization for Parallax-Tolerant Image Stitching

Abstract

Parallax processing has long been a significant and challenging task in image stitching. In this paper, we study a new hybrid warping model based on multi-homography and structure preservation to achieve accurate alignment of regions at different depths while preserving local and global image structures. The homographies of different depth regions are estimated by dividing matching feature pairs into multiple layers. Then, layered warping is performed by determining the spatial relationships between the image mesh and these multi-homography, and then refining the local and global structural distortions through mesh optimization. Four constraints are considered during the local optimization process, including the local alignment error, global alignment error, and similarity error. In addition, we explore and introduce collinear structures into an objective function as a constraint for mesh optimization warping, which can preserve salient line structures while alleviating distortions in nonoverlapping areas. Furthermore, we develop an optimal seam search method based on seam error evaluation to improve the quality of the seams. Experimental results demonstrate that compared to existing methods, the proposed algorithm presents more accurate stitching results for images with large parallax and preserves salient image structures, and outperforming the existing methods both qualitatively and quantitatively.