As-Aligned-As-Possible Image Stitching Based on Deviation-Corrected Warping With Global Similarity Constraints

Jialiang Li,Haiying Xia,Shuxiang Song,Pinqun Jiang,Min Jiang

doi:10.1109/access.2019.2944852

Abstract

Handling local misalignment caused by the local warp remains a common and challenging task for image stitching. Moreover, the stitched image is prone to appearing ghosting due to the variations of the image viewpoint between images. To solve the problem of local misalignment, we propose a projection deviation-corrected local warping method with a global similarity constraint for image stitching. Recent warps prove that the warp of the local mesh guide image effectively improves the accuracy of image alignment. Geometric projection deviation is well used to accurately correct pixel offsets in image warping. To correct pixel offsets, we first remove the outliers from matching points by using the normal distribution model. The retained matches are more precise and can improve the accuracy of image alignment. Next, we use the local warping model combining local homography and global similarity for image warping. To further address the misalignment problem caused by local warping, we describe the local projection deviation of the local warping model by adopting a three-dimensional mesh interpolation model. Finally, the warped images are blended by a linear smoothing model. Experimental results show that our method outperforms the state-of-the-arts in alignment accuracy, and also provides better visual effects on challenging images.

Highlights

Image stitching, a method of combining multiple images into a wide-angle panorama containing information of original images [1], is the most widely used algorithms in computer vision, such as panorama, video surveillance [2], and virtual reality [3]
Lin et al [11] proposed the adaptive as-natural-as-possible (ANAP) warp based on linearity homography and combined with global similar transformation to solve the unnatural rotation of shapepreserving half-projective (SPHP) by minimizing the rotation angle
Li et al [4] proposed a parallax-tolerant image stitching method based on global homography, which uses the thin-plate spline (TPS) function to calculate the global deformation vector to alleviate the misalignment of global homography in the overlapping region

Summary

INTRODUCTION

A method of combining multiple images into a wide-angle panorama containing information of original images [1], is the most widely used algorithms in computer vision, such as panorama, video surveillance [2], and virtual reality [3]. To address the inadequacy of global warps, several local warp models have been proposed, such as the smoothly varying affine (SVA) [8] and the as-projective-as-possible (APAP) [9] These methods calculate multiple local warps to achieve better alignment accuracy, but they only work for images with moderate parallax. A series of direct warping approaches are proposed to obtain a natural stitching image [13]–[16] These methods directly apply geometric constraints to guide mesh deformation in the localized image and can be combined with seam cutting to cope with large parallax images. Little research has focused on the removal of outliers in matching points after image alignment [4] To deal with these problems discussed above, an image stitching method combining several techniques is proposed in this paper.

RELATED WORK

LOCAL WARPING

LOCAL ALIGNMENT

EXPERIMENTAL RESULTS AND ANALYSIS

DATASETS AND EXPERIMENTAL SETUP

CONCLUSION