Multimodel fore-/background alignment for seam-based parallax-tolerant image stitching

Abstract

Image stitching with large parallax is a challenging computer vision problem. Although existing seam-based approaches were proposed to achieve pleasing results, issues like object dislocation, disappearance, and duplication can still occur. In this paper, to alleviate these problems, we propose a novel seam-based parallax-tolerant image stitching method, which relies on accurately aligning background and foreground regions using multiple warping models. To estimate various spatially smooth models based on feature correspondences from depth-varying objects, we introduce an iterative algorithm that selects inliers and solves the mesh warping model by assigning weights to data. Additionally, we construct matching confidences of foreground pixels based on selecting and grouping unaligned feature pairs, thus penalizing the duplication of seam cuts. To further improve alignment, we refine the models by minimizing pixel-level errors. We then choose the best seam among multiple candidate alignment and seam finding solutions. Finally, we re-estimate the warping model by sampling and weighting points near the seam to achieve a natural-looking stitching result. Experimental results on real-world images demonstrate the effectiveness and superiority of our proposed method over other state-of-the-arts.