Image stitching by disparity-guided multi-plane alignment

Abstract

Image stitching aims to warp and align two or more images with overlapping areas to generate a panorama with a larger field of view. Due to the wide baseline or the abrupt depth changes caused by the foreground objects, the phenomenon that adjacent pixels or regions in one image are not adjacent in another image happens. It is difficult to avoid severe parallax artifacts and get good alignment results when stitching such images. In this paper, focusing on the images with large parallax, we design a multi-plane alignment algorithm guided by the disparity map. The concept of average tolerable parallax is proposed to help distinguish one background plane and multiple foreground objects from the image, which are robust to small parallax. In order to obtain the reliable disparity map of the image pairs got from any camera with unknown camera parameters, we propose an optimal homography estimation method based on the relative projection biases. This helps to satisfy the common epipolar line constraint when applying stereo matching modules. Experimental results demonstrate that our algorithm provides accurate stitching results on images with large parallax, and outperforms other existing state-of-the-art methods both qualitatively and quantitatively.