Image stitching via deep homography estimation

Abstract

Image stitching is a well-studied problem and has many applications in a variety of fields. Traditional feature based methods rely heavily on accurate localization or even distribution of hand-crafted features, and may fail for some difficult cases. Although there are robust deep learning based homography estimation or semantic alignment methods, their accuracies are not high enough for image stitching problem. In this paper, we present a deep neural network that estimates homography accurately enough for image stitching of images with small parallax. The key components of our network are feature maps with progressively increased resolution and matching cost volumes constructed in hybrid manner. Both of these designs are illustrated to be helpful for performance improvement. We also propose a new stitching oriented loss function that takes image contents into consideration. To train our network, we prepare a synthesized training dataset, the image pairs in which are more nature and similar to those of real world image stitching problem. Experimental results demonstrate that our method outperforms existing deep learning based methods and traditional feature based method in term of quantitative evaluation, visual stitching result and robustness.