Abstract
Homography estimation refers to the problem of computing a 3 × 3 matrix which transfers image points between two images of a planar scene or two images captured from the same location. While existing algorithms exploiting hand-crafted sparse image features are well-established and efficient, recent methods based on convolutional neural networks (CNNs) achieve promising results especially for low-texture scenes. This work proposes to solve homography estimation using a hybrid framework HomoNetComb which incorporates deep learning method and energy minimization. In particular, a customized light-weight CNN named HomoNetSim is designed to calculate an initial estimation of homography, where the network is trained in an end-to-end fashion using large amount of image pairs generated from a publicly available dataset. Due to the tiny size of the employed network, the computation time of both training and inference for HomoNetSim can be reduced significantly compared with existing CNN-based homography estimation method. The initial estimate is then refined via gradient-decent algorithm by minimizing the masked pixel-level photometric discrepancy between the warped image and the destination image in a parallel fashion. Extensive experiments on the large scale synthetic dataset demonstrate that the proposed HomoNetComb improves robustness of homography estimation significantly compared with traditional methods based on sparse image features, and meanwhile HomoNetComb achieves a mean average corner error (MACE) of 0.58 pixels which outperforms previous state-of-the-art CNN-based method. Moreover, the usefulness and applicability of the proposed method is demonstrated by applying it to solve a real-world image stitching problem.
Highlights
In computer vision, the term homography is used to describe the relationship between two images of a 3D plane or two images captured by rotating the camera around its projection center
THE COMPLETE HYBRID PIPELINE The motivation of our proposed hybrid framework HomoNetComb is that convolutional neural networks (CNNs) is robust and powerful for capturing the underling feature of images and is able to generate a reasonable prediction of homography even when the scene is lack of texture, the accuracy of the solution to the homography estimation problem which is essentially a geometric problem can be further improved by utilizing explicit geometric modeling and optimization
The above experiment shows that HomoNetComb gain considerable performance improvement over PFNet-300 and HomographyNetR in the majority of cases, which suggests that the combination of our proposed CNN-based model HomoNetSim and the photometric refinement is an effective way to improve the accuracy of homography estimation while maintaining the robustness of estimator
Summary
The term homography is used to describe the relationship between two images of a 3D plane or two images captured by rotating the camera around its projection center. Most existing algorithms exploit sparse features such as points, lines, conics and curves in the image to calculate homography, among which hand-crafted corners are the most widely used ones [9]. While these methods are well-established and achieve great performance in terms of efficiency and accuracy in most. Compared with traditional methods based on sparse hand-crafted features, CNN-based homography estimation methods have better performance for handling textureless scenes, and they are easy to use since putative correspondences are established in an end-to-end fashion. The proposed HomoNetSim model and photometric refinement procedure are combined together, resulting in a hybrid homography estimation framework HomoNetComb.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
