Self-Supervised deep homography estimation with invertibility constraints

Abstract

Remarkable performance of the homography estimation has been achieved by the deep CNN based approaches. These homography estimation methods, more often than not, are supervised methods and rely too much on the ground truth annotations as they aim to learn the mapping between image pairs and homography. On the other hand, the inherent invertibility of homography is helpful to avoid over-fitting and improve the performance, which however is ignored by previous homography estimation methods. In this paper, we propose a novel homography estimation approach, named “Self-Supervised Regression Network(SSR-Net)”, which relaxes the need of ground truth annotations and takes advantage of invertibility constraints. We utilize spatial pyramid pooling modules to improve the quality of extracted features in each image by exploiting context information. To employ the invertibility constraints, we adopt the matrix representation of the homography rather than the commonly used 4-point parameterization in other methods. Our proposed SSR-Net produce homography matrices and synthetic images in a cycled way. The network are trained in a self-supervised way by minimizing the combination of photometric loss and invertibility loss. Experiments on the synthetic dataset generated from MSCOCO dataset show that our proposed method outperforms several state-of-the-art approaches.