Self-Supervised Monocular Depth Estimation With Isometric-Self-Sample-Based Learning

Abstract

Managing the dynamic regions in the photometric loss formulation has been a main issue for handling the self-supervised depth estimation problem. Most previous methods have alleviated this issue by removing the dynamic regions in the photometric loss formulation based on the masks estimated from another module, making it difficult to fully utilize the training images. In this letter, to handle this problem, we propose an isometric self-sample-based learning (ISSL) method to fully utilize the training images in a simple yet effective way. The proposed method provides additional supervision during training using self-generated images that comply with pure static scene assumption. Specifically, the isometric self-sample generator synthesizes self-samples for each training image by applying random rigid transformations on the estimated depth. Thus both the generated self-samples and the corresponding training image always follow the static scene relation. Our method can serve as a plug-and-play module for two existing models without any architectural modifications. It provides additional supervision during training phase only. Thus, there is no additional overhead on base model parameters and computation during inference phase. These properties fit well with models oriented to real-time applications. We show that plugging our ISSL module into two existing models consistently improves the performance by a large margin. In addition, it also boosts the depth accuracy over different types of scene, i.e., outdoor scenes (KITTI and Make3D) and indoor scene (NYUv2), validating its high effectiveness.