Semantic-Direct Visual Odometry

Abstract

Traditional direct SLAM methods formulate the camera pose and map estimation as minimization of the photometric error, which is tackled by the Gauss-Newton algorithm or the Levenberg-Marquardt algorithm in the optimization. However, the convexity of the photometric error only holds in a small region due to the characteristics of the convexity for grayscale images. Thus, the system may be stuck in sub-optimal local minima when tracking points have large displacement. Unlike grayscale images, the semantic probabilities omit the details inside the semantic objects while mainly reforming on the boundary of semantic objects, which has better convexity for large displacement. In this letter, we propose a novel semantic-direct visual odometry (SDVO), exploiting the direct alignment of semantic probabilities. By constructing the joint error function based on grayscale images and semantic probabilities, the joint error function achieves better convexity contrary to the photometric error. Consequently, the proposed system moves towards optima with steady steps in the optimization iterations. Experimental results on the challenging real-world dataset demonstrate a significant improvement over the baseline by integrating the direct alignment of semantic probabilities.