ST-VIO: Visual Inertial Odometry Combined with Image Segmentation and Tracking

Abstract

In this article, we develop a new monocular visual-inertial odometry system ST-VIO: visual-inertial odometry combined with image segmentation and tracking. We apply the direct method to track the keyframe of images. Meanwhile, we segment and track the keyframe with Siamese Network to achieve the medium-term tracking. This method allows the system to estimate the pose by the overall feature matching of the image in the large-scale motion. In order to obtain higher precision positioning, we tightly couple the image alignment factor, the IMU factor, and the medium-term tracking factor with iSAM2 solver using factor graph optimization. Our method can reduce the cumulative error of incremental positioning system because the medium-term tracking makes the system realize the partial loop closure detection. In order to verify the stability and accuracy of the proposed method, we have tested the algorithm by the EuRoC data set and the UAV platform, respectively. The results have been also compared with those by the OKVIS and R-VIO algorithms. The experimental tests demonstrate that the proposed ST-VIO has the ability to obtain excellent results with high real-time and robust performance.