A self-supervised monocular odometry with visual-inertial and depth representations

Abstract

Background:Self-supervised monocular visual-odometry (VO) methods have demonstrated impressive results in relative pose estimation, but they encounter challenges when visual information is unreliable. To address this, we introduce a self-supervised visual-inertial odometry (VIO) system. Methods:Our method adopts a trinocular assumption, where three consecutive frames are treated as if captured simultaneously by three cameras. This approach expands the field of view and enhances robustness in visual constraints for 3D structures. Additionally, we incorporate Inertial Measurement Unit (IMU) data into the system. This integration, achieved by an attention-based fusion strategy, allows our visual-inertial odometry (VIO) system to adaptively select the most reliable representations for accurate relative pose estimation. Results:Experiments on public and self-collected datasets show that the proposed method is state-of-the-art in visual odometry. Conclusion:The encouraging results show that it is feasible to apply this learning-based VIO system in autonomous cars and robots.