Radar Visual Inertial Odometry and Radar Thermal Inertial Odometry: Robust Navigation even in Challenging Visual Conditions

Abstract

In this paper, we propose to fuse radar measurements with Visual Inertial Odometry (RVIO) or Thermal Inertial Odometry (RTIO). FMCW radar sensor data enables to estimate the 3D ego velocity independent of the visual conditions. Fusion with VIO or TIO heavily improves the robustness in challenging conditions such as darkness, direct sunlight or fog.Specifically, we propose RRxIO: An extension to the state of the art filter based VIO framework Robust Visual Inertial Odometry (ROVIO) to fuse radar sensor data. Due to the drift free 3D radar ego velocity estimates, scale errors are reduced and only a small number of features needs to be tracked to achieve good results. This yields faster runtimes while outperforming state of the art VIO frameworks regarding accuracy. RVIO is able to bridge phases of degraded or even no visual features resulting in a low cost system even for poor visual conditions. RTIO is robust even in environments with small temperature gradients and bridges phases of no thermal images caused by Non-Uniformity Corrections (NUCs).We evaluated our system with various experiments in different environments and visual conditions. Comparison to other state of the art dual domain approaches including radar-inertial, visual-inertial and thermal-inertial proves superior performance of our approach regarding accuracy and processing time.