PC-SD-VIO: A constant intensity semi-direct monocular visual-inertial odometry with online photometric calibration

Abstract

The brightness constancy assumption is the cornerstone of direct or semi-direct visual odometry (VO) and visual simultaneous localization and mapping (SLAM). However, due to the existence of automatic exposure time, nonlinear camera response function, and vignetting, this assumption is difficult to hold in practical applications. Therefore, the corresponding algorithm performs poorly on arbitrary video sequences and uncalibrated cameras. Hence, we propose a novel constant intensity semi-direct visual-inertial odometry (VIO) integrated with online photometric calibration, which combines the exactness of the feature-based method and the quickness of the direct method. We combine gain-adaptive direct image alignment and gain-adaptive Kanade–Lucas–Tomasi (KLT) optical flow tracking to complete the feature matching and use it as the input for the back-end optimization and online photometric calibration. Our photometric calibration module can complete the estimation of all photometric parameters without any prior knowledge and cooperate with the front-end to complete the real-time photometric calibration of the latest frame. Experiments on the TUM Mono VO dataset, EuRoC dataset, and real environments prove that the algorithm can reliably calibrate the photometric parameters of an arbitrary video sequence. The semi-direct VIO algorithm integrated with the photometric calibration algorithm achieves a good balance between speed, accuracy, and robustness.