PLS-VINS: Visual Inertial State Estimator With Point-Line Features Fusion and Structural Constraints

Abstract

One of the limitations of visual Simultaneous Localization and Mapping (VSLAM) of point-based is that it relies too much on scene features. When the texture information in the scene is missing or the image is blurred due to the rapid movement of the camera, the number of point features is often small, which affects the accuracy of pose estimation. Line feature information contains more geometric elements than point feature information. We propose a visual-inertial state estimator system based on point-line features and structure constraints (PLS-VINS), which combines both of points and line segments to enhance the performance of feature extraction in a wider variety of scenarios and optimizes the system states by jointly minimizing the pre-integration constraints of inertial measurement unit (IMU). Particularly in those where point features are scarce or not well-distributed in the image, PLS-VINS can extract many line features from artificial objects, which can provide more structural constraints (collinear, vertical, and parallel) for visual navigation, such as vertical and parallel constraints. Compared with the classic SLAM methods which use point-line fusion algorithm, we have adopted a new line feature error reprojection model and added structural constraints of man-made buildings in the back-end optimization to improve the robustness and accuracy in an illumination-changing environment. The proposed PLS-VINS has been extensively tested in both synthetic and real-world experiments whose results demonstrate that the proposed system outperforms the state-of-the-art methods in various scenarios. Compared with the VINS-Mono, the positioning RMSE in this study increases by 29.46% and 23.79% in rotation and translation for Dataset 1–1 with EuRoc Dataset, and by 55.64% in translation for Dataset 2–1 of actual scene with weak texture environment, respectively.