Monocular Visual-inertial Odometry Research Articles

Visual–Inertial Odometry of Structured and Unstructured Lines Based on Vanishing Points in Indoor Environments

In conventional point-line visual–inertial odometry systems in indoor environments, consideration of spatial position recovery and line feature classification can improve localization accuracy. In this paper, a monocular visual–inertial odometry based on structured and unstructured line features of vanishing points is proposed. First, the degeneracy phenomenon caused by a special geometric relationship between epipoles and line features is analyzed in the process of triangulation, and a degeneracy detection strategy is designed to determine the location of the epipoles. Then, considering that the vanishing point and the epipole coincide at infinity, the vanishing point feature is introduced to solve the degeneracy and direction vector optimization problem of line features. Finally, threshold constraints are used to categorize straight lines into structural and non-structural features under the Manhattan world assumption, and the vanishing point measurement model is added to the sliding window for joint optimization. Comparative tests on the EuRoC and TUM-VI public datasets validated the effectiveness of the proposed method.

PL-EVIO: Robust Monocular Event-Based Visual Inertial Odometry With Point and Line Features

PL-EVIO: Robust Monocular Event-Based Visual Inertial Odometry With Point and Line Features

Tightly coupled integration of monocular visual-inertial odometry and UC-PPP based on factor graph optimization in difficult urban environments

Tightly coupled integration of monocular visual-inertial odometry and UC-PPP based on factor graph optimization in difficult urban environments

Fast and Robust Semidirect Monocular Visual-Inertial Odometry for UAV

Fast and Robust Semidirect Monocular Visual-Inertial Odometry for UAV

Robust Monocular Visual Inertial Odometry in γ Radioactive Environments using Edge-based Point Features

Abstract In the γ radioactive environment, high-energy photons induce degradation of the image sensor, which effects feature detection and tracking in the Visual Inertial Odometry (VIO) algorithm and deteriorates its localization performance. To address this issue, in this work, we propose a monocular VIO method using edge-based point features. To mitigate the effects of radiation noise, firstly, in the image preprocessing module, the median filter is used for real-time image denoising. Secondly, in the data association module, both Shi-Tomasi and edge-based point features are detected. The edge-based point feature is the endpoint or corner point in the salient edge map, which is more robust to radiation noise. Then, the bi-directional motion parallaxes and the RANdom SAmple Consensus (RANSAC) method are exploited to reject outliers. Finally, the point features measurements and Inertial Measurement Unit (IMU) pre-integration measurements are added into a tightly-coupled sliding window optimization VIO framework for localization estimation. The proposed method is verified by synthetic and real γ radioactive environment datasets. The experimental results show that the proposed method achieves more accurate and robust localization than the state-of-the-art VIO approaches in the γ radioactive environments.

PVI-DSO: Leveraging Planar Regularities for Direct Sparse Visual-Inertial Odometry

The monocular visual-inertial odometry (VIO) based on the direct method can leverage all available pixels in the image to simultaneously estimate the camera motion and reconstruct the denser map of the scene in real time. However, the direct method is sensitive to photometric changes, which can be compensated by introducing geometric information in the environment. In this paper, we propose a monocular direct sparse visual-inertial odometry, which exploits the planar regularities (PVI-DSO). Our system detects the planar regularities from the 3D mesh built on the estimated map points. To improve the pose estimation accuracy with the geometric information, a tightly coupled coplanar constraint expression is used to express photometric error in the direct method. Additionally, to improve the optimization efficiency, we elaborately derive the analytical Jacobian of the linearization form for the coplanar constraint. Finally, the inertial measurement error, coplanar point photometric error, non-coplanar photometric error, and prior error are added into the optimizer, which simultaneously improves the pose estimation accuracy and mesh itself. We verified the performance of the whole system on simulation and real-world datasets. Extensive experiments have demonstrated that our system outperforms the state-of-the-art counterparts.

PLI-VIO: Real-time Monocular Visual-inertial Odometry Using Point and Line Interrelated Features

PLI-VIO: Real-time Monocular Visual-inertial Odometry Using Point and Line Interrelated Features

Improved monocular visual-inertial odometry with point and line features using adaptive line feature extraction

Improved monocular visual-inertial odometry with point and line features using adaptive line feature extraction

Improving the Computational Efficiency of ROVIO

ROVIO is one of the state-of-the-art monocular visual inertial odometry algorithms. It uses an Iterative Extended Kalman Filter (IEKF) to align visual features and update the vehicle state simultaneously by including the feature locations in the state vector of the IEKF. This algorithm is single-core intensive, which allows the other cores to be used for other algorithms, such as object detection and path optimization. However, the computational cost of the algorithm grows rapidly with the maximum number of features to track. Each feature adds three new states (a 2D bearing vector and inverse depth), leading to bigger matrix multiplications that are computationally expensive. The main computational load of ROVIO is the iterative update step of the IEKF. In this work, we reduce the average computational cost of ROVIO by [Formula: see text] on an NVIDIA Jetson TX2, without affecting the accuracy of the algorithm. This computational gain is mainly achieved by exploiting the sparse matrices in ROVIO. Furthermore, we reduce the computational peaks by pre-selecting new features based on their already calculated FAST score. The combination of both modifications allows us to run ROVIO on the computationally restricted Raspberry Pi Zero 2[Formula: see text]W.

RD-VIO: Relative-depth-aided visual-inertial odometry for autonomous underwater vehicles

Because of the autonomous underwater vehicles' (AUVs') slow motion and insufficient excitations, monocular visual-inertial odometry (VIO) systems for AUVs still have large-scale errors and a lengthy initialization period. In this paper, we present a relative-depth-aided monocular visual-inertial odometry (RD-VIO) system based on two innovative methods: Inertial Measurement Unit (IMU)-depth joint initialization and relative depth tightly coupled with visual-inertial. The RD-VIO performs reprojection bundle adjustment with dynamic weight factors for visual residuals. Depth measurements acquired by depth gauge or pressure sensor are compensated with reliable inter-image IMU pre-integration to account for different sensor frequencies. Also, we adopt an IMU-depth joint initialization using relative depth measurements and IMU excitations to estimate the scale and gravity direction together, which captures the complete AUV motion changes, improving scale estimation and decreasing initialization time. To reduce scale drift and improve global accuracy and consistency, a tight coupling of relative depth and visual-inertial in the RD-VIO is employed to optimize the AUV pose from depth measurements. Using two open-source datasets and our experiments from Sanya, we evaluated the performance of RD-VIO. The evaluation results show that the proposed system can achieve a more accurate trajectory than the state-of-the-art algorithms with less system initialization time and scale error.

Monocular visual-inertial odometry leveraging point-line features with structural constraints

Monocular visual-inertial odometry leveraging point-line features with structural constraints

Editorial Issue 34.2

Editorial Issue 34.2

UAV Navigation With Monocular Visual Inertial Odometry Under GNSS-Denied Environment

UAV Navigation With Monocular Visual Inertial Odometry Under GNSS-Denied Environment

Robust Tightly-Coupled Visual-Inertial Odometry with Pre-built Maps in High Latency Situations.

In this paper, we present a novel monocular visual-inertial odometry system with pre-built maps deployed on the remote server, which can robustly run in real-time on a mobile device even in high latency situations. By tightly coupling VIO with geometric priors from pre-built maps, our system can tolerate the high latency and low frequency of global localization service, which is especially suitable for practical applications when the localization service is deployed on the remote server. Firstly, sparse point clouds are obtained from the dense mesh by the ray casting method according to the localization results. The dense mesh can be reconstructed from the point clouds generated by Structure-from-Motion. We directly use the sparse point clouds in feature tracking and state update to suppress drift. In the process of feature tracking, the high local accuracy of VIO is fully utilized to effectively remove outliers and make our system robust. The experiments on EurocMav datasets and simulation datasets show that compared with state-of-the-art methods, our method can achieve better results in terms of both precision and robustness. The effectiveness of the proposed method is further demonstrated through a real-time AR demo on a mobile phone with the aid of visual localization on the remote server.

DM-VIO: Delayed Marginalization Visual-Inertial Odometry

We present DM-VIO, a monocular visual-inertial odometry system based on two novel techniques called delayed marginalization and pose graph bundle adjustment. DM-VIO performs photometric bundle adjustment with a dynamic weight for visual residuals. We adopt marginalization, which is a popular strategy to keep the update time constrained, but it cannot easily be reversed, and linearization points of connected variables have to be fixed. To overcome this we propose delayed marginalization: The idea is to maintain a second factor graph, where marginalization is delayed. This allows us to later readvance this delayed graph, yielding an updated marginalization prior with new and consistent linearization points. In addition, delayed marginalization enables us to inject IMU information into already marginalized states. This is the foundation of the proposed pose graph bundle adjustment, which we use for IMU initialization. In contrast to prior works on IMU initialization, it is able to capture the full photometric uncertainty, improving the scale estimation. In order to cope with initially unobservable scale, we continue to optimize scale and gravity direction in the main system after IMU initialization is complete. We evaluate our system on the EuRoC, TUM-VI, and 4Seasons datasets, which comprise flying drone, large-scale handheld, and automotive scenarios. Thanks to the proposed IMU initialization, our system exceeds the state of the art in visual-inertial odometry, even outperforming stereo-inertial methods while using only a single camera and IMU. The code will be published at http://vision.in.tum.de/dm-vio

SelfVIO: Self-supervised deep monocular Visual–Inertial Odometry and depth estimation

In the last decade, numerous supervised deep learning approaches have been proposed for visual–inertial odometry (VIO) and depth map estimation, which require large amounts of labelled data. To overcome the data limitation, self-supervised learning has emerged as a promising alternative that exploits constraints such as geometric and photometric consistency in the scene. In this study, we present a novel self-supervised deep learning-based VIO and depth map recovery approach (SelfVIO) using adversarial training and self-adaptive visual–inertial sensor fusion. SelfVIO learns the joint estimation of 6 degrees-of-freedom (6-DoF) ego-motion and a depth map of the scene from unlabelled monocular RGB image sequences and inertial measurement unit (IMU) readings. The proposed approach is able to perform VIO without requiring IMU intrinsic parameters and/or extrinsic calibration between IMU and the camera. We provide comprehensive quantitative and qualitative evaluations of the proposed framework and compare its performance with state-of-the-art VIO, VO, and visual simultaneous localization and mapping (VSLAM) approaches on the KITTI, EuRoC and Cityscapes datasets. Detailed comparisons prove that SelfVIO outperforms state-of-the-art VIO approaches in terms of pose estimation and depth recovery, making it a promising approach among existing methods in the literature.

A New Visual Inertial Simultaneous Localization and Mapping (SLAM) Algorithm Based on Point and Line Features

In view of traditional point-line feature visual inertial simultaneous localization and mapping (SLAM) system, which has weak performance in accuracy so that it cannot be processed in real time under the condition of weak indoor texture and light and shade change, this paper proposes an inertial SLAM method based on point-line vision for indoor weak texture and illumination. Firstly, based on Bilateral Filtering, we apply the Speeded Up Robust Features (SURF) point feature extraction and Fast Nearest neighbor (FLANN) algorithms to improve the robustness of point feature extraction result. Secondly, we establish a minimum density threshold and length suppression parameter selection strategy of line feature, and take the geometric constraint line feature matching into consideration to improve the efficiency of processing line feature. And the parameters and biases of visual inertia are initialized based on maximum posterior estimation method. Finally, the simulation experiments are compared with the traditional tightly-coupled monocular visual–inertial odometry using point and line features (PL-VIO) algorithm. The simulation results demonstrate that the proposed an inertial SLAM method based on point-line vision for indoor weak texture and illumination can be effectively operated in real time, and its positioning accuracy is 22% higher on average and 40% higher in the scenario that illumination changes and blurred image.

Monocular Visual-Inertial Odometry for Agricultural Environments

The accuracy of autonomous robot localization using a monocular visual-inertial odometry system (VIO) is significantly reduced in an agricultural environment compared to an urban and indoor environment due to the unstructured scenes with unstable features, variation of light conditions, and rugged terrain. To address those challenges, we propose a monocular VIO system with modifications to the existing state-of-the-art monocular VIO system VINS-mono. Three modules of VINS-mono have improved in this work: vision processing front end, pose optimization, failure detection and recovery. In the vision processing front-end module, we proposed a keyframe selection algorithm base on vertical movement smoothness verification to prevent the rapid loss of feature tracks. In the pose optimization module, some feature points are removed using a depth-limited approach to improve the efficiency and accuracy of trajectory tracking. In the failure detection and recovery module, we propose a failure recovery method using the old trajectory poses, which can recover the trajectory tracking process interrupted by the failure and alleviate the orientation drift. Experiments on the Rosario dataset have shown that our system outperforms VINS-mono, and the absolute trajectory error in the agricultural environment can be at least reduced by 69% compared with the VINS-mono, which can effectively improve the localization accuracy of agriculture robots in the agricultural environment.

Visual-Pressure Fusion for Underwater Robot Localization With Online Initialization

The motion of underwater robot is usually slow, which leads to large scale error easily introduced in the positioning method based on monocular Visual Inertial Odometry (VIO). To solve the above problem, we present a positioning method based on visual-pressure fusion in this letter. Firstly, it is proved by observability analysis that when the robot's motion is not on the same horizontal plane, the system scale can be uniquely determined by monocular image and depth information; secondly, a novel visual-pressure initialization method is proposed, which can calculate the system scale and initial attitude with respect to world frame online; finally, we integrate our method into current advanced monocular Simultaneous Localization And Mapping (SLAM) system, and the effectiveness and superiority of the method are verified by experiments on simulation datasets and real datasets.

Semi-Direct Monocular Visual-Inertial Odometry Using Point and Line Features for IoV

The precise measuring of vehicle location has been a critical task in enhancing the autonomous driving in terms of intelligent decision making and safe transportation. Internet of Vehicles ( IoV ) is an important infrastructure in support of autonomous driving, allowing real-time road information exchanging and sharing for localizing vehicles. Global positioning System ( GPS ) is widely used in the traditional IoV system. GPS is unable to meet the key application requirements of autonomous driving due to meter level error and signal deterioration. In this article, we propose a novel solution, named Semi-Direct Monocular Visual-Inertial Odometry using Point and Line Features ( SDMPL-VIO ) for precise vehicle localization. Our SDMPL-VIO model takes advantage of a low-cost Inertial Measurement Unit ( IMU ) and monocular camera, using them as the sensor to acquire the surrounding environmental information. Visual-Inertial Odometry ( VIO ), taking into account both point and line features, is proposed, which is able to deal with both weak texture and dynamic environment. We use a semi-direct method to deal with keyframes and non-keyframes, respectively. Dual sliding window mechanisms can effectively fuse point-line and IMU information. To evaluate our SDMPL-VIO system model, we conduct extensive experiments on both an indoor dataset (i.e., EuRoC) and an outdoor dataset (i.e., KITTI) from the real-world applications, respectively. The experimental results show that the accuracy of SDMPL-VIO proposed by us is better than the mainstream VIO system at present. Especially in the weak texture of the datasets, fast-moving datasets, and other challenging datasets, SDMPL-VIO has a relatively high robustness.