Inertial Measurement Unit Preintegration Research Articles

A Spatiotemporal Calibration Algorithm for IMU-LiDAR Navigation System Based on Similarity of Motion Trajectories.

The fusion of light detection and ranging (LiDAR) and inertial measurement unit (IMU) sensing information can effectively improve the environment modeling and localization accuracy of navigation systems. To realize the spatiotemporal unification of data collected by the IMU and the LiDAR, a two-step spatiotemporal calibration method combining coarse and fine is proposed. The method mainly includes two aspects: (1) Modeling continuous-time trajectories of IMU attitude motion using B-spline basis functions; the motion of the LiDAR is estimated by using the normal distributions transform (NDT) point cloud registration algorithm, taking the Hausdorff distance between the local trajectories as the cost function and combining it with the hand–eye calibration method to solve the initial value of the spatiotemporal relationship between the two sensors’ coordinate systems, and then using the measurement data of the IMU to correct the LiDAR distortion. (2) According to the IMU preintegration, and the point, line, and plane features of the lidar point cloud, the corresponding nonlinear optimization objective function is constructed. Combined with the corrected LiDAR data and the initial value of the spatiotemporal calibration of the coordinate systems, the target is optimized under the nonlinear graph optimization framework. The rationality, accuracy, and robustness of the proposed algorithm are verified by simulation analysis and actual test experiments. The results show that the accuracy of the proposed algorithm in the spatial coordinate system relationship calibration was better than 0.08° (3δ) and 5 mm (3δ), respectively, and the time deviation calibration accuracy was better than 0.1 ms and had strong environmental adaptability. This can meet the high-precision calibration requirements of multisensor spatiotemporal parameters of field robot navigation systems.

A Coupled Visual and Inertial Measurement Units Method for Locating and Mapping in Coal Mine Tunnel.

Mobile robots moving fast or in scenes with poor lighting conditions often cause the loss of visual feature tracking. In coal mine tunnels, the ground is often bumpy and the lighting is uneven. During the movement of the mobile robot in this scene, there will be violent bumps. The localization technology through visual features is greatly affected by the illumination and the speed of the camera movement. To solve the localization and mapping problem in an environment similar to underground coal mine tunnels, we improve a localization and mapping algorithm based on a monocular camera and an Inertial Measurement Unit (IMU). A feature-matching method that combines point and line features is designed to improve the robustness of the algorithm in the presence of degraded scene structure and insufficient illumination. The tightly coupled method is used to establish visual feature constraints and IMU pre-integration constraints. A keyframe nonlinear optimization algorithm based on sliding windows is used to accomplish state estimation. Extensive simulations and practical environment verification show that the improved simultaneous localization and mapping (SLAM) system with a monocular camera and IMU fusion can achieve accurate autonomous localization and map construction in scenes with insufficient light such as coal mine tunnels.

Semi-Direct Point-Line Visual Inertial Odometry for MAVs

Traditional Micro-Aerial Vehicles (MAVs) are usually equipped with a low-cost Inertial Measurement Unit (IMU) and monocular cameras, how to achieve high precision and high reliability navigation under the framework of low computational complexity is the main problem for MAVs. To this end, a novel semi-direct point-line visual inertial odometry (SDPL-VIO) has been proposed for MAVs. In the front-end, point and line features are introduced to enhance image constraints and increase environmental adaptability. At the same time, the semi-direct method combined with IMU pre-integration is used to complete motion estimation. This hybrid strategy combines the accuracy and loop closure detection performance of the feature-based method with the rapidity of the direct method, and tracks keyframes and non-keyframes, respectively. In the back-end, the sliding window mechanism is adopted to limit the computation, while the improved marginalization method is used to decompose the high-dimensional matrix corresponding to the cost function to reduce the computational complexity in the optimization process. The comparison results in the EuRoC datasets demonstrate that SDPL-VIO performs better than the other state-of-the-art visual inertial odometry (VIO) methods, especially in terms of accuracy and real-time performance.

Impact of the Earth Rotation Compensation on MEMS-IMU Preintegration of Factor Graph Optimization

In filtering-based global navigation satellite system (GNSS) / inertial navigation system (INS) integrated navigation systems, the precise INS mechanization has fully considered the impact of the Earth rotation, Coriolis acceleration, etc. However, most inertial measurement unit (IMU) preintegration models in factor graph optimization (FGO)-based integrated navigation systems are rough and ignore these factors. We propose an FGO-based GNSS/INS integrated navigation system to analyze and evaluate the impact of the Earth rotation on MEMS-IMU preintegration. The proposed FGO-based integration is based on a refined IMU preintegration, in which the Earth rotation is compensated. The proposed GNSS/INS integration is a sliding-window optimizer in which the GNSS positioning and the IMU preintegration are tightly fused within the FGO framework. The simulated GNSS outage in field tests, an effective method, is adopted to evaluate the IMU preintegration quantitatively. The results demonstrate that the refined IMU preintegration can achieve the same accuracy as the precise INS mechanization. In contrast, the rough IMU preintegration and INS mechanization without compensating for the Earth rotation yields notable accuracy degradation. When the GNSS-outage time is 60 seconds, the degradation can be 200% for the industrial-grade MEMS module and more than 10% for the consumer-grade MEMS chip. Besides, the degradation can be more significant if the GNSS-outage time is longer. The proposed FGO-based GNSS/INS integration ( <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/i2Nav-WHU/OB_GINS</uri> ) and the employed datasets ( <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/i2Nav-WHU/awesome-gins-datasets</uri> ) are open-sourced on GitHub.

LIDAR-Inertial Real-Time State Estimator with Rod-Shaped and Planar Feature

State estimation and mapping based on Light Detection and Ranging (LIDAR) are important for autonomous systems. Point cloud registration is a crucial module affecting the accuracy and real-time performance of LIDAR simultaneous localization and mapping (SLAM). In this paper, a novel point cloud feature selection for LIDAR-inertial tightly coupled systems is proposed. In the front-end, a point cloud registration is carried out after marking rod-shaped and planar feature information which is different from the existing LIDAR and inertial measurement unit (IMU) integration scheme. This preprocessing method subsequently reduces the outliers. IMU pre-integration outputs high-frequency result and is used to provide the initial value for LIDAR solution. In the scan-to-map module, a computationally efficient graph optimization framework is applied. Moreover, the LIDAR odometry further constrains the IMU states. In the back-end, the optimization based on sliding-window incorporates the LIDAR-inertial measurement and loop closure global constraints to reduce the cumulative error. Combining the front-end and back-end, we propose the low drift and high real-time LIDAR-inertial positioning system. Furthermore, we conducted an exhaustive comparison in open data sequences and real-word experiments. The proposed system outperforms much higher positioning accuracy than the state-of-the-art methods in various scenarios. Compared with the LIO-SAM, the absolute trajectory error (ATE) average RMSE (Root Mean Square Error) in this study increases by 64.45% in M2DGR street dataset (street_01, 04, 07, 10) and 24.85% in our actual scene datasets. In the most time-consuming mapping module of each system, our system runtime can also be significantly reduced due to the front-end preprocessing and back-end graph model.

An Enhanced Hybrid Visual-Inertial Odometry System for Indoor Mobile Robot.

As mobile robots are being widely used, accurate localization of the robot counts for the system. Compared with position systems with a single sensor, multi-sensor fusion systems provide better performance and increase the accuracy and robustness. At present, camera and IMU (Inertial Measurement Unit) fusion positioning is extensively studied and many representative Visual–Inertial Odometry (VIO) systems have been produced. Multi-State Constraint Kalman Filter (MSCKF), one of the tightly coupled filtering methods, is characterized by high accuracy and low computational load among typical VIO methods. In the general framework, IMU information is not used after predicting the state and covariance propagation. In this article, we proposed a framework which introduce IMU pre-integration result into MSCKF framework as observation information to improve the system positioning accuracy. Additionally, the system uses the Helmert variance component estimation (HVCE) method to adjust the weight between feature points and pre-integration to further improve the positioning accuracy. Similarly, this article uses the wheel odometer information of the mobile robot to perform zero speed detection, zero-speed update, and pre-integration update to enhance the positioning accuracy of the system. Finally, after experiments carried out in Gazebo simulation environment, public dataset and real scenarios, it is proved that the proposed algorithm has better accuracy results while ensuring real-time performance than existing mainstream algorithms.

VIRAL-Fusion: A Visual-Inertial-Ranging-Lidar Sensor Fusion Approach

In recent years, onboard self-localization (OSL) methods based on cameras or lidar have achieved many significant progresses. However, some issues such as estimation drift and robustness in low-texture environment still remain inherent challenges for OSL methods. On the other hand, infrastructure-based methods can generally overcome these issues, but at the expense of some installation cost. This poses an interesting problem of how to effectively combine these methods, so as to achieve localization with long-term consistency as well as flexibility compared to any single method. To this end, we propose a comprehensive optimization-based estimator for the 15-D state of an unmanned aerial vehicle (UAV), fusing data from an extensive set of sensors: inertial measurement unit (IMU), ultrawideband (UWB) ranging sensors, and multiple onboard visual-inertial and lidar odometry subsystems. In essence, a sliding window is used to formulate a sequence of robot poses, where relative rotational and translational constraints between these poses are observed in the IMU preintegration and OSL observations, while orientation and position are coupled in the <i>body-offset</i> UWB range observations. An optimization-based approach is developed to estimate the trajectory of the robot in this sliding window. We evaluate the performance of the proposed scheme in multiple scenarios, including experiments on public datasets, high-fidelity graphical-physical simulation, and field-collected data from UAV flight tests. The result demonstrates that our integrated localization method can effectively resolve the drift issue, while incurring minimal installation requirements.

Associating Uncertainty to Extended Poses for on Lie Group IMU Preintegration With Rotating Earth

The recently introduced matrix group <inline-formula><tex-math notation="LaTeX">$\mathbf {SE_2(3)}$</tex-math></inline-formula> provides a 5<inline-formula><tex-math notation="LaTeX">$\mathbf {\times }$</tex-math></inline-formula>5 matrix representation for the orientation, velocity, and position of an object in the 3-D space, a triplet we call &#x201C;extended pose.&#x201D; In this article, we build on this group to develop a theory to associate uncertainty with extended poses represented by 5<inline-formula><tex-math notation="LaTeX">$\mathbf {\times }$</tex-math></inline-formula>5 matrices. Our approach is particularly suited to describe how uncertainty propagates when the extended pose represents the state of an inertial measurement unit (IMU). In particular, it allows revisiting the theory of IMU preintegration on manifold and reaching a further theoretic level in this field. Exact preintegration formulas that account for rotating earth, that is, centrifugal force and Coriolis force, are derived as a byproduct, and the factors are shown to be more accurate. The approach is validated through extensive simulations and applied to sensor fusion where a loosely coupled fixed-lag smoother fuses IMU and LiDAR on 1-h-long experiments using our experimental car. It shows how handling rotating earth may be beneficial for long-term navigation within incremental smoothing algorithms.

A novel Stereo-IMU system based on accurate pose parameterization of two-wheeled inverted pendulum (TWIP) robot in localization and mapping

PurposeThis paper aims to propose a method to solve the problem of localization and mapping of a two-wheeled inverted pendulum (TWIP) robot on the ground using the Stereo–inertial measurement unit (IMU) system. This method reparametrizes the pose according to the motion characteristics of TWIP and considers the impact of uneven ground on vision and IMU, which is more adaptable to the real environment.Design/methodology/approachWhen TWIP moves, it is constrained by the ground and swings back and forth to maintain balance. Therefore, the authors parameterize the robot pose as SE(2) pose plus pitch according to the motion characteristics of TWIP. However, the authors do not omit disturbances in other directions but perform error modeling, which is integrated into the visual constraints and IMU pre-integration constraints as an error term. Finally, the authors analyze the influence of the error term on the vision and IMU constraints during the optimization process. Compared to traditional algorithms, the algorithm is simpler and better adapt to the real environment.FindingsThe results of indoor and outdoor experiments show that, for the TWIP robot, the method has better positioning accuracy and robustness compared with the state-of-the-art.Originality/valueThe algorithm in this paper is proposed for the localization and mapping of a TWIP robot. Different from the traditional positioning method on SE(3), this paper parameterizes the robot pose as SE(2) pose plus pitch according to the motion of TWIP and the motion disturbances in other directions are integrated into visual constraints and IMU pre-integration constraints as error terms, which simplifies the optimization parameters, better adapts to the real environment and improves the accuracy of positioning.

An improved SLAM based on RK-VIF: Vision and inertial information fusion via Runge-Kutta method

Simultaneous Localization and Mapping (SLAM) is the foundation of autonomous navigation for unmanned systems. The existing SLAM solutions are mainly divided into the visual SLAM(vSLAM) equipped with camera and the lidar SLAM equipped with lidar. However, pure visual SLAM have shortcomings such as low positioning accuracy, the paper proposes a visual-inertial information fusion SLAM based on Runge-Kutta improved pre-integration. First, the Inertial Measurement Unit (IMU) information between two adjacent keyframes is pre-integrated at the front-end to provide IMU constraints for visual-inertial information fusion. In particular, to improve the accuracy in pre-integration, the paper uses the Runge-Kutta algorithm instead of Euler integral to calculate the pre-integration value at the next moment. Then, the IMU pre-integration value is used as the initial value of the system state at the current frame time. We combine the visual reprojection error and imu pre-integration error to optimize the state variables such as speed and pose, and restore map points' three-dimensional coordinates. Finally, we set a sliding window to optimize map points' coordinates and state variables. The experimental part is divided into dataset experiment and complex indoor-environment experiment. The results show that compared with pure visual SLAM and the existing visual-inertial fusion SLAM, our method has higher positioning accuracy.

A dual-mode automatic switching feature points matching algorithm fusing IMU data

Feature point extraction and matching work together and play a crucial role in many machine vision tasks, such as visual simultaneous localization and mapping (SLAM) which is the basis of robotics. However, feature matching has not been fully studied as much as feature extraction. The most widely used feature matching method is still the classical brute-force matching. In this paper, by fusing the data of inertial measurement unit (IMU), a novel feature points matching algorithm with online learning fault detection, where the global matcher and local matcher operate alternately and switch automatically, is proposed to improve matching accuracy. Taking the uncertainties of the initial states of camera and IMU into account in the initialization phase, the system starts with the global mode to accomplish the matching task and initialize the state variables which are used to align the data of two sensors. Then, once completing the initialization, the algorithm will automatically switch to our new-designed local matcher with feature compensation, which bounds the search space and improves the matching accuracy via performing on the support region according to the results of IMU pre-integration rather than the whole image. In addition, for the purpose of effectively dealing with the misestimation of support region induced by IMU failures, an online learning fault detector based on support vector machines is newly developed, which can trigger the action of mode switcher and runs simultaneously in another thread. Furthermore, a timer is attached to the fault detector in order to activate the detection periodically, which can further reduces the cost of computing resources. Finally, the performance of the present algorithm is validated on public datasets.

Optimal Time Difference-Based TDCP-GPS/IMU Navigation Using Graph Optimization

Global positioning system (GPS) has been widely used to estimate the absolute position of outdoor vehicles. Global positioning system and inertial measurement unit (IMU) can be integrated to deliver an enhanced effect over the individual system. GPS/IMU navigation solution that utilizes time-differenced carrier phase (TDCP) as observation can achieve high-precision positioning. The traditional TDCP-GPS/IMU navigation methods are compromised by the loss of constraints, especially when cycle slip occurs. This article proposes an optimal time difference-based TDCP-GPS/IMU navigation using graph optimization. Different from current methods, an optimal time difference strategy is designed to formulate integer-free constraints that span multiple epochs in a time window. Each satellite can be fully used to restrain the accumulation errors even though cycle slip happens. Then, the integer-free constraints are fused with relative constraints from IMU preintegration and TDCP between two successive epochs using graph optimization. Field tests are carried out to validate the performance of the proposed method. The results show that the proposed method can make the most of window length constraints to improve the positioning performance compared with traditional methods.

Mobile Robot Localization Based on Vision and Multisensor

To deal with the low accuracy of positioning for mobile robots when only using visual sensors and an IMU, a method based on tight coupling and nonlinear optimization is proposed to obtain a high-precision visual positioning scheme by combining measured value of the preintegrated inertial measurement unit (IMU) and values of the odometer and characteristic observations. First, the preprocessing part of the observation data includes tracking of the image data and the odometer data, and preintegration of IMU data. Second, the initialization part of the above three sensors includes IMU preintegration, odometer preintegration, and gyroscope bias calculation. It also includes the alignment of speed, gravity, and scale. Finally, a local BA (bundle adjustment) joint optimization and global graph optimization are established, so as to obtain more accurate positioning results.

Closed-form preintegration methods for graph-based visual–inertial navigation

In this paper, we propose a new analytical preintegration theory for graph-based sensor fusion with an inertial measurement unit (IMU) and a camera (or other aiding sensors). Rather than using discrete sampling of the measurement dynamics as in current methods, we derive the closed-form solutions to the preintegration equations, yielding improved accuracy in state estimation. We advocate two new different inertial models for preintegration: (i) the model that assumes piecewise constant measurements; and (ii) the model that assumes piecewise constant local true acceleration. Through extensive Monte Carlo simulations, we show the effect that the choice of preintegration model has on estimation performance. To validate the proposed preintegration theory, we develop both direct and indirect visual–inertial navigation systems (VINSs) that leverage our preintegration. In the first, within a tightly coupled, sliding-window optimization framework, we jointly estimate the features in the window and the IMU states while performing marginalization to bound the computational cost. In the second, we loosely couple the IMU preintegration with a direct image alignment that estimates relative camera motion by minimizing the photometric errors (i.e., image intensity difference), allowing for efficient and informative loop closures. Both systems are extensively validated in real-world experiments and are shown to offer competitive performance to state-of-the-art methods.

Direct visual-inertial odometry with semi-dense mapping

The paper presents a direct visual-inertial odometry system. In particular, a tightly coupled nonlinear optimization based method is proposed by integrating the recent development in direct dense visual tracking of camera and the inertial measurement unit (IMU) pre-integration. Then a factor graph optimization is adopted to estimate the pose and position of the camera, and a semi-dense map is created simultaneously. Two sliding windows are maintained in the proposed approach. The first one, based on direct sparse odometry (DSO), is to estimate the depths of candidate points for mapping and dense visual tracking. In the second one, measurements of both the IMU pre-integration and direct dense visual tracking are fused probabilistically based on a tightly-coupled, optimization-based sensor fusion framework. As a result, the scale drift of visual odometry is compensated by the constraints from the IMU pre-integration. Evaluations on real-world benchmark datasets show that the proposed method achieves competitive results in indoor scenes.