Vision-aided Inertial Navigation Research Articles

Vision-aided Inertial Navigation for Planetary Landing without Feature Extraction and Matching

In this paper, a vision-aided inertial navigation method for planetary landing based on the direct method is proposed. This approach requires only a monocular camera and an inertial measurement unit (IMU) to be equipped on the lander. Unlike traditional feature-based visual-aided inertial navigation algorithms, which entail extracting and matching image features and thus incurring high computational costs, the proposed method utilizes the direct method that tracks any pixel with a sufficient intensity gradient. By jointly optimizing photometric and inertial errors, the proposed algorithm estimates the camera motion and sparse map of the scene. Simulation results demonstrate that the proposed method reduces computation by nearly 65% compared to the feature point method, with a slight decrease in navigation accuracy.

MSCEqF: A Multi State Constraint Equivariant Filter for Vision-Aided Inertial Navigation

MSCEqF: A Multi State Constraint Equivariant Filter for Vision-Aided Inertial Navigation

Onboard Observation Planning for Sequential Image-based Planetary Landing Navigation in Unknown Environments

Autonomous navigation based on sequential images is widely accepted as an effective method to guarantee pinpoint landing and precise obstacle avoidance in planetary landings. Observing a large number of landmarks and processing a large number of images places a computational burden on landers with limited computational resources, but existing studies lack observation planning methods for landmarks in unknown environments that can be operated autonomously on landers. In this paper, we analyze the observability of a vision-aided inertial navigation (VAIN) system and propose an observation planning strategy. The minimum number of landmarks and the minimum number of observations are obtained from the observability analysis, which is the boundary condition for observation planning. The landmark configuration planning in the observation planning strategy optimizes only univariate convex functions to efficiently select a small number of landmarks in unknown environments with high navigation accuracy, and the observation interval planning adaptively reduces the number of observations without significantly affecting the navigation accuracy. Simulation results verify the correctness of the observability analysis results, and it is found that the proposed observation planning strategy outperforms traditional observability degrees in terms of both improved navigation accuracy and computational speed when observing a small number of landmarks, and can effectively reduce the number of observations.

Observer-based controller for VTOL-UAVs tracking using direct Vision-Aided Inertial Navigation measurements

This paper proposes a novel observer-based controller for Vertical Take-Off and Landing (VTOL) Unmanned Aerial Vehicle (UAV) designed to directly receive measurements from a Vision-Aided Inertial Navigation System (VA-INS) and produce the required thrust and rotational torque inputs. The VA-INS is composed of a vision unit (monocular or stereo camera) and a typical low-cost 6-axis Inertial Measurement Unit (IMU) equipped with an accelerometer and a gyroscope. A major benefit of this approach is its applicability for environments where the Global Positioning System (GPS) is inaccessible. The proposed VTOL-UAV observer utilizes IMU and feature measurements to accurately estimate attitude (orientation), gyroscope bias, position, and linear velocity. Ability to use VA-INS measurements directly makes the proposed observer design more computationally efficient as it obviates the need for attitude and position reconstruction. Once the motion components are estimated, the observer-based controller is used to control the VTOL-UAV attitude, angular velocity, position, and linear velocity guiding the vehicle along the desired trajectory in six degrees of freedom (6 DoF). The closed-loop estimation and the control errors of the observer-based controller are proven to be exponentially stable starting from almost any initial condition. To achieve global and unique VTOL-UAV representation in 6 DoF, the proposed approach is posed on the Lie Group and the design in unit-quaternion is presented. Although the proposed approach is described in a continuous form, the discrete version is provided and tested.

Closed-Form Error Propagation on $SE_{n}(3)$ Group for Invariant EKF With Applications to VINS

Pose estimation is important for robotic perception, path planning, etc. Robot poses can be modeled on matrix Lie groups and are usually estimated via filter-based methods. In this letter, we establish the closed-form formula for the error propagation for the Invariant extended Kalman filter (IEKF) in the presence of random noises and apply it to vision-aided inertial navigation. Moreover, we use the theoretic results to add the compensation parts for IEKF. We evaluate our algorithms via numerical simulations and experiments on the OPENVINS platform. Both simulations and the experiments performed on the public EuRoC MAV datasets demonstrate that our algorithm in particular parameters settings outperforms some state-of-art filter-based methods such as the quaternion-based EKF, first estimates Jacobian EKF, etc. The techniques of choice on the parameters are worth further investigating.

Nonlinear Observers Design for Vision-Aided Inertial Navigation Systems

This paper deals with the simultaneous estimation of the attitude, position and linear velocity for vision-aided inertial navigation systems. We propose a nonlinear observer on $SO(3)\times \mathbb{R}^{15}$ relying on body-frame acceleration, angular velocity and (stereo or monocular) bearing measurements of some landmarks that are constant and known in the inertial frame. Unlike the existing local Kalman-type observers, our proposed nonlinear observer guarantees almost global asymptotic stability and local exponential stability. A detailed uniform observability analysis has been conducted and sufficient conditions are derived. Moreover, a hybrid version of the proposed observer is provided to handle the intermittent nature of the measurements in practical applications. Simulation and experimental results are provided to illustrate the effectiveness of the proposed state observer.

Research on A Low Computational Cost Vision-aided Inertial Navigation Method for Precision Landing on Asteroid

Asteroid exploration missions demand more autonomous navigation system that are precise and in real-time. Visual-aided inertial navigation systems combine the advantages of autonomy of inertial navigation systems and accuracy of visual navigation systems. However, both increase the computational burden for the information processing. With the goal of decreasing the computational load, this paper proposes a low cost vision-aided inertial navigation method which can calculate increment position, velocity and attitude directly in inertial frame by pre-integrating IMU measurements. For visual information, specific combinations of the features are selected according to information entropy theory to reduce computation. Simulation results show that the standard deviations of the spacecraft’s estimated position error is less than 1 m, the velocity less than 0.002 m/s and the attitude angle less than 0.005 deg, compatible with precision landing on asteroids.

Landmark database selection for vision-aided inertial navigation in planetary landing missions

Vision-aided inertial navigation (VAIN) system offers a means to enable high-accuracy autonomous navigation for planetary landing. The VAIN system provides precise state estimates for the landing vehicle by combining measurements from an inertial measurement unit (IMU) with visual observations of the mapped landmarks contained in a predetermined terrain database. Due to the constraints of the onboard storage and computation capabilities of the landing vehicle, a terrain database containing the minimum number of landmarks should be found to guarantee the requirements of the navigation task. In this paper, a landmark database selection algorithm based on linear covariance (LinCov) technique is presented. The algorithm selects landmarks into the terrain database by minimizing the uncertainties of the chosen state parameters at each imaging time point along the nominal trajectory. The uncertainties are calculated by LinCov which is an efficient approach to analyze the effect of landmark databases on the navigation performance and allows for considering the visibility constraints of landmarks prior to actual flight. Extensive simulations are performed to evaluate the proposed landmark selection algorithm, and the results show that the proposed method is effective.

Robust Vision-Aided Inertial Navigation System for Protection Against Ego-Motion Uncertainty of Unmanned Ground Vehicle

The vision-aided inertial navigation system is of vital importance and great urgency to guarantee accurate and reliable navigation information for unmanned ground vehicles (UGV) in Global Navigation Satellite System-challenged/denied environments. Yet, it is almost inevitable to occur unpredictable stochastic disturbances caused by ego-motion uncertainty due to uneven pavements, unstructured and dynamic scenes, which may lead to the divergence of estimation algorithms and the unreliability of positioning. In this article, we aim at developing a robust strategy for vision-aided inertial navigation to resist uncertainties, and modifying unscented Kalman filter by formulating a novel adaptive factor according to the degree of abnormity (DoA). Applying the robust strategy and algorithms, a monocular camera and an inertial measurement unit (IMU) are tightly fused to construct a robust vision-aided inertial navigation system. During the fusion process of integrated IMU and camera, the new method can automatically estimate the DoA of UGV ego-motion uncertainty online, considering the relationship between the factual and estimated innovation covariances. Thus, we can adaptively adjust the estimation error covariance matrix and the Kalman gain for different scenarios on the basis of DoA levels. As the modification is made in the framework of Kalman filter, the real-time performance, robustness, and high-accuracy positioning can be achieved using the proposed adjustment strategy. Experiments with the KITTI dataset are carried out under different road conditions. The experimental results show that the robust vision-aided inertial navigation system possesses good robustness and self-adaptation for protection against ego-motion uncertainty, and can improve the positioning accuracy of UGV.

Variational Bayesian adaptive high‐degree cubature Huber‐based filter for vision‐aided inertial navigation on asteroid missions

Vision-aided inertial navigation (VAIN) is a prospective technique for determining the pose of the spacecraft during asteroid missions. The VAIN system can fuse the inertial and visual data by employing the high-degree cubature Kalman filter (HCKF) because it can accurately handle non-linear problems. However, the visual measurements can be corrupted by non-Gaussian noise with unknown time-varying covariance, resulting in severe degradation of the HCKF. To improve the navigational accuracy of the spacecraft in these situations, the authors propose a novel adaptive robust HCKF known as variational Bayesian (VB) adaptive high-degree cubature Huber-based filter (VB-AHCHF). In the novel algorithm, the fifth-degree cubature rule and VB theory are combined to estimate the state and track the non-stationary statistical characteristics of the measurement noise. In addition, utilising the M-estimation, which is defined as the Huber technique, it modifies the update step of the formal Bayesian filtering. Therefore, the VB-AHCHF can exhibit adaptability and robustness to the covariance uncertainty and non-Gaussianity of the measurement noise. Their simulation results show that the estimation accuracy of VB-AHCHF, as well as its adaptability and robustness, is superior to all state-of-the-art algorithms, e.g. HCKF, high-degree cubature Huber-based filter, and the VB adaptive HCKF.

Camera Intrinsic Parameters Estimation by Visual–Inertial Odometry for a Mobile Phone With Application to Assisted Navigation

The increasing computing and sensing capabilities of modern mobile phones have spurred research interests in developing new visual–inertial odometry (VIO) techniques to turn a smartphone into a self-contained vision-aided inertial navigation system for various applications. Smartphones nowadays use cameras with optical image stabilization (OIS) technology to reduce image blurs. However, the mechanism may result in varying camera intrinsic parameters (CIP), which must be taken into account in VIO computation. In this article, we first develop a linear model to relate the CIP with the inertial measurement unit measured acceleration. Based on the model, we introduce a new VIO method, called CIP-VMobile, which treats CIP as state variables and tightly couples them with other state variables in a graph optimization process to estimate the optimal state. The method uses the linear model to construct a factor graph and uses the linear-model-computed values as initial CIP estimates to speed up the VIO computation and attain a better pose estimation result. Simulation and experimental results with an iPhone 7 validate the method's efficacy. Based on CIP-VMobile, we fabricated a robotic navigation aid (RNA) based on an iPhone 7 for assisted navigation. Experimental results with the RNA demonstrate CIP-VMobile's promise in real-world navigation applications.

Robust Outlier-Adaptive Filtering for Vision-Aided Inertial Navigation

With the advent of unmanned aerial vehicles (UAVs), a major area of interest in the research field of UAVs has been vision-aided inertial navigation systems (V-INS). In the front-end of V-INS, image processing extracts information about the surrounding environment and determines features or points of interest. With the extracted vision data and inertial measurement unit (IMU) dead reckoning, the most widely used algorithm for estimating vehicle and feature states in the back-end of V-INS is an extended Kalman filter (EKF). An important assumption of the EKF is Gaussian white noise. In fact, measurement outliers that arise in various realistic conditions are often non-Gaussian. A lack of compensation for unknown noise parameters often leads to a serious impact on the reliability and robustness of these navigation systems. To compensate for uncertainties of the outliers, we require modified versions of the estimator or the incorporation of other techniques into the filter. The main purpose of this paper is to develop accurate and robust V-INS for UAVs, in particular, those for situations pertaining to such unknown outliers. Feature correspondence in image processing front-end rejects vision outliers, and then a statistic test in filtering back-end detects the remaining outliers of the vision data. For frequent outliers occurrence, variational approximation for Bayesian inference derives a way to compute the optimal noise precision matrices of the measurement outliers. The overall process of outlier removal and adaptation is referred to here as “outlier-adaptive filtering”. Even though almost all approaches of V-INS remove outliers by some method, few researchers have treated outlier adaptation in V-INS in much detail. Here, results from flight datasets validate the improved accuracy of V-INS employing the proposed outlier-adaptive filtering framework.

A Hybrid Sliding Window Optimizer for Tightly-Coupled Vision-Aided Inertial Navigation System.

The fusion of visual and inertial measurements for motion tracking has become prevalent in the robotic community, due to its complementary sensing characteristics, low cost, and small space requirements. This fusion task is known as the vision-aided inertial navigation system problem. We present a novel hybrid sliding window optimizer to achieve information fusion for a tightly-coupled vision-aided inertial navigation system. It possesses the advantages of both the conditioning-based method and the prior-based method. A novel distributed marginalization method was also designed based on the multi-state constraints method with significant efficiency improvement over the traditional method. The performance of the proposed algorithm was evaluated with the publicly available EuRoC datasets and showed competitive results compared with existing algorithms.

A smart hybrid vision aided inertial navigation system approach for UAVs in a GNSS denied environment

This paper presents a smart hybrid vision-aided inertial navigation system (VAINS) for enhancing the navigation accuracy of UAVs in GNSS-denied environments. The proposed system consists of GNSS, INS, monocular visual odometry (VO) based on optical flow and regression trees, a Gaussian process regression (GPR), and extended Kalman filter (EKF) for data fusion. Although a variety of monocular VO-based on photogrammetric, structure from motion, and machine learning approaches have been proposed to assist the navigation process, the accuracy of 3-D positioning using these techniques is still affected by some factors such as the lack of the observed features, incorrect matches, and the accumulated positioning drift errors. Therefore, the development of a reliable navigation system that can handle such challenges is necessary. In this system, the GPR algorithm is deployed to model these errors when the GNSS signal is available and to correct them when the GNSS signal is lost.

Vision-aided localization and navigation based on trifocal tensor geometry

In this paper, a new method for vision-aided navigation based on trifocal tensor geometry is presented. The main goal of the proposed method is to estimate the position of vehicles in global positioning system-denied environments, using a standard inertial navigation system and only a single camera. The geometric trifocal tensor relationship between three images is used as measurement information from the camera, and the primary contribution of this work is the derivation of a measurement model that is able to express the geometric constraints of the trifocal tensor in the global frame. This measurement model does not require including the three-dimensional feature positions in the state vector. In other words, the proposed method does not entail reconstructing the environment. Rather, the method only considers the vehicle state. The vision-aided inertial navigation algorithm that we propose has computational complexity only with regard to the number of features at the current time, and the algorithm is capable of estimating the pose in real environments. Experiments were conducted to show the effectiveness of the proposed method in simulations and real environments.

Factored Extended Kalman Filter for Monocular Vision-Aided Inertial Navigation

This paper describes a novel visual simultaneous localization and mapping system based on the UD factored extended Kalman filter. A novel method for marginalizing and initializing state variables is presented, allowing for features to be added and removed directly to and from the covariance factors. An analysis of the number of operations in the marginalization and initialization algorithm are presented. A randomized analysis demonstrates the improvement in numerical stability over the standard and Joseph-form extended Kalman filter by as much as three orders of magnitude. The navigation system is implemented on an 80 kg unmanned aerial vehicle and a 0.5 kg unmanned aerial vehicle, and flight-test and simulation results with a controller in the loop are presented. The flight-test results agree with the simulation results and show a low state error and a consistent error covariance.

EXTENSION AND EVALUATION OF THE AGAST FEATURE DETECTOR

Vision-aided inertial navigation is a navigation method which combines inertial navigation with computer vision techniques. It can provide a six degrees of freedom navigation solution from passive measurements without external referencing (e.g. GPS). Thus, it can operate in unknown environments without any prior knowledge. Such a system, called IPS (Integrated Positioning System) is developed by the German Aerospace Center (DLR). <br><br> For optical navigation applications, a reliable and efficient feature detector is a crucial component. With the publication of AGAST, a new feature detector has been presented, which is faster than other feature detectors. To apply AGAST to optical navigation applications, we propose several methods to improve its performance. Based on a new non-maximum suppression algorithm, automatic threshold adaption algorithm in combination with an image split method, the optimized AGAST provides higher reliability and efficiency than the original implementation using the Kanade Lucas Tomasi (KLT) feature detector. Finally, we compare the performance of the optimized AGAST with the KLT feature detector in the context of IPS. The presented approach is tested using real data from typical indoor scenes, evaluated on the accuracy of the navigation solution. The comparison demonstrates a significant performance improvement achieved by the optimized AGAST.

EXTENSION AND EVALUATION OF THE AGAST FEATURE DETECTOR

Abstract. Vision-aided inertial navigation is a navigation method which combines inertial navigation with computer vision techniques. It can provide a six degrees of freedom navigation solution from passive measurements without external referencing (e.g. GPS). Thus, it can operate in unknown environments without any prior knowledge. Such a system, called IPS (Integrated Positioning System) is developed by the German Aerospace Center (DLR). For optical navigation applications, a reliable and efficient feature detector is a crucial component. With the publication of AGAST, a new feature detector has been presented, which is faster than other feature detectors. To apply AGAST to optical navigation applications, we propose several methods to improve its performance. Based on a new non-maximum suppression algorithm, automatic threshold adaption algorithm in combination with an image split method, the optimized AGAST provides higher reliability and efficiency than the original implementation using the Kanade Lucas Tomasi (KLT) feature detector. Finally, we compare the performance of the optimized AGAST with the KLT feature detector in the context of IPS. The presented approach is tested using real data from typical indoor scenes, evaluated on the accuracy of the navigation solution. The comparison demonstrates a significant performance improvement achieved by the optimized AGAST.

Observability Criteria and Null-Measurement Kalman Filter for Vision-Aided Inertial Navigation

This work presents a linear time-varying approach to real-time vision-aided inertial navigation. An exact measurement model is provided, which linearly depends on the state and is therefore suitable for Kalman filtering. Sufficient observability and nonobservability criteria are derived, which can be used to identify observable trajectories. Moreover, the paper proposes a filter able to determine the absolute scale of the camera motion without any need for estimating three-dimensional feature positions or a priori knowledge of the absolute attitude. The effectiveness of the filter on low-cost hardware is demonstrated in flight tests with a small fixed-wing unmanned aerial vehicle.

Visual–inertial navigation for pinpoint planetary landing using scale-based landmark matching

Landing an autonomous spacecraft within 100 m of a mapped target is a navigation challenge in planetary exploration. Vision-based approaches attempt to pair 2D features detected in camera images with 3D mapped landmarks to reach the required precision. This paper presents a vision-aided inertial navigation system for pinpoint planetary landing called LION (Landing Inertial and Optical Navigation). It can fly over any type of terrain, whatever the topography. LION uses measurements from a novel image-to-map matcher in order to update through a tight data fusion scheme the state of an extended Kalman filter propagated with inertial data. The image processing uses the state and covariance predictions from the filter to determine the regions and extraction scales in which to search for non-ambiguous landmarks in the image. The image scale management process operates per landmark and greatly improves the repeatability rate between the map and descent images. A lunar-representative optical test bench called Visilab was also designed in order to test LION. The observability of absolute navigation performances in Visilab is evaluated with a model developed specifically for this purpose. Finally, the system performances are evaluated at a number of altitudes along with its robustness to off-nadir camera angle, illumination changes, a different map generation process and non-planar topography. The error converges to a mean of 4 m and a 3-RMS dispersion of 47 m at 3 km of altitude on the test setup at scale.