IMU Measurements Research Articles

Robust Inertial Motion Tracking through Deep Sensor Fusion across Smart Earbuds and Smartphone

IMU based inertial tracking plays an indispensable role in many mobility centric tasks, such as robotic control, indoor navigation and virtual reality gaming. Despite its mature application in rigid machine mobility (e.g., robot and aircraft), tracking human users via mobile devices remains a fundamental challenge due to the intractable gait/posture patterns. Recent data-driven models have tackled sensor drifting, one key issue that plagues inertial tracking. However, these systems still assume the devices are held or attached to the user body with a relatively fixed posture. In practice, natural body activities may rotate/translate the device which may be mistaken as whole body movement. Such motion artifacts remain as the dominating factor that fails existing inertial tracing systems in practical uncontrolled settings. Inspired by the observation that human heads induces far less intensive movement relative to the body during walking, compared to other parts, we propose a novel multi-stage sensor fusion pipeline called DeepIT, which realizes inertial tracking by synthesizing the IMU measurements from a smartphone and an associated earbud. DeepIT introduces a data-driven reliability aware attention model, which assesses the reliability of each IMU and opportunistically synthesizes their data to mitigate the impacts of motion noise. Furthermore, DeepIT uses a reliability aware magnetometer compensation scheme to combat the angular drifting problem caused by unrestricted motion artifacts. We validate DeepIT on the first large-scale inertial navigation dataset involving both smartphone and earbud IMUs. The evaluation results show that DeepIT achieves multiple folds of accuracy improvement on the challenging uncontrolled natural walking scenarios, compared with state-of-the-art closed-form and data-driven models.

A nonlinear optimization-based monocular dense mapping system of visual-inertial odometry

Nowadays, 3D reconstruction mainly uses depth cameras. For the disadvantages that depth cameras cannot get depth value outdoor and requires careful scanning to obtain accurate reconstruction effects, we propose a monocular dense mapping system. Since vision-only methods and IMU sensors are both easy to drift, we optimize the propagation strategies during the IMU preintegration. We marginalize the frames in the sliding window to bound the computational complexity. The back-end minimizes the marginalized IMU measurement residuals and visual residuals together. The four-level quadtree is applied to estimate the depth and reconstruct the map. We compare our method with five advanced odometry systems and compare the developed method with and without the IMU sensor on public EuRoC datasets. The results show that the method which involves IMU performs better than the vision-only case in general. The reconstruction result is accurate and clear.

Towards High-Performance Solid-State-LiDAR-Inertial Odometry and Mapping

We present a novel tightly-coupled LiDAR-inertial odometry and mapping scheme for both solid-state and mechanical LiDARs. As frontend, a feature-based lightweight LiDAR odometry provides fast motion estimates for adaptive keyframe selection. As backend, a hierarchical keyframe-based sliding window optimization is performed through marginalization for directly fusing IMU and LiDAR measurements. For the Livox Horizon, a newly released solid-state LiDAR, a novel feature extraction method is proposed to handle its irregular scan pattern during preprocessing. LiLi-OM (Livox LiDAR-inertial odometry and mapping) is real-time capable and achieves superior accuracy over state-of-the-art systems for both LiDAR types on public data sets of mechanical LiDARs and in experiments using the Livox Horizon. Source code and recorded experimental data sets are available at https://github.com/KIT-ISAS/lili-om.

Indoor way-finding method using IMU and magnetic tensor sensor measurements for visually impaired users

This paper introduces a new indoor way-finding method for the visually impaired person (VIP) by utilizing the naturally-generated inertial and geomagnetic information. Reliable and accurate indoor orientation and localization are provided by newly designed sensor fusion algorithms, which take advantage of inertial and geomagnetic information and overcome the inherent problems of the naturally-generated signals, such as low signal-to-noise ratio (SNR) and high environmental sensitivity. Geomagnetic information compensates the sensor drift and accumulative error of the inertial sensors whereas the inertial sensors help to correct the orientation-related errors and drift of the magnetic fields. A parameter derived from the magnetic tensor is introduced for obstacle avoidance and object/destination approach, especially when the relatively large localization uncertainty exists. With a prototype developed based on the system design, several experiments under different indoor scenarios demonstrate that the proposed indoor-way finding method can guide the VIPs and avoid obstacles indoor efficiently and accurately.

Parameter Estimation of an Industrial Car-Like Tractor

This letter presents a practical method for parameter estimation of a full-size industrial car-like tractor. With only on-vehicle sensors including three encoders, an IMU and a 3D LiDAR, we estimate the effective radius of rear wheels, ratio between the steering wheel angle and road wheel angle, the longitudinal position of center of gravity (COG), the yaw moment of inertia and the longitudinal and cornering tire stiffnesses. This letter innovatively introduces nonlinear vehicle-dynamics constraints into a factor-graph estimation framework that also fuses IMU and LiDAR measurements, thus achieving an easy-to-tune and highly-accurate parameter estimation solution. The batch maximum a posterior (MAP) problem is formulated and solved efficiently for both states and parameters. To make it even easier to tune, we perform specifically-designed motion patterns to simplify the required description model, and we estimate as fewer parameters as possible for one motion pattern. The dynamic constraints are also formulated according to the motion properties. Performance and results of the proposed method are validated in detail with experiments on the industrial car-like tractor.

An Efficient Approach to Initialization of Visual-Inertial Navigation System using Closed-Form Solution for Autonomous Robots

The visual-inertial navigation system using a single camera and IMU requires an accurate initialization without increasing the processing cost and complexity for real-time deployment. The processing cost in the existing solutions can be traced to the gyroscopic bias estimation using 1) Closed-form solutions (Martinelli, Int. J. Comput. Vis. 106(2), 138–152, 2014; Kaiser et al., IEEE Robot. Autom. Lett. 2(1), 18–25, 2017 and 2) Loosely coupled schemes using visual-inertial alignment (Mur-Artal and Tardos IEEE Robot. Autom. Lett. 2(2), 796–803 2017); Qin, IEEE Trans. Robot. 34(4), 1004–1020 2018). The complexity arises because of the non-linear nature of the system to estimate the gyro bias, which is solved either by directly solving the non-linear, non-convex problem or by decoupling the vision and IMU measurements using linear models. The termination conditions are based on condition number or covariance of the estimated variables, which varies from one experiment to another. The present paper seeks to improve the closed-form solution with higher accuracy and less processing cost per frame. The proposed method separates the gyroscope bias estimation from the closed-form solution using tightly coupled but linear models with reduced number of variables in the closed-form solution. This paper also addresses the problem inherent to the closed-form solutions, which requires sufficient motion in the initialization window with a minimum number of common features. Towards this, a novel method of propagating the past information into the present initialization window is presented. This reduces the total processing cost per frame by limiting the initialization window to 10 frames ≈ 1s without compromising the motion inside the window. We also present a common and intuitive termination criteria which is independent from the experiment scenario. This helps to increase the robustness in the initialization by removing erroneous solutions. The proposed method is evaluated with EuRoC Micro Aerial Vehicle (MAV) dataset (Burri et al. 2016) sequences. We compare the proposed method with a recently proposed loosely coupled method, which shows the improved accuracy, processing cost and robustness in the initialization.

Monocular Visual Inertial Direct SLAM with Robust Scale Estimation for Ground Robots/Vehicles

In this paper, we present a novel method for visual-inertial odometry for land vehicles. Our technique is robust to unintended, but unavoidable bumps, encountered when an off-road land vehicle traverses over potholes, speed-bumps or general change in terrain. In contrast to tightly-coupled methods for visual-inertial odometry, we split the joint visual and inertial residuals into two separate steps and perform the inertial optimization after the direct-visual alignment step. We utilize all visual and geometric information encoded in a keyframe by including the inverse-depth variances in our optimization objective, making our method a direct approach. The primary contribution of our work is the use of epipolar constraints, computed from a direct-image alignment, to correct pose prediction obtained by integrating IMU measurements, while simultaneously building a semi-dense map of the environment in real-time. Through experiments, both indoor and outdoor, we show that our method is robust to sudden spikes in inertial measurements while achieving better accuracy than the state-of-the art direct, tightly-coupled visual-inertial fusion method.

Real-Time Autonomous Vehicle Localization Based on Particle and Unscented Kalman Filters

In this paper, a real-time Monte Carlo localization (RT_MCL) method for autonomous cars is proposed. Unlike the other localization approaches, the balanced treatment of both pose estimation accuracy and its real-time performance is the main contribution. The RT_MCL method is based on the fusion of lidar and radar measurement data for object detection, a pole-like landmarks probabilistic map and a tailored particle filter for pose estimation. The lidar and radar are fused using the unscented Kalman filter (UKF) to provide pole-like static-object pose estimations that are well suited to serve as landmarks for vehicle localization in urban environments. These pose estimations are then clustered using the grid-based density-based spatial clustering of applications with noise algorithm to represent each pole landmark in the form of a source-point model to reduce computational cost and memory requirements. A reference map that includes pole landmarks is generated offline and extracted from a 3-D lidar to be used by a carefully designed particle filter for accurate ego-car localization. The particle filter is initialized by the fused GPS + IMU measurements and used an ego-car motion model to predict the states of the particles. The data association between the estimated landmarks by the UKF and that in the reference map is performed using the iterative closest point algorithm. The RT_MCL is implemented using the high-performance language C++ and utilizes highly optimized math and optimization libraries for best real-time performance. Extensive simulation studies have been carried out to evaluate the performance of the RT_MCL in both longitudinal localization and lateral localization.

Position Detection for Electric Vehicle Dwcs Using Vi-Slam Method

The dynamic wireless charging system (DWCS) is developed to solve the problems of large battery volume and mileage anxiety of electric vehicles. However, the accurate position detection for electric vehicle DWCS is facing challenge. The traditional communication, detection and estimation methods are difficult to accurately obtain the position. To tackle this problem, the visual inertial simultaneous localization and mapping (VI-SLAM) method is applied to the electric vehicles DWCS. Firstly, the graph optimization based tight coupling method is used to integrate the monocular visual and IMU measurements. Secondly, the NVIDIA TX2 and MYNT VI-sensor suite are assembled, which the MTi300-IMU is treated as the ground truth system. Finally, the mobile vehicle is controlled to race on the simulated DWCS pathway. The experimental result shows that the method achieves great performance with the accuracy of centimeter level. In particular, the root mean square error (RMSE) in X, Y, Z directions are 0.086,0.092,0.102, respectively.

On State Estimation for Legged Locomotion Over Soft Terrain

Locomotion over soft terrain remains a challenging problem for legged robots. Most of the work done on state estimation for legged robots is designed for rigid contacts and does not take into account the physical parameters of the terrain. That said, this letter answers the following questions: How and why does soft terrain affect state estimation for legged robots? To do so, we utilized a state estimator that fuses IMU measurements with leg odometry that is designed with rigid contact assumptions. We experimentally validated the state estimator with the HyQ robot trotting over both soft and rigid terrain. We demonstrate that soft terrain negatively affects state estimation for legged robots and that the state estimates have a noticeable drift over soft terrain compared to rigid terrain.

TLIO: Tight Learned Inertial Odometry

In this work we propose a tightly-coupled Extended Kalman Filter framework for IMU-only state estimation. Strap-down IMU measurements provide relative state estimates based on IMU kinematic motion model. However the integration of measurements is sensitive to sensor bias and noise, causing significant drift within seconds. Recent research by Yan et al. (RoNIN) and Chen et al. (IONet) showed the capability of using trained neural networks to obtain accurate 2D displacement estimates from segments of IMU data and obtained good position estimates from concatenating them. This paper demonstrates a network that regresses 3D displacement estimates and its uncertainty, giving us the ability to tightly fuse the relative state measurement into a stochastic cloning EKF to solve for pose, velocity and sensor biases. We show that our network, trained with pedestrian data from a headset, can produce statistically consistent measurement and uncertainty to be used as the update step in the filter, and the tightly-coupled system outperforms velocity integration approaches in position estimates, and AHRS attitude filter in orientation estimates.

Improved IMU Preintegration with Gravity Change and Earth Rotation for Optimization-Based GNSS/VINS

IMU preintegration technology has been widely used in the optimization-based sensor fusion framework, in order to avoid reintegrating the high-frequency IMU measurements at each iteration and maintain the ability of bias correction when bias estimation changes. Since IMU preintegration technology was first proposed, several improved versions have been designed by changing the attitude parameterization or the numerical integration method in the most current related research. However, all of these versions have failed to take the change of gravity and the earth rotation into consideration. In this paper, we redesign the IMU preintegration algorithm in which the earth rotation and gravity vector are calculated from the geodetic position. Compared with the covariance matrix form, in this paper, the uncertainty of the preintegrated IMU measurements is propagated in the form of a square root information matrix (SRIM) for better numerical stability and easy use in the optimization-based framework. We evaluate the improved IMU preintegration algorithm by using the dataset collected by our sensor platform equipped with two different-grade IMUs. The test results show that the improved IMU preintegration algorithm can cope well with the gravity change and earth rotation. The earth rotation must be taken into consideration for the high-grade IMU that can effectively sense the earth rotation. If the change of gravity is omitted, the root-mean-square error (RMSE) of the horizontal attitude is about 1.38 times greater than the geodetic displacement. Additionally, the positioning RMSE does not increase obviously within a limited range, which means tens of kilometers and several hundred meters for the low-grade and high-grade IMU used in the experiment, respectively.

Accurate and robust odometry by fusing monocular visual, inertial, and wheel encoder

Tracking the pose of a robot has been gaining importance in the field of Robotics, e.g., paving the way for robot navigation. In recent years, monocular visual–inertial odometry (VIO) is widely used to do the pose estimation due to its good performance and low cost. However, VIO cannot estimate the scale or orientation accurately when robots move along straight lines or circular arcs on the ground. To address the problem, in this paper we take the wheel encoder into account, which can provide us with stable translation information as well as small accumulated errors and momentary slippage errors. By jointly considering the kinematic constraints and the planar moving features, an odometry algorithm tightly coupled with monocular camera, IMU, and wheel encoder is proposed to get robust and accurate pose sensing for mobile robots, which mainly contains three steps. First, we present the wheel encoder preintegration theory and noise propagation formula based on the kinematic mobile robot model, which is the basis of accurate estimation in backend optimization. Second, we adopt a robust initialization method to obtain good initial values of gyroscope bias and visual scale in reality, by making full use of the camera, IMU and wheel encoder measurements. Third, we bound the high computation complexity with a marginalization strategy that conditionally eliminates unnecessary measurements in the sliding window. We implement a prototype and several extensive experiments showing that our system can achieve robust and accurate pose estimation, in terms of the scale, orientation and location, compared with the state-of-the-art.

SD-VIS: A Fast and Accurate Semi-Direct Monocular Visual-Inertial Simultaneous Localization and Mapping (SLAM).

In practical applications, how to achieve a perfect balance between high accuracy and computational efficiency can be the main challenge for simultaneous localization and mapping (SLAM). To solve this challenge, we propose SD-VIS, a novel fast and accurate semi-direct visual-inertial SLAM framework, which can estimate camera motion and structure of surrounding sparse scenes. In the initialization procedure, we align the pre-integrated IMU measurements and visual images and calibrate out the metric scale, initial velocity, gravity vector, and gyroscope bias by using multiple view geometry (MVG) theory based on the feature-based method. At the front-end, keyframes are tracked by feature-based method and used for back-end optimization and loop closure detection, while non-keyframes are utilized for fast-tracking by direct method. This strategy makes the system not only have the better real-time performance of direct method, but also have high accuracy and loop closing detection ability based on feature-based method. At the back-end, we propose a sliding window-based tightly-coupled optimization framework, which can get more accurate state estimation by minimizing the visual and IMU measurement errors. In order to limit the computational complexity, we adopt the marginalization strategy to fix the number of keyframes in the sliding window. Experimental evaluation on EuRoC dataset demonstrates the feasibility and superior real-time performance of SD-VIS. Compared with state-of-the-art SLAM systems, we can achieve a better balance between accuracy and speed.

An Enhanced INS/GNSS Tightly Coupled Navigation System Using Time-Differenced Carrier Phase Measurement

To enhance the long-term localization accuracy of inertial measurement unit/time-differenced carrier phase (IMU/TDCP) tightly coupled navigation system (TCS), efforts from three aspects have been made in this article. First, we make a comprehensive analysis, which has not been done before, on the errors of TDCP measurement. Then, these errors are modeled and mostly compensated using the polynomial fitting method. Even so, the errors cannot be eliminated completely. Positioning performance will get worse slowly over time due to the error remained in the error-compensated TDCP (CTDCP). Therefore, the final effort has been done in introducing a novel measurement, ionosphere-free wide-lane carrier phase (IFWLCP), to control the deterioration instead of the conventional coarse pseudorange (PR). The CTDCP and IFWLCP are both integrated with IMU measurements using the dual-rate Kalman filter, and the proposed methodology’s effectiveness on improving the positioning accuracy is validated by a road experiment.

Fast and Robust Monocular Visua-Inertial Odometry Using Points and Lines.

When the camera moves quickly and the image is blurred or the texture in the scene is missing, the Simultaneous Localization and Mapping (SLAM) algorithm based on point feature experiences difficulty tracking enough effective feature points, and the positioning accuracy and robustness are poor, and even may not work properly. For this problem, we propose a monocular visual odometry algorithm based on the point and line features and combining IMU measurement data. Based on this, an environmental-feature map with geometric information is constructed, and the IMU measurement data is incorporated to provide prior and scale information for the visual localization algorithm. Then, the initial pose estimation is obtained based on the motion estimation of the sparse image alignment, and the feature alignment is further performed to obtain the sub-pixel level feature correlation. Finally, more accurate poses and 3D landmarks are obtained by minimizing the re-projection errors of local map points and lines. The experimental results on EuRoC public datasets show that the proposed algorithm outperforms the Open Keyframe-based Visual-Inertial SLAM (OKVIS-mono) algorithm and Oriented FAST and Rotated BRIEF-SLAM (ORB-SLAM) algorithm, which demonstrates the accuracy and speed of the algorithm.

MotionTransformer: Transferring Neural Inertial Tracking between Domains

Inertial information processing plays a pivotal role in egomotion awareness for mobile agents, as inertial measurements are entirely egocentric and not environment dependent. However, they are affected greatly by changes in sensor placement/orientation or motion dynamics, and it is infeasible to collect labelled data from every domain. To overcome the challenges of domain adaptation on long sensory sequences, we propose MotionTransformer - a novel framework that extracts domain-invariant features of raw sequences from arbitrary domains, and transforms to new domains without any paired data. Through the experiments, we demonstrate that it is able to efficiently and effectively convert the raw sequence from a new unlabelled target domain into an accurate inertial trajectory, benefiting from the motion knowledge transferred from the labelled source domain. We also conduct real-world experiments to show our framework can reconstruct physically meaningful trajectories from raw IMU measurements obtained with a standard mobile phone in various attachments.

Force from Motion: Decoding Control Force of Activity in a First-Person Video.

A first-person video delivers what the camera wearer (actor) experiences through physical interactions with surroundings. In this paper, we focus on a problem of Force from Motion-estimating the active force and torque exerted by the actor to drive her/his activity-from a first-person video. We use two physical cues inherited in the first-person video. (1) Ego-motion: the camera motion is generated by a resultant of force interactions, which allows us to understand the effect of the active force using Newtonian mechanics. (2) Visual semantics: the first-person visual scene is deployed to afford the actor's activity, which is indicative of the physical context of the activity. We estimate the active force and torque using a dynamical system that can describe the transition (dynamics) of the actor's physical state (position, orientation, and linear/angular momentum) where the latent physical state is indirectly observed by the first-person video. We approximate the physical state with the 3D camera trajectory that is reconstructed up to scale and orientation. The absolute scale factor and gravitation field are learned from the ego-motion and visual semantics of the first-person video. Inspired by an optimal control theory, we solve the dynamical system by minimizing reprojection error. Our method shows quantitatively equivalent reconstruction comparing to IMU measurements in terms of gravity and scale recovery and outperforms the methods based on 2D optical flow for an active action recognition task. We apply our method to first-person videos of mountain biking, urban bike racing, skiing, speedflying with parachute, and wingsuit flying where inertial measurements are not accessible.

Analysis of the performance of 17 algorithms from a systematic review: Influence of sensor position, analysed variable and computational approach in gait timing estimation from IMU measurements

BackgroundThe quantification of gait temporal parameters (i.e. step time, stance time) is crucial in human motion analysis and requires the accurate identification of gait events (i.e. heel strike, toe off). With the widespread use of inertial wearable sensors, many algorithms were proposed and applied for the purpose. Nevertheless, only few studies addressed the assessment of the actual performance of these algorithms, rather considering each proposed algorithm as a whole. Research questionHow different implementation characteristics influence the assessment of gait events and temporal parameters from inertial sensor measures in terms of accuracy and repeatability? MethodsSeventeen different algorithms were identified from a systematic review and classified based on: 1) sensor position, 2) target variable, 3) computational approach. The influence of these characteristics was analysed on walking data of 35 healthy volunteers mounting 5 tri-axial inertial sensors. Foot contact events identified by 2 force platforms were assumed as gold standard. Temporal parameters were calculated from gait events. Algorithm performance was analysed in terms of accuracy (error median value) and repeatability (error 25th and 75th percentile values). ResultsShank- and foot-based algorithms performed better (in terms of accuracy and repeatability) in gait events detection and stance time estimation than lower trunk-based ones, while sensor position did not affect step estimate, given the error bias characteristics. Angular velocity-based algorithms performed significantly better than acceleration-based ones for toe off detection in terms of repeatability (68 ms and 102 ms, 25th–75th percentile error range, respectively) and, for heel strike detection, showed better repeatability (40 ms and 111 ms) and comparable accuracy (65 ms and 60 ms median error, respectively) than acceleration-based ones. The performance of different computational approaches varied depending on sensor positioning. SignificancePresent results support the selection of the proper algorithm for the estimation of gait events and temporal parameters in relation to the specific application.

A direct optic flow-based strategy for inverse flight altitude estimation with monocular vision and IMU measurements

With tiny and limited nervous systems, insects demonstrate a remarkable ability to fly through complex environments. Optic flow has been identified to play a crucial role in regulating flight conditions and navigation in flies and bees. In robotics, optic flow has been widely studied thanks to the low computational requirements. However, with only monocular visual information, optic flow is inherently devoid of a scale factor required for estimating the absolute distance. In this paper, we propose a strategy for estimating the flight altitude of a flying robot with a ventral camera by combining the optic flow with measurements from an inertial measurement unit. Instead of using the prevalent feature-based approach for calculation of optic flow, we implement a direct method that evaluates the flow information via image gradients. We show that the direct approach notably simplifies the computation steps compared to the feature-based method. When combined with an extended Kalman filter for fusion of inertial measurement units measurements, the flight altitude can be estimated in real time. We carried out extensive flight tests in different settings. Among 31 hovering and vertical flights near the altitude of 40 cm, we achieved the RMS errors in the altitude estimate of 2.51 cm. Further analysis of factors that affect the quality of the flow and the distance estimate is also provided.