IMU Data Research Articles

An improved semi-synthetic approach for creating visual-inertial odometry datasets

Capturing outdoor visual-inertial datasets is a challenging yet vital aspect of developing robust visual-inertial odometry (VIO) algorithms. A significant hurdle is that high-accuracy-ground-truth systems (e.g., motion capture) are not practical for outdoor use. One solution is to use a “semi-synthetic” approach that combines rendered images with real IMU data. This approach can produce sequences containing challenging imagery and accurate ground truth but with less simulated data than a fully synthetic sequence. Existing methods (used by popular tools/datasets) record IMU measurements from a visual-inertial system while measuring its trajectory using motion capture, then rendering images along that trajectory. This work identifies a major flaw in that approach, specifically that using motion capture alone to estimate the pose of the robot/system results in the generation of inconsistent visual-inertial data that is not suitable for evaluating VIO algorithms. However, we show that it is possible to generate high-quality semi-synthetic data for VIO algorithm evaluation. We do so using an open-source full-batch optimisation tool to incorporate both mocap and IMU measurements when estimating the IMU’s trajectory. We demonstrate that this improved trajectory results in better consistency between the IMU data and rendered images and that the resulting data improves VIO trajectory error by 79% compared to existing methods. Furthermore, we examine the effect of visual-inertial data inconsistency (as a result of trajectory noise) on VIO performance to provide a foundation for future work targeting real-time applications.

A Robust Foot-Mounted Positioning System Based on Dual IMU Data and Ultrasonic Ranging

Foot-mounted positioning system (FPS) is regarded as an important application for GPS-denied scenes, including but not limited to emergency rescue, medical health, and individual soldier positioning. With the assistance of zero-velocity update technology (ZUPT), the basic positioning function can be realized, while the largest problem is that the accumulation of systematic heading drift and positioning error cannot be completely constrained. This article presents a robust FPS based on dual inertial measurement unit data and ultrasonic ranging (FPS-DU). Bidirectional long short-term memory (Bi-LSTM) network is developed for the recognition of quasi-static (QS) period for the performance enhancement of ZUPT, and multilevel constraints are modeled for decreasing the cumulative error of the single foot-mounted model. In addition, a unified Kalman-based location estimator is developed combining dual foot-mounted inertial data and sphere constraint-based ultrasonic ranging results, enhanced by a robust ultrasonic outlier detector. Comprehensive experiments demonstrate that the proposed FPS-DU realizes much more accurate and robust positioning accuracy (between 0.371% and 0.548% under different test environments and traveled distances) compared with traditional single FPSs (S-FPSs).

Testing decision-making skills in football using augmented reality

Decision-making skills are fundamental to successful performance in team sports. Current approaches to assess decision-making skills are incredibly resource intensive for both athletes and researchers. Athletes are required to travel to testing facilities and researchers invest time and effort to create representative experiments. A less resource intensive procedure is highly desired.
 In this study, the feasibility of an on-field decision-making task was assessed. The task was presented in augmented reality. The situations in the task were retrieved from open-source tracking data. Short sequences of build-up play were arranged into a testing battery based on a similarity score. For the similarity score, we harnessed the Kuhn-Munkres algorithm (Kuhn, 1955; Munkres, 1957) to select similar situations based on player positions and movement directions. The sampled situations preserve the natural variability experienced by football players in the performance environment. With the algorithmic approach, the need for selecting and creating situations manually was minimized.
 The display, which rendered the view of a regular 11v11 football situation was shown from the perspective of a midfield player. The content was displayed using a head mounted augmented reality device. The participant was free to move on a real football field onto which the other players were augmented as virtual avatars. The participant was presented with a freeze-frame of the situation and triggered the situation by moving into a visually designated area. Participants were instructed verbally to offer support to the ball-carrying team-mate. The situation ended with the participant receiving a virtual pass from the team-mate.
 Experienced football players were drawn from a pool of university students to participate in the study. During the experiment, the movement trajectory of the participant was recorded using IMU data retrieved from the head-mounted display. The movement patterns were analyzed using a timeseries-clustering approach to examine the characteristics of different movement solutions the participants deployed across the situations (Chow et al., 2008). Deviations from the original trajectories (i.e., the open-source data) were measured to gaze at the natural variability that occurs in build-up situations in football.
 In conclusion, the feasibility study attempted to facilitate the assessment of decision-making skills in team-sports. Two promising means were introduced: (1) The selection of the situations was automated to reduce efforts needed by the researcher. (2) The use of an augmented-reality device enabled testing directly on the field. The decision-making skills were assessed through the observed movement patterns which provided new insights on how players move when making decisions.
 References
 Chow, J. Y., Davids, K., Button, C., & Rein, R. (2008). Dynamics of movement patterning in learning a discrete multiarticular action. Motor Control, 12(3), 219-240. https://doi.org/10.1123/mcj.12.3.219
 Kuhn, H. W. (1955). The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1‐2), 83-97. https://doi.org/10.1002/nav.3800020109
 Munkres, J. (1957). Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics, 5(1), 32-38.

KISS-ICP: In Defense of Point-to-Point ICP – Simple, Accurate, and Robust Registration If Done the Right Way

Robust and accurate pose estimation of a robotic platform, so-called sensor-based odometry, is an essential part of many robotic applications. While many sensor odometry systems made progress by adding more complexity to the ego-motion estimation process, we move in the opposite direction. By removing a majority of parts and focusing on the core elements, we obtain a surprisingly effective system that is simple to realize and can operate under various environmental conditions using different LiDAR sensors. Our odometry estimation approach relies on point-to-point ICP combined with adaptive thresholding for correspondence matching, a robust kernel, a simple but widely applicable motion compensation approach, and a point cloud subsampling strategy. This yields a system with only a few parameters that in most cases do not even have to be tuned to a specific LiDAR sensor. Our system performs on par with state-of-the-art methods under various operating conditions using different platforms using the same parameters: automotive platforms, UAV-based operation, vehicles like segways, or handheld LiDARs. We do not require integrating IMU data and solely rely on 3D point clouds obtained from a wide range of 3D LiDAR sensors, thus, enabling a broad spectrum of different applications and operating conditions. Our open-source system operates faster than the sensor frame rate in all presented datasets and is designed for real-world scenarios.

A GNSS/LiDAR/IMU Pose Estimation System Based on Collaborative Fusion of Factor Map and Filtering

One of the core issues of mobile measurement is the pose estimation of the carrier. The classic Global Navigation Satellite System/Inertial Measurement Unit (GNSS/IMU) integrated navigation scheme has high positioning accuracy but is vulnerable to Global Navigation Satellite System (GNSS) signal occlusion and multipath effect. Simultaneous Localization and Mapping (SLAM) is not affected by signal occlusion, but there are problems such as error accumulation and scene degradation. The multi-sensor fusion scheme combining the two technologies can effectively expand the scene coverage and improve the positioning accuracy and system robustness. However, the current scheme has some limitations. On the one hand, GNSS plays a less important role in most SLAM systems, only for initialization or as a closed-loop factor and other auxiliary work. On the other hand, in the fusion method, most of the current systems only use filtering or graph optimization, without taking into account the advantages of both. Aiming at pose estimation for mobile carriers, this paper combines the advantages of the global optimization of the factor graph and the local optimization of filtering and proposes a GNSS-IMU-LiDAR Constraint Kalman Filter (abbreviated as GIL-CKF), which has the characteristics of full coverage and effectively improving absolute accuracy and high output frequency. The scheme proposed in this paper consists of three parts. Firstly, an extensible factor map is used to fuse the positioning information from different sources, including GNSS, IMU, LiDAR, and a closed-loop map, to maintain a high-precision SLAM system, and the output is used as Multi-Sensor-Fusion-Odometry (MSFO). Then, the scene is divided according to the GNSS quality factor, and a Scene Optimizer (SO) is designed to filter GNSS pose and MSFO. Finally, the results are input into the Extended Kalman Filter (EKF) together with the original IMU data for further optimization and smoothing. The experimental results show that the integration of high-precision GNSS positioning information with IMU, LiDAR, a closed-loop map, and other information through the factor map can effectively suppress error accumulation and improve the overall accuracy of the SLAM system. The SO based on GNSS indicators can fully retain high-precision GNSS positioning information, effectively play their respective advantages of filtering and graph optimization, and alleviate the conflict between global and local optimization. SO with EKF filtering furthers optimization, can improve the output frequency, and smooth the trajectory. GIL-CKF can effectively improve the accuracy and robustness of pose estimation and has obvious advantages in multi-sensor scene complementarity, partial road section accuracy improvement, and high input frequency.

FusePose: IMU-Vision Sensor Fusion in Kinematic Space for Parametric Human Pose Estimation

There exist challenging problems in 3D human pose estimation mission, such as poor performance caused by occlusion and self-occlusion. Recently, IMU-vision sensor fusion is regarded as valuable for solving these problems. However, previous researches on the fusion of IMU and vision data, which is heterogeneous, fail to adequately utilize either IMU raw data or reliable high-level vision features. To facilitate a more efficient sensor fusion, in this work we propose a framework called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FusePose</i> under a parametric human kinematic model. Specifically, we aggregate different information of IMU or vision data and introduce three distinctive sensor fusion approaches: NaiveFuse, KineFuse and AdaDeepFuse. NaiveFuse servers as a basic approach that only fuses simplified IMU data and estimated 3D pose in euclidean space. While in kinematic space, KineFuse is able to integrate the calibrated and aligned IMU raw data with converted 3D pose parameters. AdaDeepFuse further develops this kinematical fusion process to an adaptive and end-to-end trainable manner. Comprehensive experiments with ablation studies demonstrate the rationality and superiority of the proposed framework. The performance of 3D human pose estimation is improved compared to the baseline result. On Total Capture dataset, KineFuse surpasses previous state-of-the-art which uses IMU only for testing by 8.6%. AdaDeepFuse surpasses state-of-the-art which uses IMU for both training and testing by 8.5%. Moreover, we validate the generalization capability of our framework through experiments on Human3.6M dataset.

HMM-based IMU data processing for arm gesture classification and motion tracking

HMM-based IMU data processing for arm gesture classification and motion tracking

HMM-based IMU data processing for arm gesture classification and motion tracking

HMM-based IMU data processing for arm gesture classification and motion tracking

Evaluation of Sources of Artificial Light at Night With an Autonomous Payload in a Sounding Balloon Flight

The presence of artificial light at night is not only limiting astronomical observations but has been linked to negative effects on human health and behavior of wildlife. New measurement systems are therefore needed to monitor artificial light emissions and their time evolution; MINLU autonomous payload has been designed and tested at University of Padova to provide complete aerial observations of artificial light sources over extended areas, with the capability to be integrated either on stratospheric balloons or drones. The implemented architecture is based on commercial components and is controlled by a Raspberry PI single board computer with the capability of uninterrupted operation up to 5 hours. The payload was successfully launched with a stratospheric sounding balloon on July 8th 2021 from Lajatico (Tuscany) and performed continuous analysis of emission sources up to the burst altitude of 34 km. The work will describe the calibration activity of the imaging unit which includes commercial cameras with dedicated filters used as luminance measuring device and raw spectrometer and present the elaboration of georeferenced images after reconstruction of the unit's inertial pointing along flight trajectory using combined GPS and IMU data integration.

Real-time sub-assembly identification through IMU data fusion with vision sensor for an inspection system

Real-time sub-assembly identification through IMU data fusion with vision sensor for an inspection system

Spatiotemporal Co-Attention Hybrid Neural Network for Pedestrian Localization Based on 6D IMU

In this paper, we propose spatiotemporal co-attention hybrid neural network (SC-HNN), a novel hybrid neural network model with both spatial and temporal attention mechanisms for pose-invariant inertial odometry. The main idea is to extract both local and global features from a window of IMU measurements for velocity prediction. SC-HNN leverages the convolutional neural network (CNN) to capture the sectional features and long short-term memory (LSTM) recurrent neural network (RNN) to extract the long-range dependencies. Attention mechanisms are designed and embedded in both CNN and LSTM modules for better model representation. Specifically, in the CNN attention block, the convolved features are refined along both channel and element dimensions. For the LSTM module, softmax scoring is applied to update the weights of the hidden states along the temporal axis. We evaluate SC-HNN on the benchmark with the largest and most natural IMU data, RoNIN. Extensive ablation experiments demonstrate the effectiveness of our SC-HNN model. Compared with the state of the art, the 50th percentile accuracy of SC-HNN is 18.21% higher and the 90th percentile accuracy is 21.15% higher for all the phone holders not appeared in the training set. The real scenario inertial tracking trials in the CUHK campus further prove the superior generalization ability of the SC-HNN model. Note to Practitioners—This paper aims at improving the localization accuracy of deep inertial odometry. We focus on the problem of indoor localization only from the low-cost IMU embedded in the smartphone without any restriction on the phone’s daily use. IMU is a perfect solution for indoor localization because of its low power consumption, high privacy protection, and external infrastructure free. This paper suggests a novel hybrid convolutional and recurrent neural network with a set of carefully designed attention mechanisms to improve the representation ability of deep inertial odometry model. Specifically, the convolutional layer is applied to extract the local spatial features among the 6D IMU signals, following a cascaded channel attention module and element attention module to boost the representation ability of CNN. The complex long-term dependencies are then identified by the LSTM layers. To adaptively capture the temporal features of the multimodal inertial signals, an attention mechanism is applied to weigh the hidden states for the generation of the final features. The effectiveness of the SC-HNN design is validated by extensive ablation studies. To the best of our knowledge, our model is the first HNN fused attention mechanism for inertial tracking. Extensive experiments show that the proposed method outperforms the state of the art.

A Bio-Inspired Simultaneous Surface and Underwater Risk Assessment Method Based on Stereo Vision for USVs in Nearshore Clean Waters

Ensuring safety of unmanned surface vehicles (USVs) during nearshore field operations is challenging as they face unknown risks of collision with surface and underwater obstacles. To address this problem, we propose a bio-inspired collision risk-assessment method for ensuring safe USVs operation in nearshore clean waters. Collision risks are typically unknown due to the imprecision of underwater measurements for the water-air boundary light refraction, and disturbances caused by wave-induced USV motion. To tackle these two causes of imprecision, our collision risk-assessment method employs one single stereo camera and an IMU. We use visual information from the stereo camera to model the light rays at the water-air interface, rectify the underwater measurements, and monitor for surface obstacles, and IMU data to model wave disturbances via time-frequency analysis. We verify the performance of this new method in both the indoor and outdoor experiments, which demonstrates that it achieves a 65.44% improvement in the depth accuracy at the water-air interface compared with raw depth measurements, and provides a range of safety levels for guiding USV operations. This new collision risk-assessment method significantly expands the USVs working areas while ensuring safety during field exploration. Additionally, the simplicity of the sensor setup and thus minimal cost of this method underscore its utility for supporting USV-based various explorations in clear waters.

Real-time sub-assembly identification through IMU data fusion with vision sensor for an inspection system

Real-time sub-assembly identification through IMU data fusion with vision sensor for an inspection system

Improved LiDAR Localization Method for Mobile Robots Based on Multi-Sensing

In this paper, we propose a localization method applicable to 3D LiDAR by improving the LiDAR localization algorithm, such as AMCL (Adaptive Monte Carlo Localization). The method utilizes multiple sensing information, including 3D LiDAR, IMU and the odometer, and can be used without GNSS. Firstly, the wheel speed odometer and IMU data of the mobile robot are multi-source fused by EKF (Extended Kalman Filter), and the sensor data obtained after multi-source fusion are used as the motion model to participate in the positional prediction of the particle set in AMCL to obtain the initial positioning information of the mobile robot. Then, the position pose difference values output by AMCL at adjacent moments are substituted into the PL-ICP algorithm as the initial position pose transformation matrix, and the 3D laser point cloud is aligned with the nonlinear system using the PL-ICP algorithm. The three-dimensional laser odometer is obtained by LM (Levenberg--Marquard) iterative solution in the PL-ICP algorithm. Finally, the initial position pose output by AMCL is corrected by the three-dimensional laser odometer, and the AMCL particles are weighted and sampled to output the final positioning result of the mobile robot. Through simulation and practical experiments, it is verified that the improved AMCL algorithm has higher positioning accuracy and stability compared to the AMCL algorithm.

An improved semi-synthetic approach for creating visual-inertial odometry datasets

Capturing outdoor visual-inertial datasets is a challenging yet vital aspect of developing robust visual-inertial odometry (VIO) algorithms. A significant hurdle is that high-accuracy-ground-truth systems (e.g., motion capture) are not practical for outdoor use. One solution is to use a “semi-synthetic” approach that combines rendered images with real IMU data. This approach can produce sequences containing challenging imagery and accurate ground truth but with less simulated data than a fully synthetic sequence. Existing methods (used by popular tools/datasets) record IMU measurements from a visual-inertial system while measuring its trajectory using motion capture, then rendering images along that trajectory. This work identifies a major flaw in that approach, specifically that using motion capture alone to estimate the pose of the robot/system results in the generation of inconsistent visual-inertial data that is not suitable for evaluating VIO algorithms. However, we show that it is possible to generate high-quality semi-synthetic data for VIO algorithm evaluation. We do so using an open-source full-batch optimization tool to incorporate both mocap and IMU measurements when estimating the IMU’s trajectory. We demonstrate that this improved trajectory results in better consistency between the IMU data and rendered images and that the resulting data improves VIO trajectory error by 79% compared to existing methods. Furthermore, we examine the effect of visual-inertial data inconsistency (as a result of trajectory noise) on VIO performance to provide a foundation for future work targeting real-time applications.

Modeling and Prediction of User Stability and Comfortability on Autonomous Wheelchairs With 3-D Mapping

Traditional manual wheelchairs have a fixed seat with no movement or angle adjustment, which can seriously affect the user's comfort and greatly limit user experience. However, the electric wheelchair relies on strong intelligence and automatic features; it can not only realize the multidegree freedom adjustment of the human body and the seat but also has a rich and powerful man–machine control interface, which greatly facilitates and improves the user experience. This study upgraded a Permobil C400-powered wheelchair with multisensor data fusion technology to enrich its terrain recognition, tipping stability, and comfortability prediction. The tipping stability modeling of the wheelchair dummy system is carried out using multibody dynamics and vibration mechanics to obtain the tipping stability limit and the comfort evaluation of the wheelchair vibration acceleration on the human body during travel. Based on the elevation mapping method, the wheelchair can estimate the terrain from the local point of view at any point in time. At the same time, the RGB-D depth camera is connected to the robot operating system (ROS) system, and the open-source algorithm package RTAB-MAP is used to complete the MAP construction and collect the 3-D point-cloud terrain data. Then, the real 3-D terrain files are generated through the point-cloud stitching technology for stability simulation of the wheelchair–human system. The tipping stability and comfort indexes of the wheelchair–human system when passing over different physical terrains can be obtained. The experimental results show that the IMU data located on the human chest agree well with the simulation analysis data and are suitable for a variety of complex real-terrain conditions, verifying the accuracy of the wheelchair–human system dynamics model and the feasibility of the simulation analysis process. Thus, this modeling and simulation method can predict wheelchair stability and user comfortability well and ensure a high-performance experience.

Gait Trajectory Prediction on an Embedded Microcontroller Using Deep Learning.

Achieving a normal gait trajectory for an amputee's active prosthesis is challenging due to its kinematic complexity. Accordingly, lower limb gait trajectory kinematics and gait phase segmentation are essential parameters in controlling an active prosthesis. Recently, the most practiced algorithm in gait trajectory generation is the neural network. Deploying such a complex Artificial Neural Network (ANN) algorithm on an embedded system requires performing the calculations on an external computational device; however, this approach lacks mobility and reliability. In this paper, more simple and reliable ANNs are investigated to be deployed on a single low-cost Microcontroller (MC) and hence provide system mobility. Two neural network configurations were studied: Multi-Layered Perceptron (MLP) and Convolutional Neural Network (CNN); the models were trained on shank and foot IMU data. The data were collected from four subjects and tested on a fifth to predict the trajectory of 200 ms ahead. The prediction was made for two cases: with and without providing the current phase of the gait. Then, the models were deployed on a low-cost microcontroller (ESP32). It was found that with fewer data (excluding the current gait phase), CNN achieved a better correlation coefficient of 0.973 when compared to 0.945 for MLP; when including the current phase, both network configurations achieved better correlation coefficients of nearly 0.98. However, when comparing the execution time required for the prediction on the intended MC, MLP was much faster than CNN, with an execution time of 2.4 ms and 142 ms, respectively. In summary, it was found that when training data are scarce, CNN is more efficient within the acceptable execution time, while MLP achieves relative accuracy with low execution time with enough data.

Contrast Maximization-Based Feature Tracking for Visual Odometry with an Event Camera

As a new type of vision sensor, the dynamic and active-pixel vision sensor (DAVIS) outputs image intensity and asynchronous event streams in the same pixel array. We present a novel visual odometry algorithm based on the DAVIS in this paper. The Harris detector and the Canny detector are utilized to extract an initialized tracking template from the image sequence. The spatio-temporal window is selected by determining the life cycle of the asynchronous event streams. The alignment on timestamps is achieved by tracking the motion relationship between the template and events within the window. A contrast maximization algorithm is adopted for the estimation of the optical flow. The IMU data are used to calibrate the position of the templates during the update process that is exploited to estimate camera trajectories via the ICP algorithm. In the end, the proposed visual odometry algorithm is evaluated in several public object tracking scenarios and compared with several other algorithms. The tracking results show that our visual odometry algorithm can achieve better accuracy and lower latency tracking trajectory than other methods.

An ISB-consistent upper limb model based on the Denavit-Hartenberg convention for joint angle estimation during prolonged rehabilitation exercises

An accurate and robust joint kinematics description is a crucial prerequisite for the development of tele- rehabilitation applications. Miniaturized wearable sensors (IMUs), along with the use of efficient sensor fusion algorithms, represent a suitable solution. However, orientation drift can affect real-time joint kinematics over prolonged acquisition. In this work, we proposed an optimization kinematics framework which exploits a Denavit-Hartenberg (DH) model of the upper limbs, compliant with the guidelines of the International Society of Biomechanics. The accuracy of the estimated 3D shoulder and elbow kinematics was evaluated using data recorded by two IMUs attached to a robotic arm during a 20- minute planar exercise and compared with the errors obtained using a model-free approach. The upper limb was modelled as a three-segment chain including trunk, upper arm (UA), and the forearm (FA) (Fig. 1). Shoulder and elbow joints were modelled with three (φ1, φ2, φ3) and two degrees of freedom (φ4, φ6), respectively. The carrying angle (φ5) was introduced as a fixed subject-specific parameter to describe the physiological abduction of the FA with respect to the UA. The joint angles were computed in an optimization process which minimizes at each time step the difference between the orientation predicted using the DH model and the corresponding orientation estimated by a sensor fusion algorithm without the magnetometer [1]. In addition, joint angle constraints were set based on the maximum angular velocity, the physiological angular limits, and the specific range of movement when known a-priori . To validate the methods, two IMUs (Xsens – MTw) were attached to the UA and FA of a robotic arm (Kinova – Jaco2) programmed to mimic a shoulder flexion-extension (φ1, φ2, φ3), an elbow flexion-extension (φ4), and a forearm prono-supination (φ6), simultaneously, for 20 minutes (~150 cycles) [3]. The φ5 was equal to zero for the robot. IMU and reference data were collected at 100 Hz. The gyroscope offset was computed during a preliminary acquisition and then subtracted from the measurements. The accuracy was evaluated in terms of root mean square difference (RMSD) between the model-based optimized and robot reference joint angles. Furthermore, the RMSD corresponding to the Euler inversion of the FA-to-UA relative orientation (model-free) was also computed. The RMSD (deg) for the joint angles obtained with the optimization (model-free) process amounted to 1 (10.2), 0.4 (0.2), 0.9 (1.4), 3.0 (3.1), 0 (8.2), and 1 (6) for φ1, φ2, φ3, φ4, φ5, φ6, respectively. The proposed methods allowed to reduce joint angles errors of almost a factor 10 over long period compared to model-free approach (1.1 vs 8.9 deg on average). These results if further confirmed on human experiments can contribute to the design of tele-rehabilitation applications relying on the accurate upper limb joint kinematics estimates. Funded by Sardegna Ricerche (POR FESR 2014/2020).

Ctrl-VIO: Continuous-Time Visual-Inertial Odometry for Rolling Shutter Cameras

In this letter, we propose a probabilistic continuous-time visual-inertial odometry (VIO) for rolling shutter cameras. The continuous-time trajectory formulation naturally facilitates the fusion of asynchronized high-frequency IMU data and motion-distorted rolling shutter images. To prevent intractable computation load, the proposed VIO is sliding-window and keyframe-based. We propose to probabilistically marginalize the control points to keep the constant number of keyframes in the sliding window. Furthermore, the line exposure time difference (line delay) of the rolling shutter camera can be online calibrated in our continuous-time VIO. To extensively examine the performance of our continuous-time VIO, experiments are conducted on publicly-available WHU-RSVI, TUM-RSVI, and SenseTime-RSVI rolling shutter datasets. The results demonstrate the proposed continuous-time VIO significantly outperforms the existing state-of-the-art VIO methods.