Onboard Inertial Measurement Unit Research Articles

Terrain Classification Based on Sensed Leg Compliance for Amphibious Crab Robot

Characterization of terrain in real time allows autonomous legged robots to modify their gait to better suit their environment and recognize hazardous conditions. We present a novel approach for terrain classification for legged robots with passively compliant components, in which terrain information is gathered by measuring component deflection during operation by means of low cost Hall effect magnetometers and embedded magnets. The effectiveness of this approach is demonstrated on a hexapod robot designed to operate in the surf zones of beaches. Datasets of sensor measurements corresponding to three types of granular terrain are collected in both a laboratory and outdoor beach environment, and are used to train support vector machine (SVM) classifiers. Results show that, using time domain features of measurements from each leg, a mean accuracy of 99.3% and 92.4% through 10-fold cross-validation can be achieved for the laboratory and beach datasets, respectively. For the beach dataset, this accuracy is found to be slightly greater than for a classifier trained on measurements from an on-board inertial measurement unit (IMU) (91.6%), while an accuracy of 95.1% can be achieved by combining the information from both sensor modalities as inputs to a single classifier. The effect of the Earth’s magnetic field on the measurements and considerations for data collection are also discussed.

An application of metadata-based image retrieval system for facility management

For facility management, photography is an efficient and accurate method of recording the physical state of infrastructure. However, without an effective organizational scheme, the difficulty of retrieving relevant photos from historical databases can become overly burdensome for highly complex or long-lived assets. To make strategic decisions, it is crucial to retrieve the right information from a plurality of sources in a timely manner. The main objective of this paper is to present a method for organizing and retrieving photos from massive facility management photo databases using photo-metadata: photographed location, camera perspective, and image semantic content information. Indoor localization experiments were performed using Bluetooth technology to infer the location information. Perspective is inferred from the device’s on-board inertial measurement unit (IMU). Image semantic content is inferred using a Convolutional Neural Network (CNN)-based deep learning algorithm. Fusing these three features, seven query options were provided for the user when retrieving images. Leveraging Building Information Modeling (BIM) as a process and Geographic Information Systems (GIS) as a framework, this paper also envisions a federated information management by connecting 2D and 3D facility assets with our real-world map which can be smoothly bridged with our image retrieval system. The realization of the integrated application with BIM and GIS is significantly beneficial for the facility management domain by advancing the understanding of projects in a broader view with a federated data platform. In this research, the framework is illustrated with 21 institutional buildings within the University of Texas at Austin’s main campus, and the authors conclude that the proposed metadata-based image retrieval system can ultimately enhance the better-informed decision-making process through rapid information retrieval.

Validation of model-based correction for non-Stokesian tracers

In-situ flow-tracking measurements at scales on the order of 10 m3 and larger remain a challenge. The large size of the tracers required for optical visibility results in an inertial lag and inherently low seeding density. For instance, natural snowfall, fake snow and soap bubbles on the order of 2 cm have been used as tracers for field measurements and extracted statistical quantities (Nemes et al., 2017; Wei et al., 2021; Rosi et al., 2014). There is also growing interest in networks of sensors for remote- measurement where optical access is impossible (Bolt et al. 2020; Villa et al. 2016). Onboard inertial measurement units (IMU) are a promising tool for high-resolution measurements over large spatial domains without optical access. However, due to the intrinsic lag, a dynamic-model-based correction is required for the tracking of transient phenomena, sketched in figure 1. In the present study, the tracer-velocity correction is evaluated by quantifying the residual error in measured flow velocity after the method of Galler et al. (2021) is applied.

A probabilistic approach to online classification of bicycle crashes

When a bicycle crash takes place, it is paramount for an emergency center to recognize the physical state of the cyclist as early as possible. However, an injured bicyclist may be incapable of making a phone call to the emergency center. In this study, we propose an online approach to classify bicycle crashes based on signals from an onboard inertial measurement unit (IMU), which can be used as a trigger function for an automatic emergency system. For this purpose, we define several bicycle crash features according to the kinematic properties of bicycle accidents. The input signals (variables) influencing the individual crash features are determined by the ANOVA method (analysis of variance). With the determined input signals, probabilistic models for each crash feature are trained on the base of logit models and 20,000 km naturalistic driving data including 20 real crashes. In addition, further crash and corner case data has been collected for the model training. A decision tree describing all probabilistic crash features is used to classify different kinematic events and crash scenarios. A series of driving tests with a crash-dummy and a crash-car are performed to verify the proposed crash classification approach, showing a sensitivity of 96.8%, a specificity of 99.6% and an accuracy of 99.5% of the trained model.

Experiments and Analysis for Measuring Mechanical Motion with Event Cameras

Event cameras—which detect pixelized, spatiotemporal changes asynchronously as opposed to frame-based triggered cameras—are commercially available and the topic of considerable research recently for computer vision and robotics applications. We hypothesize that these novel devices can be of utility in understanding mechanical motion by performing functions similar to high speed cameras, but with reduced bandwidth and simpler processing. To test this hypothesis, we applied multiple measurement modalities to a simple high-speed mechanical tool, including accelerometer, acoustic, and event-camera images. We also explore the utility of onboard inertial measurement units which are often integrated into these devices. Our analysis shows that these measurements are possible and consistent across modalities, possibly leading to novel new architectures for image and sensor based measurements.

Viewpoint Selection for Rover Relative Pose Estimation Driven by Minimal Uncertainty Criteria

Pose estimation is critical for mobile robots to fulfill various tasks, such as path following or mapping the environment. This is usually accomplished by simultaneous localization and mapping (SLAM). However, computationally constrained systems, such as planetary rovers, rely on less intensive guidance navigation and control (GNC) solutions generally derived solely from visual odometry (VO), wheel odometry, and the onboard inertial measurement unit. Although providing adequate localization performances, the drift accumulated overtime is not compensated by loop closing capabilities, typical of SLAM. Usually, rovers send surface images to the ground station, and these images are used for multiple purposes, such as scientific and operational planning. The number of images is constrained by the communication bandwidth and power budget. The set of transmitted images can be used as a means to correct the robot’s trajectory in an off-line manner. In this work, a solution is presented to the problem of selecting the optimal set of viewpoints belonging to the planned path from which to capture and transmit images: 1) it guarantees accurate trajectory correction and 2) complies with the maximum number of images that can be transmitted to ground control given the available data budget. To this end, it is proposed: 1) a delocalized/decentralized sensor fusion approach based on pose graph optimization and structure from motion and 2) a strategy to select a minimal set of viewpoints along the trajectory that, given a tentative geometry of the environment and the global path that the rover must follow, minimizes the uncertainty of all the robot poses. Optimal camera viewpoint positions are selected as a function of the planned trajectory, the approximate scene geometry, and the maximum transmittable number of images. The proposed method has been tested on a dataset of stereo-images collected in a representative Martian environment, the ALTEC Mars Terrain Simulator (MTS), with the ExoMars testing rover (ExoTeR—European Space Agency, Paris, France, property). Rover stereo-images ground truth was given with millimetric accuracy by a motion capture (MC) system.

Sensitivity Analysis and Flight Tests Results for a Vertical Cold Launch Missile System

In vertical cold launch the missile starts without the function of the main engine. Over the launcher, the attitude of the missile is controlled by a set of lateral thrusters. However, a quick turn might be disturbed by various uncertainties. This study discusses the problem of the influences of disturbances and the repeatability of lateral thrusters’ ignition on the pitch maneuver quality. The generic 152.4 mm projectile equipped in small, solid propellant lateral thrusters was used as a test platform. A six degree of freedom mathematical model was developed to execute the Monte-Carlo simulations of the launch phase and to prepare the flight test campaign. The parametric analysis was performed to investigate the influence of system uncertainties on quick turn repeatability. A series of ground laboratory trials was accomplished. Thirteen flight tests were completed on the missile test range. The flight parameters were measured using an onboard inertial measurement unit and a ground vision system. It was experimentally proved that the cold vertical launch maneuver could be realized properly with at least two lateral motors. It was found that the initial roll rate of the projectile and the lateral thrusters ‘igniters’ uncertainties could affect the pitch angle achieved and must be minimized to reduce the projectile dispersion.

Motion Reveal Emotions: Identifying Emotions From Human Walk Using Chest Mounted Smartphone

Emotion recognition via gait analysis is an active and key area of research because of its significant academic and commercial potential. With recent developments in hardware technology, the use of inertial sensors allows researchers to effectively capture the human motion data for gait analysis. To this end, the aim of this paper is to identify emotions from the inertial signals of human gait recorded by means of body mounted smartphones. We extracted a manually-crafted set of features computed from the human inertial gait data which are used to train and precisely predict the human emotions. Specifically, we collected the inertial gait data of 40 volunteers by means of smartphone’s on-board inertial measurement units (3D accelerometer, 3D gyroscope) attached at the chest in six basic emotions including sad, happy, anger, surprise, disgust and fear . Using stride based segmentation, the raw signals are first decomposed into individual strides. For each stride, a set of 296 spectro-temporal features are computed, which are fed into two supervised learning predictors namely Support Vector Machines and Random Forest. The classification results obtained with the proposed methodology and validated with k-fold validation procedure show classification accuracy of 95% for binary emotions and 86% for all six categories of emotions.

Dimensionless Model-Based System Tracking Via Augmented Kalman Filter for Multiscale Unmanned Ground Vehicles

In recent years, many unmanned vehicle designs and autonomous driving functions have been introduced in the automotive industry to increase the safety and versatility of multiple vehicle designs. It is critical to select a plant vehicle model and a compact uncertainty representation regardless of the vehicle scale for vast deployment. This article introduces a dimensionless representation of a dynamic vehicle model that is suitable for generalized dynamic analysis. Demonstrations included in this article showed that the compact uncertainty bounds of the dimensionless parameters could be used to generate an adaptive observer suitable for full-sized and corresponding scaled vehicles. The proposed dimensionless model-based observer can assess motion states, mass, center of gravity displacement, and the tire stiffness online with common onboard inertial measurement unit (IMU). The effectiveness and sensitivity of the proposed technique are highlighted through simulations and experiments on scaled test vehicles.

A Real Time Localization System for Vehicles Using Terrain-Based Time Series Subsequence Matching

Global Navigation Satellite Systems (GNSSs) are commonly used for positioning vehicles in open areas. Yet a GNSS frequently encounters loss of lock in urban areas. This paper presents a new real-time localization system using measurements from vehicle odometer data and data from an onboard inertial measurement unit (IMU), in the case of lacking GNSS information. A Dead Reckoning model integrates odometer data, IMU angular and velocity data to estimate the rough position of the vehicle. We then use an R-Tree structured reference road map of pitch data to boost spatial search efficiency. An optimized time series subsequence matching method matches the measured pitch data and the stored pitch data in reference road map for more accurate position estimation. The two estimated positions are fused using an extended Kalman filter model for final localization. The proposed localization system was tested for computational complexity with a median runtime of 12 ms, and for positioning accuracy with a median position error of 0.3 m.

Hyper resolution image mosaics with unbounded vertical field of view

This paper presents the framework for the creation of hyper resolution image mosaics (HRIM) with a full horizontal and unbounded vertical field of view. Additionally, the hyper-resolution images are mapped to a format compatible with a geo referencing frame. In this manner the images can be viewed using standard mapping services. The measurement device consists of eight cameras covering the full 360 degree horizontal field. The unlimited vertical field of view is obtained by moving the device, e.g., by sinking it down a mine shaft. The cameras are synchronized so that with each frame the full horizontal field of view is captured. The camera sensors are calibrated to an onboard inertial measurement unit (IMU) which allows the reconstruction of the position of each sensor in a world coordinate frame for each captured image using rigid transformations. Perspective projection onto an open manifold (cylinder) allows for the integration of all images into an HRIM. Constrained tensor-polynomial approximations are used to obtain geometric maps, i.e., a distortion lattice, from the image coordinate system onto an open cylindrical manifold. From this lattice binary masks are derived for each projected image. A multi band blending algorithm is implemented for the final composition. The HRIM are then tiled and stored on a GIS mapping server from where the HRIM can be viewed on any device running an HTML browser. It should be noted that this approach provides a significant advantage since viewing the large HRIM is not computationally intensive and can even be done on tablets. The framework has been applied to the remote inspection of deep vertical shafts.

Measurement Characterization and Autonomous Outlier Detection and Exclusion for Ground Vehicle Navigation With Cellular Signals

An autonomous measurement outlier detection and exclusion framework for ground vehicle navigation using cellular signals is developed. The ground vehicle aids its onboard inertial measurement unit (IMU) with cellular signals in a tightly-coupled fashion in the absence of global navigation satellite system (GNSS) signals. First, cellular pseudoranges are characterized from an extensive wardriving campaign collected with a ground vehicle in different environments: open sky, urban, and deep urban. Then, a framework is developed, which accounts for outliers due to line-of-sight blockage and short multipath delays. These outliers induce biases in cellular pseudoranges and compromise the integrity of the navigation solution. Experimental results are presented evaluating the efficacy of the proposed framework on a ground vehicle navigating in the absence of GNSS signals. The results demonstrate the proposed framework detecting and excluding outliers, reducing the position root mean squared error (RMSE) by 41.5% and the maximum position error by 43.1%.

Multi-Sensor Orientation Tracking for a Façade-Cleaning Robot.

Glass-façade-cleaning robots are an emerging class of service robots. This kind of cleaning robot is designed to operate on vertical surfaces, for which tracking the position and orientation becomes more challenging. In this article, we have presented a glass-façade-cleaning robot, Mantis v2, who can shift from one window panel to another like any other in the market. Due to the complexity of the panel shifting, we proposed and evaluated different methods for estimating its orientation using different kinds of sensors working together on the Robot Operating System (ROS). For this application, we used an onboard Inertial Measurement Unit (IMU), wheel encoders, a beacon-based system, Time-of-Flight (ToF) range sensors, and an external vision sensor (camera) for angular position estimation of the Mantis v2 robot. The external camera is used to monitor the robot’s operation and to track the coordinates of two colored markers attached along the longitudinal axis of the robot to estimate its orientation angle. ToF lidar sensors are attached on both sides of the robot to detect the window frame. ToF sensors are used for calculating the distance to the window frame; differences between beam readings are used to calculate the orientation angle of the robot. Differential drive wheel encoder data are used to estimate the robot’s heading angle on a 2D façade surface. An integrated heading angle estimation is also provided by using simple fusion techniques, i.e., a complementary filter (CF) and 1D Kalman filter (KF) utilizing the IMU sensor’s raw data. The heading angle information provided by different sensory systems is then evaluated in static and dynamic tests against an off-the-shelf attitude and heading reference system (AHRS). It is observed that ToF sensors work effectively from 0 to 30 degrees, beacons have a delay up to five seconds, and the odometry error increases according to the navigation distance due to slippage and/or sliding on the glass. Among all tested orientation sensors and methods, the vision sensor scheme proved to be better, with an orientation angle error of less than 0.8 degrees for this application. The experimental results demonstrate the efficacy of our proposed techniques in this orientation tracking, which has never applied in this specific application of cleaning robots.

GaitLock: Protect Virtual and Augmented Reality Headsets Using Gait

With the fast penetration of commercial Virtual Reality (VR) and Augmented Reality (AR) systems into our daily life, the security issues of those devices have attracted significant interests from both academia and industry. Modern VR/AR systems typically use head-mounted devices (i.e., headsets) to interact with users, and often store private user data, e.g., social network accounts, online transactions or even payment information. This poses significant security threats, since in practice the headset can be potentially obtained and accessed by unauthenticated parties, e.g., identity thieves, and thus cause catastrophic breach. In this paper, we propose a novel GaitLock system, which can reliably authenticate users using their gait signatures. Our system doesn't require extra hardware, e.g., fingerprint sensors or retina scanners, but only uses the on-board inertial measurement units (IMUs) equipped in almost all mainstream VR/AR headsets to authenticate the legitimate users from intruders, by simply asking them to walk a few steps. To achieve that, we propose a new gait recognition model Dynamic-SRC, which combines the strength of Dynamic Time Warping (DTW) and Sparse Representation Classifier (SRC), to extract unique gait patterns from the inertial signals during walking. We implement GaitLock on Google Glass (a typical AR headset), and extensive experiments show that GaitLock outperforms the state-of-the-art systems significantly in recognition accuracy ($>$>98 percent success in 5 steps), and is able to run in-situ on the resource-constrained VR/AR headsets without incurring high energy cost.

Steering Angle Assisted Vehicular Navigation Using Portable Devices in GNSS-Denied Environments.

Recently, land vehicle navigation, and especially by the use of low-cost sensors, has been the object of a huge level of research interest. Consumer Portable Devices (CPDs) such as tablets and smartphones are being widely used by many consumers all over the world. CPDs contain sensors (accelerometers, gyroscopes, magnetometer, etc.) that can be used for many land vehicle applications such as navigation. This paper presents a novel approach for estimating steering wheel angles using CPD accelerometers by attaching CPDs to the steering wheel. The land vehicle change of heading is then computed from the estimated steering wheel angle. The calculated change of heading is used to update the navigation filter to aid the onboard Inertial Measurement Unit (IMU) through the use of an Extended Kalman Filter (EKF) in GNSS-denied environments. Four main factors that may affect the steering wheel angle accuracy are considered and modeled during steering angle estimations: static onboard IMU leveling, inclination angle of the steering wheel, vehicle acceleration, and vehicle inclination. In addition, these factors are assessed for their effects on the final result. Therefore, three methods are proposed for steering angle estimation: non-compensated, partially-compensated, and fully-compensated methods. A road experimental test was carried out using a Pixhawk (PX4) navigation system, iPad Air, and the OBD-II interface. The average Root Mean Square Error (RMSE) of the change of heading estimated by the proposed method was 0.033 rad/s. A navigation solution was estimated while changes of heading and forward velocity updates were used to aid the IMU during different GNSS signal outages. The estimated navigation solution is enhanced when applying the proposed updates to the navigation filter by 91% and 97% for 60 s and 120 s of GNSS signal outage, respectively, compared to the IMU standalone solution.

Relative Heading Estimation and Its Application in Target Handoff in GPS-Denied Environments

In this paper, we address the problem of target handoff between two fixed wing aircrafts that do not have access to Global Positioning Systems. The problem requires estimating the relative pose between the vehicles. We assume that onboard inertial measurement units can provide roll and pitch estimates of the aircraft attitude. We examine the observability of other relative states needed to do the handoff problem. In particular, we consider two different scenarios. In the first scenario, we assume that the relative position between the aircrafts is measured, as would be the case for a radar or a lidar sensor. We assume that the two aircrafts do not exchange their airspeed and turn rate information. Under mild assumptions, we show that the relative heading between the two aircrafts is observable. In the second scenario, we assume that only the bearing angle between the two aircrafts is measured, as would be the case for a vision sensor. We prove state observability for maneuvers that are relevant to target tracking and handoff. We also present an estimation algorithm that uses a bank of extended Kalman filters to estimate the relative states. Simulation results with full vehicle dynamics demonstrate the feasibility of the proposed approach.

Multi-parameter stochastic inversion for first and second moment mass properties of a model-scale ship with topside ice accumulation

This work investigates the indirect monitoring of ice accretion on ship surfaces using a stochastic inversion framework. An accurate assessment of a ship's mass properties during operation is an important concern for ships traveling within the Arctic, where ice accumulation is a concern. Within such contexts, the actual (i.e., current) first and second moment properties of the vessel, including accumulated topside icing, must be considered within the associated equations of motion for a given ship. By leveraging instrumentation such as an existing on-board inertial measurement unit (IMU), in conjunction with existing seakeeping software, the proposed framework recovers a statistical description of two mass properties, simultaneously. The subsequent stochastic inverse problem is solved using the proposed framework in arriving at two mass properties in particular: the vertical center of gravity (a first moment mass property) and the roll gyradius (a second moment mass property).The inversion scheme requires two main inputs: an observed ground truth for the roll period, and an associated signal-to-noise ratio for the roll period measurement. The framework applies a Markov chain Monte Carlo (MCMC) inversion scheme, implemented in Python, that leverages the Standard Ship Motion Program (SMP95) software in order to build a posterior distribution for the desired mass properties. Experimental model results for Research Vessel (R/V) Melville are used to validate the proposed framework. Six different configurations, including one case of no icing and five cases of topside icing, are investigated within the context of this framework in order to invert for their respective roll gyradii and vertical centers of gravity. Icing configurations include both asymmetric and symmetric ice accumulation under moderate to heavy icing conditions. Recommendations concerning strategies for applying the proposed framework are offered.

An Augmented Reality Geo-Registration Method for Ground Target Localization from a Low-Cost UAV Platform.

This paper presents an augmented reality-based method for geo-registering videos from low-cost multi-rotor Unmanned Aerial Vehicles (UAVs). The goal of the proposed method is to conduct an accurate geo-registration and target localization on a UAV video stream. The geo-registration of video stream requires accurate attitude data. However, the Inertial Measurement Unit (IMU) sensors on most low-cost UAVs are not capable of being directly used for geo-registering the video. The magnetic compasses on UAVs are more vulnerable to the interferences in the working environment than the accelerometers. Thus the camera yaw error is the main sources of the registration error. In this research, to enhance the low accuracy attitude data from the onboard IMU, an extended Kalman Filter (EKF) model is used to merge Real Time Kinematic Global Positioning System (RTK GPS) data with the IMU data. In the merge process, the high accuracy RTK GPS data can be used to promote the accuracy and stability of the 3-axis body attitude data. A method of target localization based on the geo-registration model is proposed to determine the coordinates of the ground targets in the video. The proposed method uses a modified extended Kalman Filter to combine the data from RTK GPS and the IMU to improve the accuracy of the geo-registration and the localization result of the ground targets. The localization results are compared to the reference point coordinates from satellite image. The comparison indicates that the proposed method can provide practical geo-registration and target localization results.

Stereo obstacle detection for unmanned surface vehicles by IMU-assisted semantic segmentation

A new obstacle detection algorithm for unmanned surface vehicles (USVs) is presented. A state-of-the-art graphical model for semantic segmentation is extended to incorporate boat pitch and roll measurements from the on-board inertial measurement unit (IMU), and a stereo verification algorithm that consolidates tentative detections obtained from the segmentation is proposed. The IMU readings are used to estimate the location of horizon line in the image, which automatically adjusts the priors in the probabilistic semantic segmentation model. We derive the equations for projecting the horizon into images, propose an efficient optimization algorithm for the extended graphical model, and offer a practical IMU–camera–USV calibration procedure. Using an USV equipped with multiple synchronized sensors, we captured a new challenging multi-modal dataset, and annotated its images with water edge and obstacles. Experimental results show that the proposed algorithm significantly outperforms the state of the art, with nearly 30% improvement in water-edge detection accuracy, an over 21% reduction of false positive rate, an almost 60% reduction of false negative rate, and an over 65% increase of true positive rate, while its Matlab implementation runs in real-time.

External Wrench Estimation, Collision Detection, and Reflex Reaction for Flying Robots

Flying in unknown environments may lead to unforeseen collisions, which may cause serious damage to the robot and/or its environment. In this context, fast and robust collision detection combined with safe reaction is, therefore, essential and may be achieved using external wrench information. Also, deliberate physical interaction requires a control loop designed for such a purpose and may require knowledge of the contact wrench. In principle, the external wrench may be measured or estimated. Whereas measurement poses large demands on sensor equipment, additional weight, and overall system robustness, in this paper we present a novel model-based method for external wrench estimation in flying robots. The algorithm is based on the onboard inertial measurement unit and the robot's dynamics model only. We design admittance and impedance controllers that use this estimate for sensitive and robust physical interaction. Furthermore, the performance of several collision detection and reaction schemes is investigated in order to ensure collision safety. The identified collision location and associated normal vector located on the robot's convex hull may then be used for sensorless tactile sensing. Finally, a low-level collision reflex layer is provided for flying robots when obstacle avoidance fails, also under wind influence. Our experimental and simulation results show evidence that the methodologies are easily implemented and effective in practice.