Sensor Fusion Algorithm Research Articles

How Magnetic Disturbance Influences the Attitude and Heading in Magnetic and Inertial Sensor-Based Orientation Estimation.

With the advancements in micro-electromechanical systems (MEMS) technologies, magnetic and inertial sensors are becoming more and more accurate, lightweight, smaller in size as well as low-cost, which in turn boosts their applications in human movement analysis. However, challenges still exist in the field of sensor orientation estimation, where magnetic disturbance represents one of the obstacles limiting their practical application. The objective of this paper is to systematically analyze exactly how magnetic disturbances affects the attitude and heading estimation for a magnetic and inertial sensor. First, we reviewed four major components dealing with magnetic disturbance, namely decoupling attitude estimation from magnetic reading, gyro bias estimation, adaptive strategies of compensating magnetic disturbance and sensor fusion algorithms. We review and analyze the features of existing methods of each component. Second, to understand each component in magnetic disturbance rejection, four representative sensor fusion methods were implemented, including gradient descent algorithms, improved explicit complementary filter, dual-linear Kalman filter and extended Kalman filter. Finally, a new standardized testing procedure has been developed to objectively assess the performance of each method against magnetic disturbance. Based upon the testing results, the strength and weakness of the existing sensor fusion methods were easily examined, and suggestions were presented for selecting a proper sensor fusion algorithm or developing new sensor fusion method.

Accuracy of angular displacements and velocities from inertial-based inclinometers

The objective of this study was to evaluate the accuracy of various sensor fusion algorithms for measuring upper arm elevation relative to gravity (i.e., angular displacement and velocity summary measures) across different motion speeds. Thirteen participants completed a cyclic, short duration, arm-intensive work task that involved transfering wooden dowels at three work rates (slow, medium, fast). Angular displacement and velocity measurements of upper arm elevation were simultaneously measured using an inertial measurement unit (IMU) and an optical motion capture (OMC) system. Results indicated that IMU-based inclinometer solutions can reduce root-mean-square errors in comparison to accelerometer-based inclination estimates by as much as 87%, depending on the work rate and sensor fusion approach applied. The findings suggest that IMU-based inclinometers can substantially improve inclinometer accuracy in comparison to traditional accelerometer-based inclinometers. Ergonomists may use the non-proprietary sensor fusion algorithms provided here to more accurately estimate upper arm elevation.

Development of an economic wireless human motion analysis device for quantitative assessment of human body joint

In recent years, the study of human body dynamics has been attracting a significant amount of attention. Currently there are many camera or active sensor based motion analysis systems available on the market. They have been extensively adopted and used by the film and animation or entertainment industries such as film and video game producers. More recently their potential in studying human dynamics/motion for medical purposes has been realised to the extent that they are now used to study full body human biomechanics in the form of gait analysis systems. Most orthopaedic surgeries are usually about joint repair or implants. According health line, revision surgery is usually due to infection, continued pain, joint stiffness, wear, instability, loosening. Apart from infection, the rest can be linked to the operation itself. Currently, surgical planning and placing implants is performed in a subjective manner, relying on the surgeon’s experience and instinct, current systems to help the surgeon to place implant are also bulky, expensive, slow and not user friendly. The aim of this project is to develop an economic and portable motion assessment system which involves a wireless inertial measurement unit (IMU) dedicated to study and assess body joints. Through the data collected from the IMU, the system is capable real time measurement of relative position and orientation of the human joint. Several tests were conducted to validate the data extracted from gyroscope and accelerometer of the IMU. The joint motion results analysed using the device was compared with the results analysed using commercial video motion analysis software and it shows good correlation. It is found that the gyroscope of the IMU under DMP sensor fusion algorithm and calibration capability is able to give the angular velocity with less than 5% error. This has led to a more accurate orientation data which gives 7% error in average bending angle.

Development of a self-driving car that can handle the adverse weather

Lane and road recognition are essential for self-driving where GPS solution is inaccurate due to the signal block or multipath in an urban environment. Vision based lane or road recognition algorithms have been studied extensively, but they are not robust to changes in weather or illumination due to the characteristic of the sensor. Lidar is a sensor for measuring distance, but it also contains intensity information. The road mark on the road is made to look good with headlight at night by using a special paint with good reflection on the light. With this feature, road marking can be detected with lidar even in the case of changes in illumination due to the rain or shadow. In this paper, we propose equipping autonomous cars with sensor fusion algorithms intended to operate in a different weather conditions. The proposed algorithm was applied to the self-driving car EureCar (KAIST) in order to test its feasibility for real-time use.

Noncontact Sleep Study by Multi-Modal Sensor Fusion.

Polysomnography (PSG) is considered as the gold standard for determining sleep stages, but due to the obtrusiveness of its sensor attachments, sleep stage classification algorithms using noninvasive sensors have been developed throughout the years. However, the previous studies have not yet been proven reliable. In addition, most of the products are designed for healthy customers rather than for patients with sleep disorder. We present a novel approach to classify sleep stages via low cost and noncontact multi-modal sensor fusion, which extracts sleep-related vital signals from radar signals and a sound-based context-awareness technique. This work is uniquely designed based on the PSG data of sleep disorder patients, which were received and certified by professionals at Hanyang University Hospital. The proposed algorithm further incorporates medical/statistical knowledge to determine personal-adjusted thresholds and devise post-processing. The efficiency of the proposed algorithm is highlighted by contrasting sleep stage classification performance between single sensor and sensor-fusion algorithms. To validate the possibility of commercializing this work, the classification results of this algorithm were compared with the commercialized sleep monitoring device, ResMed S+. The proposed algorithm was investigated with random patients following PSG examination, and results show a promising novel approach for determining sleep stages in a low cost and unobtrusive manner.

3D Reconstruction in the Presence of Glass and Mirrors by Acoustic and Visual Fusion.

We present a practical and inexpensive method to reconstruct 3D scenes that include transparent and mirror objects. Our work is motivated by the need for automatically generating 3D models of interior scenes, which commonly include glass. These large structures are often invisible to cameras or even to our human visual system. Existing 3D reconstruction methods for transparent objects are usually not applicable in such a room-sized reconstruction setting. Our simple hardware setup augments a regular depth camera (e.g., the Microsoft Kinect camera) with a single ultrasonic sensor, which is able to measure the distance to any object, including transparent surfaces. The key technical challenge is the sparse sampling rate from the acoustic sensor, which only takes one point measurement per frame. To address this challenge, we take advantage of the fact that the large scale glass structures in indoor environments are usually either piece-wise planar or a simple parametric surface. Based on these assumptions, we have developed a novel sensor fusion algorithm that first segments the (hybrid) depth map into different categories such as opaque/transparent/infinity (e.g., too far to measure) and then updates the depth map based on the segmentation outcome. We validated our algorithms with a number of challenging cases, including multiple panes of glass, mirrors, and even a curved glass cabinet.

Sensor Fusion Algorithm that Has Resilience under Autonomous Driving Conditions: A Survey

Autonomous cars recognize the surroundings through multiple sensors and make decisions to control the car in order to arrive at destination without driver's interventions. In such environment, if sensor data forgery occurs, it could lead to a big (critical) accident that could threaten the life of the driver. In the paper, a research on a way to get accurate driving information through sensor fusion algorithm that has resilience against data forgery and modulation will be discussed.

A synthesizer framework for multimodal cardiorespiratory signals

Multimodal biosignals play an increasing role in medical signal processing. While many novel methods for acquisition and sensorfusion are developed and evaluated, the modeling aspect sees relatively little attention. To overcome this, a synthesizer framework for the generation of multimodal cardiorespiratory signals is presented. The first part of the model consists of a dynamic system of six coupled nonlinear ordinary differential equations. The resulting oscillators generate modality-independent cardiac and respiratory phase signals, which are coupled by respiratory sinus arrhythmia. Moreover, Mayer oscillations of the heart rate are simulated. The second part of the model consists of modality-dependent waveform generators. For each modality, these waveform generators operate with a specific cardiac template that is modulated depending on the respiratory phase signal. The approach is validated qualitatively and quantitatively on three different databases for a variety of biosignals, namely standard electrocardiography (ECG), blood pressure signals, ballistocardiography, photoplethysmography, capacitively coupled ECG as well as respiratory flow and effort. Finally, the possibility to simulate a multimodal recording by combining templates obtained from different databases is demonstrated. The resulting multimodal biosignals mimic important physiological aspects such as modulation due to respiration and exhibit plausible phase relationships. An application of the model to evaluate an algorithm for sensor fusion is demonstrated.

Safe Autonomy Under Perception Uncertainty Using Chance-Constrained Temporal Logic

Autonomous vehicles have found wide-ranging adoption in aerospace, terrestrial as well as marine use. These systems often operate in uncertain environments and in the presence of noisy sensors, and use machine learning and statistical sensor fusion algorithms to form an internal model of the world that is inherently probabilistic. Autonomous vehicles need to operate using this uncertain world-model, and hence, their correctness cannot be deterministically specified. Even once probabilistic correctness is specified, proving that an autonomous vehicle will operate correctly is a challenging problem. In this paper, we address these challenges by proposing a correct-by-synthesis approach to autonomous vehicle control. We propose a probabilistic extension of temporal logic, named Chance Constrained Temporal Logic (C2TL), that can be used to specify correctness requirements in presence of uncertainty. C2TL extends temporal logic by including chance constraints as predicates in the formula which allows modeling of perception uncertainty while retaining its ease of reasoning. We present a novel automated synthesis technique that compiles C2TL specification into mixed integer constraints, and uses second-order (quadratic) cone programming to synthesize optimal control of autonomous vehicles subject to the C2TL specification. We also present a risk distribution approach that enables synthesis of plans with lower cost without increasing the overall risk. We demonstrate the effectiveness of the proposed approach on a diverse set of illustrative examples.

An Adaptive Orientation Estimation Method for Magnetic and Inertial Sensors in the Presence of Magnetic Disturbances.

Magnetic and inertial sensors have been widely used to estimate the orientation of human segments due to their low cost, compact size and light weight. However, the accuracy of the estimated orientation is easily affected by external factors, especially when the sensor is used in an environment with magnetic disturbances. In this paper, we propose an adaptive method to improve the accuracy of orientation estimations in the presence of magnetic disturbances. The method is based on existing gradient descent algorithms, and it is performed prior to sensor fusion algorithms. The proposed method includes stationary state detection and magnetic disturbance severity determination. The stationary state detection makes this method immune to magnetic disturbances in stationary state, while the magnetic disturbance severity determination helps to determine the credibility of magnetometer data under dynamic conditions, so as to mitigate the negative effect of the magnetic disturbances. The proposed method was validated through experiments performed on a customized three-axis instrumented gimbal with known orientations. The error of the proposed method and the original gradient descent algorithms were calculated and compared. Experimental results demonstrate that in stationary state, the proposed method is completely immune to magnetic disturbances, and in dynamic conditions, the error caused by magnetic disturbance is reduced by 51.2% compared with original MIMU gradient descent algorithm.

Direct Georeferencing for Portable Mapping Systems: In the Air and on the Ground

During the last few years, the acquisition of geometric information about objects by using a moving sensor platform has gained increasing popularity in the surveying community. A large number of companies offer vehicle-based mobile mapping systems, which usually contain a fast profile laser scanner, a high-precision inertial measurement unit, and geodetic global navigation satellite system (GNSS) receivers to directly georeference the laser scans and build a consistent point cloud of the object. In contrast, there is a growing number of companies offering small and lightweight unmanned aerial vehicles (UAVs), which automatically fly over the area of interest and create dense point clouds of the environment by using camera images and photogrammetric-processing software. In this case, the georeferencing is usually realized by ground control points. This contribution summarizes activities performed by the authors to bring both fields closer together by developing a small (11 × 10 × 5 cm) and lightweight (240-g) direct-georeferencing unit, which is able to provide accurate position (<5 cm) and orientation (<1°) information in real time. The authors describe the development of the sensor unit and the sensor-fusion algorithms, address the topics of calibration and accuracy evaluation, and provide an overview of different applications in which the unit has been already used. These include the high-resolution acquisition of crop-surface models by using UAV-based imagery, the direct georeferencing of an autonomously flying robot with a laser scanner and multiple cameras, and the generation of laser point clouds using a human-carried mobile mapping system.

Satellite and Vision-Aided Sensor Fusion for Cooperative Navigation of Unmanned Aircraft Swarms

This paper presents a cooperative unmanned aerial vehicle navigation algorithm that allows a chief vehicle (equipped with inertial and magnetic sensors, a Global Positioning System receiver, and a vision system) to improve its navigation performance (in real time or in postprocessing phase), exploiting line-of-sight measurements from formation-flying deputies equipped with Global Positioning System receivers. The key concept is to integrate differential Global Positioning System and visual tracking information within a sensor fusion algorithm based on the extended Kalman filter. The developed concept and processing architecture are described, with a focus on the filtering algorithm. Then, flight-testing strategy and experimental results are presented. In particular, cooperative navigation output is compared with the estimates provided by the onboard autopilot system of a customized quadrotor. The analysis shows the potential of the developed approach, mainly deriving from the possibility to exploit accurate magnetic- and inertial-independent information.

Development of a low-cost GPS/INS integrated system for tractor automatic navigation

The use of low-cost single GPS receivers and inertial sensors for auto-guidance applications has been limited by their reduced accuracy and signal drift over time compared to real-time kinematic (RTK) differential GPS units and fiber-optic gyroscope (FOG) sensors. In this study, a prototype low-cost GPS/INS integrated system consisting of a triangle-shaped array of three Garmin 19x GPS receivers and an Xsens inertial measurement unit (IMU) to improve the accuracy of position and heading angle measured with a single GPS receiver was developed. A triangular algorithm that uses data collected from the three single GPSs mounted on the angular points of a triangular frame was designed. A sensor fusion algorithm based on the Kalman filter combining the GPS and IMU data was developed by integrating position data and heading angles of a triangular array of GPS receivers. The optimized values of two noise covariance matrixes (Q and R) for the Kalman filtering were determined using the Central Composite Design (CCD) method. As compared to the use of a single Garmin GPS receiver, use of the developed GPS/INS system showed improved accuracy performance in terms of both position and heading angle, with reductions in root mean square errors (RMSEs) from 2.7 m to 0.64 m for position and from 8.9o to 2.1o for heading angle. The accuracy improvements show new potential for agricultural auto-guidance applications. Keywords: global positioning system, tractor, automatic navigation, sensor fusion, Kalman filter, inertial sensor, heading angle DOI: 10.3965/j.ijabe.20171002.3070 Citation: Han X Z, Kim H J, Jeon C W, Moon H C, Kim J H. Development of a low-cost GPS/INS integrated system for tractor automatic navigation. Int J Agric & Biol Eng, 2017; 10(2): 123–131.

Guidance and Attitude Control of Unstable Rigid Bodies With Single-Use Thrusters

In this paper, we introduce a novel computationally efficient and practical algorithm for robust guidance and attitude control of unstable rigid bodies using the discrete fixed-magnitude, fixed-duration actuation representative of single-use thrusters. New guidance and attitude continuous control laws are developed based on the nonlinear rigid-body motion model, where the objective is to intercept a target and maintain the near zero angle of attack. A methodology is then introduced to adapt the continuous control system to discrete impulsive forces with constant magnitude and duration. An actuation management scheme is presented based on the equivalence of the total impulse of the continuous control law and thruster force. Realistic time delays and noise distributions are included in a sensor fusion algorithm based on the extended Kalman filter. Simultaneous control for course corrections to ensure the target interception and stabilization of the vehicle attitude is presented through Monte Carlo simulations with various uniformly distributed direction and velocity errors.

Robust Biomechanical Model-Based 3-D Indoor Localization and Tracking Method Using UWB and IMU

This paper proposes a robust sensor fusion algorithm to accurately track the spatial location and motion of a human under various dynamic activities, such as walking, running, and jumping. The position accuracy of the indoor wireless positioning systems frequently suffers from non-line-of-sight and multipath effects, resulting in heavy-tailed outliers and signal outages. We address this problem by integrating the estimates from an ultra-wideband (UWB) system and inertial measurement units, but also taking advantage of the estimated velocity and height obtained from an aiding lower body biomechanical model. The proposed method is a cascaded Kalman filter-based algorithm where the orientation filter is cascaded with the robust position/velocity filter. The outliers are detected for individual measurements using the normalized innovation squared, where the measurement noise covariance is softly scaled to reduce its weight. The positioning accuracy is further improved with the Rauch-Tung-Striebel smoother. The proposed algorithm was validated against an optical motion tracking system for both slow (walking) and dynamic (running and jumping) activities performed in laboratory experiments. The results show that the proposed algorithm can maintain high accuracy for tracking the location of a subject in the presence of the outliers and UWB signal outages with a combined 3-D positioning error of less than 13 cm.

Attitude Determination Method by Fusing Single Antenna GPS and Low Cost MEMS Sensors Using Intelligent Kalman Filter Algorithm

For meeting the demands of cost and size for micronavigation system, a combined attitude determination approach with sensor fusion algorithm and intelligent Kalman filter (IKF) on low cost Micro-Electro-Mechanical System (MEMS) gyroscope, accelerometer, and magnetometer and single antenna Global Positioning System (GPS) is proposed. The effective calibration method is performed to compensate the effect of errors in low cost MEMS Inertial Measurement Unit (IMU). The different control strategies fusing the MEMS multisensors are designed. The yaw angle fusing gyroscope, accelerometer, and magnetometer algorithm is estimated accurately under GPS failure and unavailable sideslip situations. For resolving robust control and characters of the uncertain noise statistics influence, the high gain scale of IKF is adjusted by fuzzy controller in the transition process and steady state to achieve faster convergence and accurate estimation. The experiments comparing different MEMS sensors and fusion algorithms are implemented to verify the validity of the proposed approach.

V2V-Communication, LiDAR System and Positioning Sensors for Future Fusion Algorithms in Connected Vehicles

Abstract The automotive industry is changing from conventional driving into connected and later on autonomous driving. The fundamental principle of this alteration is communication and exchange of data between vehicles and other kind of traffic objects, for example traffic lights. The knowing about basic conditions from all traffic objects within a close proximity can ensure a more precise reaction of Advanced Driver Assistant Systems (ADAS). Thereby Vehicle-to-Vehicle (V2V) technology contributes to increase traffic safety. This paper describes an investigation of V2V communication based on commercial On-Board-Units (OBU). These units, integrated in two test-vehicles, transmit and receive data based on the IEEE 802.11p standard (ETSI ITS-G5). The messages include basic conditions like position, motion vector and vehicle configuration parameters. Those information ensure a relative positioning which is also implemented in the paper on hand. The calculations provide the prerequisites for additional autonomous vehicle system applications like autonomous braking or steering maneuver. To improve precision and reliability of ADAS, statements about integrity are inevitable. These can be achieved by a fusion of different sensor information. Therefore the paper also presents possibilities to ensure an accurate localization and object detection, regarding a Light Detection and Ranging (LiDAR) System (Velodyne VLP-16) and an Differential Global Positioning System (DGPS) approach. The interaction of communication, sensor fusion algorithms and integrity considerations form the basis for autonomous driving.

Special Issue on Future Active Safety Technology towards Zero-Traffic Accidents (FAST-zero) [Guest Editorial

The articles in this special section focus on traffic safety technology design to avoid traffic accidents. Innovation in the field of active safety is the key driving force towards the ultimate goal of realizing zero traffic-accidents. Researchers around the world have been investigating methods for active safety to reduce and possibly eliminate traffic accidents as well as road fatalities. Automated and connected vehicles in particular have the potential to enhance safety through new sensors, dynamic maps, artificial intelligence and other technologies that have experienced great improvements in recent years. Sensor fusion algorithms and platforms are intended to enhance the performance of active safety systems, and thus implement the ultimate goal of zero traffic accidents.

Implementation of Multisensor Data Fusion Algorithm

Multisensor data fusion is the process of combining observations from a number of different sensors to provide a robust and complete description of an environment or process of interest. Data fusion finds wide application in many areas of robotics such as object recognition, environment mapping, and localization. This work has three parts: methods, architectures and applications. Most current data fusion methods employ probabilistic descriptions of observations and processes and use Bayes’ rule to combine this information. Data fusion systems are often complex combinations of sensor devices, processing and fusion algorithms. This work provides an overview of key principles in data fusion architectures from both a hardware and algorithmic viewpoint. The applications of data fusion are pervasive in UAV and underlay the core problem of sensing, estimation and perception. The highlighted is many applications that bring out these features. The first describes a navigation or self-tracking application for an autonomous vehicle. The second describes an application in mapping and environment modeling. The essential algorithmic tools of data fusion are reasonably well established. However, the development and use of these tools in realistic robotics applications is still developing.

Radar and Camera Data Association Algorithm for Sensor Fusion

Radar and Camera Data Association Algorithm for Sensor Fusion