Sensor Fusion Algorithm Research Articles

Towards Dynamic Multi-Modal Intent Sensing Using Probabilistic Sensor Networks.

Intent sensing—the ability to sense what a user wants to happen—has many potential technological applications. Assistive medical devices, such as prosthetic limbs, could benefit from intent-based control systems, allowing for faster and more intuitive control. The accuracy of intent sensing could be improved by using multiple sensors sensing multiple environments. As users will typically pass through different sensing environments throughout the day, the system should be dynamic, with sensors dropping in and out as required. An intent-sensing algorithm that allows for this cannot rely on training from only a particular combination of sensors. It should allow any (dynamic) combination of sensors to be used. Therefore, the objective of this study is to develop and test a dynamic intent-sensing system under changing conditions. A method has been proposed that treats each sensor individually and combines them using Bayesian sensor fusion. This approach was tested on laboratory data obtained from subjects wearing Inertial Measurement Units and surface electromyography electrodes. The proposed algorithm was then used to classify functional reach activities and compare the performance to an established classifier (k-nearest-neighbours) in cases of simulated sensor dropouts. Results showed that the Bayesian sensor fusion algorithm was less affected as more sensors dropped out, supporting this intent-sensing approach as viable in dynamic real-world scenarios.

Intelligent IoT (IIoT) Device to Identifying Suspected COVID-19 Infections Using Sensor Fusion Algorithm and Real-Time Mask Detection Based on the Enhanced MobileNetV2 Model.

This paper employs a unique sensor fusion (SF) approach to detect a COVID-19 suspect and the enhanced MobileNetV2 model is used for face mask detection on an Internet-of-Things (IoT) platform. The SF algorithm avoids incorrect predictions of the suspect. Health data are continuously monitored and recorded on the ThingSpeak cloud server. When a COVID-19 suspect is detected, an emergency email is sent to healthcare personnel with the GPS position of the suspect. A lightweight and fast deep learning model is used to recognize appropriate mask positioning; this restricts virus transmission. When tested with the real-world masked face dataset (RMFD) dataset, the enhanced MobileNetV2 neural network is optimal for Raspberry Pi. Our IoT device and deep learning model are 98.50% (compared to commercial devices) and 99.26% accurate, respectively, and the time required for face mask evaluation is 31.1 milliseconds. The proposed device is useful for remote monitoring of covid patients. Thus, the method will find medical application in the detection of COVID-19-positive patients. The device is also wearable.

Test-Retest Reliability of Kinematic and Temporal Outcome Measures for Clinical Gait and Stair Walking Tests, Based on Wearable Inertial Sensors.

It is important to assess gait function in neurological disorders. A common outcome measure from clinical walking tests is average speed, which is reliable but does not capture important kinematical and temporal aspects of gait function. An extended gait analysis must be time efficient and reliable to be included in the clinical routine. The aim of this study was to add an inertial sensor system to a gait test battery and analyze the test-retest reliability of kinematic and temporal outcome measures. Measurements and analyses were performed in the hospital environment by physiotherapists using customized software. In total, 22 healthy persons performed comfortable gait, fast gait, and stair walking, with 12 inertial sensors attached to the feet, shank, thigh, pelvis, thorax, and arms. Each person participated in 2 test sessions, with about 3–6 days between the sessions. Kinematics were calculated based on a sensor fusion algorithm. Sagittal peak angles, sagittal range of motion, and stride frequency were derived. Intraclass-correlation coefficients were determined to analyze the test-retest reliability, which was good to excellent for comfortable and fast gait, with exceptions for hip, knee, and ankle peak angles during fast gait, which showed moderate reliability, and fast gait stride frequency, which showed poor reliability. In stair walking, all outcome measures except shoulder extension showed good to excellent reliability. Inertial sensors have the potential to improve the clinical evaluation of gait function in neurological patients, but this must be verified in patient groups.

Enhanced Attitude and Altitude Estimation for Indoor Autonomous UAVs

In recent years the use of Unmanned Aerial Vehicles (UAVs) has considerably grown in the civil sectors, due to their high flexibility of use. Currently, two important key points are making them more and more successful in the civil field, namely the decrease of production costs and the increase in navigation accuracy. In this paper, we propose a Kalman filtering-based sensor fusion algorithm, using a low cost navigation platform that contains an inertial measurement unit (IMU), five ultrasonic ranging sensors and an optical flow camera. The aim is to improve navigation in indoor or GPS-denied environments. A multi-rate version of the Extended Kalman Filter is considered to deal with the use of heterogeneous sensors with different sampling rates, and the presence of non-linearities in the model. The effectiveness of the proposed sensor platform is evaluated by means of numerical tests on the dynamic flight simulator of a quadrotor. Results show high precision and robustness of the attitude estimation algorithm, with a reduced computational cost, being ready to be implemented on low-cost platforms.

Big Geospatial Data Analytics, Connected Vehicle Technologies, and Visual Perception and Sensor Fusion Algorithms in Smart Transportation Networks

Big Geospatial Data Analytics, Connected Vehicle Technologies, and Visual Perception and Sensor Fusion Algorithms in Smart Transportation Networks

The Autonomous Platforms Inertial Dataset

One of the critical tasks required for fully autonomous functionality is the ability to achieve an accurate navigation solution; that is, to determine the platform position, velocity, and orientation. Various sensors, depending on the vehicle environment (air, sea, or land), are employed to achieve this goal. In parallel to the development of novel navigation and sensor fusion algorithms, machine-learning based algorithms are penetrating into the navigation and sensor fusion fields. An excellent example for this trend is pedestrian dead reckoning, used for indoor navigation, where both classical and machine learning approaches are used to improve the navigation accuracy. To facilitate machine learning algorithms’ derivation and validation for autonomous platforms, a huge quantity of recorded sensor data is needed. Unfortunately, in many situations, such datasets are not easy to collect or are not publicly available. To advance the development of accurate autonomous navigation, this paper presents the autonomous platforms inertial dataset. It contains inertial sensor raw data and corresponding ground truth trajectories. The dataset was collected using a variety of platforms including a quadrotor, two autonomous underwater vehicles, a land vehicle, a remote controlled electric car, and a boat. A total of 805.5 minutes of recordings were made using different types of inertial sensors, global navigation satellite system receivers, and Doppler velocity logs. After describing the sensors that were employed for the recordings, a detailed description of the conducted experiments is provided. The autonomous platform inertial dataset is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ansfl/Navigation-Data-Project/</uri> .

Wireless Monitoring of Ballastless Track Slab Deformation for High-Speed Railway

Ballastless slab track is the main form of ballastless track for the high-speed railway in the world. Deformation overrun of the track slab will seriously affect the smoothness of the track, and even lead to train safety accidents. The current practice mainly relies on manual periodic inspection, which is inefficient and cannot achieve continuous monitoring. Therefore, it is urgent to develop a real-time deformation monitoring solution for high-speed railway track slabs. This paper, based on multiple MEMs sensors, presents a multi-source sensor fusion and dynamic response analysis algorithm to identify the deformation mode of the track slab. Harvesting the vibration energy of the slab track instead of an electrochemical battery-only strategy could contribute to the achievement of carbon neutrality goals in the field of rail transit. Therefore, a self-contained system approach is developed for wireless monitoring of the track slab deformation. The deformation value of the track slab can be obtained with a monitoring resolution of <=1.5 mm, compliant with maintenance specifications for ballastless slab track deformation assessment (i.e., 1.5 mm for grade I threshold and 2 mm for grade II threshold). This work offers a new means for the safe operation and maintenance of high-speed railways and could also contribute to the realization of the strategic goal of the “dual-carbon project” in China.

Enhanced location tracking in sensor fusion-assisted virtual reality micro-manipulation environments.

Virtual reality (VR) technology plays a significant role in many biomedical applications. These VR scenarios increase the valuable experience of tasks requiring great accuracy with human subjects. Unfortunately, commercial VR controllers have large positioning errors in a micro-manipulation task. Here, we propose a VR-based framework along with a sensor fusion algorithm to improve the microposition tracking performance of a microsurgical tool. To the best of our knowledge, this is the first application of Kalman filter in a millimeter scale VR environment, by using the position data between the VR controller and an inertial measuring device. This study builds and tests two cases: (1) without sensor fusion tracking and (2) location tracking with active sensor fusion. The static and dynamic experiments demonstrate that the Kalman filter can provide greater precision during micro-manipulation in small scale VR scenarios.

Performance Evaluation of an Autonomously Driven Agricultural Vehicle in an Orchard Environment.

To address the problems of inefficient agricultural production and labor shortages, there has been active research to develop autonomously driven agricultural machines, using advanced sensors and ICT technology. Autonomously driven speed sprayers can also reduce accidents such as the pesticide poisoning of farmers, and vehicle overturn that frequently occur during spraying work in orchards. To develop a commercial, autonomously driven speed sprayer, we developed a prototype of an autonomously driven agricultural vehicle, and conducted performance evaluations in an orchard environment. A prototype of the agricultural vehicle was created using a rubber-tracked vehicle equipped with two AC motors. A prototype of the autonomous driving hardware consisted of a GNSS module, a motion sensor, an embedded board, and an LTE module, and it was made for less than $1000. Additional software, including a sensor fusion algorithm for positioning and a path-tracking algorithm for autonomous driving, were implemented. Then, the performance of the autonomous driving agricultural vehicle was evaluated based on two trajectories in an apple farm. The results of the field test determined the RMS, and the maximums of the path-following errors were 0.10 m, 0.34 m, respectively.

HeadUp: A Low-Cost Solution for Tracking Head Movement of Children with Cerebral Palsy Using IMU.

Cerebral palsy (CP) is a common reason for human motor ability limitations caused before birth, through infancy or early childhood. Poor head control is one of the most important problems in children with level IV CP and level V CP, which can affect many aspects of children’s lives. The current visual assessment method for measuring head control ability and cervical range of motion (CROM) lacks accuracy and reliability. In this paper, a HeadUp system that is based on a low-cost, 9-axis, inertial measurement unit (IMU) is proposed to capture and evaluate the head control ability for children with CP. The proposed system wirelessly measures CROM in frontal, sagittal, and transverse planes during ordinary life activities. The system is designed to provide real-time, bidirectional communication with an Euler-based, sensor fusion algorithm (SFA) to estimate the head orientation and its control ability tracking. The experimental results for the proposed SFA show high accuracy in noise reduction with faster system response. The system is clinically tested on five typically developing children and five children with CP (age range: 2–5 years). The proposed HeadUp system can be implemented as a head control trainer in an entertaining way to motivate the child with CP to keep their head up.

WHISPER: Wireless Home Identification and Sensing Platform for Energy Reduction

Many regions of the world benefit from heating, ventilating, and air-conditioning (HVAC) systems to provide productive, comfortable, and healthy indoor environments, which are enabled by automatic building controls. Due to climate change, population growth, and industrialization, HVAC use is globally on the rise. Unfortunately, these systems often operate in a continuous fashion without regard to actual human presence, leading to unnecessary energy consumption. As a result, the heating, ventilation, and cooling of unoccupied building spaces makes a substantial contribution to the harmful environmental impacts associated with carbon-based electric power generation, which is important to remedy. For our modern electric power system, transitioning to low-carbon renewable energy is facilitated by integration with distributed energy resources. Automatic engagement between the grid and consumers will be necessary to enable a clean yet stable electric grid, when integrating these variable and uncertain renewable energy sources. We present the WHISPER (Wireless Home Identification and Sensing Platform for Energy Reduction) system to address the energy and power demand triggered by human presence in homes. The presented system includes a maintenance-free and privacy-preserving human occupancy detection system wherein a local wireless network of battery-free environmental, acoustic energy, and image sensors are deployed to monitor homes, record empirical data for a range of monitored modalities, and transmit it to a base station. Several machine learning algorithms are implemented at the base station to infer human presence based on the received data, harnessing a hierarchical sensor fusion algorithm. Results from the prototype system demonstrate an accuracy in human presence detection in excess of 95%; ongoing commercialization efforts suggest approximately 99% accuracy. Using machine learning, WHISPER enables various applications based on its binary occupancy prediction, allowing situation-specific controls targeted at both personalized smart home and electric grid modernization opportunities.

Measurement of uni-planar and sport specific trunk motion using magneto-inertial measurement units: The concurrent validity of Noraxon and Xsens systems relative to a retro-reflective system

BackgroundThere is a range of magneto-inertial measurement unit (MIMU) systems commercially available, however sensor specifications and fusion methods vary considerably between manufacturers. Such variability can influence the concurrent validity of MIMUs relative to reference standard measurement devices. Different MIMUs have been compared during static or low-velocity conditions, with higher-velocity movements assessed in robotic-based studies. However, there is a need for the concurrent validity of higher-velocity movements to be established in human-based studies. Research questionThis study aimed to assess the concurrent validity of two commercial MIMU systems (Noraxon and Xsens), relative to a ‘gold-standard’ retro-reflective motion capture system, when measuring trunk angles during uni-planar range of motion (ROM) and cricket bowling, which involves high-speed, multi-planar movements. MethodsFor this criterion-based validity study, both MIMU systems incorporated comparable sensor specifications and employed Kalman filter sensor fusion algorithms. The MIMU based angles were compared with angles derived from concurrently captured three-dimensional retro-reflective data for 10 fast-medium bowlers. Statistical parametric mapping and root mean squared differences (RMSD) were computed for both MIMU systems. ResultsOne-dimensional statistical parametric mapping showed no significant differences for angles from both MIMU systems when compared with retro-reflective based angle outputs. The MIMU systems produced ROM RMSDs between 1.4 ± 1.0° and 2.6 ± 1.5°. One system displayed RMSDs between 4.6 ± 1.4° and 7.4 ± 1.9° during bowling, indicating functionally relevant differences to retro-reflective derived angles. There were some small but statistically significant differences in RMSDs between the MIMU systems. SignificanceMIMU-based angle accuracy is poorer during high-speed, multi-planar movement than uni-planar tasks. Comparable MIMU systems can produce varying measurements during ROM and bowling tasks. It is likely that varying sample rates and sensor fusion algorithm parameters contributed to the differences.

LiDAR-sensed tree canopy correction in uneven terrain conditions using a sensor fusion approach for precision sprayers

Precision spraying is one of the most promising techniques to produce healthy and sustainably profitable crops. However, accurate canopy density measurements for precision spraying decisions are still a challenging endeavor, especially in orchards with uneven terrain conditions. A sensor fusion-based canopy point correction system was developed with a 3D light detection and ranging (LiDAR) sensor and an inertial navigation system-global navigation satellite system (INS-GNSS) for accurate tree canopy density measurement. The LiDAR sensor was used to acquire the tree canopy architectures, while the INS-GNSS sensor was to evaluate the terrain slopes and the tree georeferenced locations. A mathematical model was developed to perform the simulation for correction of canopy points based on given changes in the roll, pitch, and yaw angles. A sensor fusion algorithm was developed to process the canopy point corrections for the tree fruit orchards with three different sloping conditions, including longitudinal, lateral, and combination of both slopes. Simulation results reported that the developed model established the correction of tree canopy points with varying roll, pitch, and yaw angles. Field evaluation results suggested that the developed system could be used for correcting canopy points at any sloping conditions in various terrains. The measured tree canopy density from the corrected canopy points reported a possible of off-target chemical reduction up to 13.87%, 5.19%, and 15.45% in orchard sites 1, 2 and 3, respectively. With the accurate tree canopy density measurement, it is anticipated that the developed system could be used to reduce the off-target deposition for precision spraying applications in uneven tree fruit orchards.

Modeling Cameras for Autonomous Vehicle and Robot Simulation: An Overview

Simulation is increasingly important in the development and testing of robots and autonomous vehicles as it opens the door for candidate navigation, perception, and sensor fusion algorithms to be expeditiously probed in complex and safety-critical scenarios. As most robots and autonomous vehicles make heavy use of cameras to perceive their surroundings, camera modeling becomes a prerequisite for the successful simulation of these autonomous agents. This contribution outlines the context in which camera models are used; provides a component-by-component analysis of the image acquisition pipeline along with algorithms used for its modeling; and closes with a discussion of data-driven approaches that embrace a different perspective on camera modeling.

A Novel Positioning Module and Fusion Algorithm for Unmanned Aerial Vehicle Monitoring

Aiming to solve the monitoring problem of unmanned aerial vehicles (UAVs), a novel positioning module is designed and implemented. The hardware design meets the prerequisites of miniaturization and low-cost. Then, a calibration procedure based on six-position calibration and ellipsoid fitting is executed to eliminate the bad influence on the system’s accuracy and stability caused by deterministic errors. Also, a stochastic error modeling procedure based on Allan variance is performed to identify stochastic errors’ model type and model parameters. Finally, data from inertial measurement units (IMUs), global navigation satellite system (GNSS), magnetometers, and a barometer are fused by an extended Kalman filter (EKF) based sensor fusion algorithm. This algorithm is applied to a UAV flight test and comparison result using calibrated data and uncalibrated data respectively are shown. Besides, the proposed EKF algorithm is compared with unscented Kalman filter (UKF) and unscented quaternion estimator (USQUE) to show its practicability in the application. The main contribution of this research is the design of the positioning module and the validation of the sensor fusion algorithm in this hardware. It is found that, under the constraint of low-cost sensors, the accuracy of positioning can achieve around 7.5m horizontally, and 2m vertically. The frequency of positioning increases to 50Hz after sensor fusion, a large improvement over the previous 1Hz GNSS positioning frequency.

Application of the unscented Kalman filter in position estimation a case study on a robot for precise positioning

High-precision measurement is a common task in many engineering applications. In these cases, sensor fusion algorithms represented by Kalman filter and its variants are effective and practical. It is introduced in the article, how to use this sensor fusion algorithm in a high-precision tracking task, where only one sensor (tacheometer) is used in the project. It is also discussed in the article how to build estimation models, including measurement model and motion model, for specific robot positioning problems. A variety of related models are compared in this article. The best model that utilize the prior knowledge of robot motion is proposed, which can not only meet the high precision requirements, but also have robustness to the robot operating environment.

An Innovative Sensor Fusion Algorithm for Motion Tracking With On-Line Bias Compensation: Application to Joint Angles Estimation in Yoga

Magnetic Inertial Measurement Units (MIMUs) represent an increasingly used technology in the field of human motion. However, their applications are strongly affected by magnetic disturbances and poor calibration, which could determine inaccurate attitude estimations. Thus, Inertial Measurement Units (IMUs), relying exclusively on accelerometers and gyroscopes, could be used as an alternative to MIMUs for motion tracking, avoiding the negative effect of magnetic disturbances. Unfortunately, gyroscope signals are characterized by several error sources, among which gyroscope bias, whose variations due to environmental factors and temporal instability strongly affect the attitude estimation accuracy. Aim of the present work is to propose a novel sensor fusion algorithm for IMU-based applications that embodies an adaptive on-line bias capture module. The accuracy of the proposed filter was tested on ten expert yoga practitioners during the execution of a sun salutation sequence. The right upper limb joint angles were estimated and their accuracy assessed in terms of Mean Absolute Error (MAE) and Pearson’s correlation coefficient by comparison with an optoelectronic motion capture system. The achieved worst-case accuracy was 5.59, 6.75 and 3.49 degrees for the wrist, elbow and shoulder joints respectively. The accuracy of the algorithm is further confirmed by the high values of the Pearson’s correlation coefficients between IMUs and OMC angles, which were greater than 0.91, 0.92 and 0.98 respectively for the three considered joints. The proposed algorithm can thus be considered as a promising tool for attitude estimation in those contexts when magnetic interferences cannot be monitored and, thus, adequately compensated.

Extension of the Rigid-Constraint Method for the Heuristic Suboptimal Parameter Tuning to Ten Sensor Fusion Algorithms Using Inertial and Magnetic Sensing.

The orientation of a magneto-inertial measurement unit can be estimated using a sensor fusion algorithm (SFA). However, orientation accuracy is greatly affected by the choice of the SFA parameter values which represents one of the most critical steps. A commonly adopted approach is to fine-tune parameter values to minimize the difference between estimated and true orientation. However, this can only be implemented within the laboratory setting by requiring the use of a concurrent gold-standard technology. To overcome this limitation, a Rigid-Constraint Method (RCM) was proposed to estimate suboptimal parameter values without relying on any orientation reference. The RCM method effectiveness was successfully tested on a single-parameter SFA, with an average error increase with respect to the optimal of 1.5 deg. In this work, the applicability of the RCM was evaluated on 10 popular SFAs with multiple parameters under different experimental scenarios. The average residual between the optimal and suboptimal errors amounted to 0.6 deg with a maximum of 3.7 deg. These encouraging results suggest the possibility to properly tune a generic SFA on different scenarios without using any reference. The synchronized dataset also including the optical data and the SFA codes are available online.

A novel tracking system for human following robots with fusion of MMW radar and monocular vision

PurposeThis paper aims to develop a robust person tracking method for human following robots. The tracking system adopts the multimodal fusion results of millimeter wave (MMW) radars and monocular cameras for perception. A prototype of human following robot is developed and evaluated by using the proposed tracking system.Design/methodology/approachLimited by angular resolution, point clouds from MMW radars are too sparse to form features for human detection. Monocular cameras can provide semantic information for objects in view, but cannot provide spatial locations. Considering the complementarity of the two sensors, a sensor fusion algorithm based on multimodal data combination is proposed to identify and localize the target person under challenging conditions. In addition, a closed-loop controller is designed for the robot to follow the target person with expected distance.FindingsA series of experiments under different circumstances are carried out to validate the fusion-based tracking method. Experimental results show that the average tracking errors are around 0.1 m. It is also found that the robot can handle different situations and overcome short-term interference, continually track and follow the target person.Originality/valueThis paper proposed a robust tracking system with the fusion of MMW radars and cameras. Interference such as occlusion and overlapping are well handled with the help of the velocity information from the radars. Compared to other state-of-the-art plans, the sensor fusion method is cost-effective and requires no additional tags with people. Its stable performance shows good application prospects in human following robots.

Secure Estimation and Attack Isolation for Connected and Automated Driving in the Presence of Malicious Vehicles

Connected and Automated Vehicles (CAVs) rely on the correctness of position and other vehicle kinematics information to fulfill various driving tasks such as vehicle following, lane change, and collision avoidance. However, a malicious vehicle may send false sensor information to the other vehicles intentionally or unintentionally, which may cause traffic inconvenience or loss of human lives. Here, we take the advantage of vehicular cloud and increase the resilience of CAVs to malicious vehicles by assuming each vehicle shares its local sensor information with other vehicles to create information redundancy on the cloud side. We exploit this redundancy and propose a sensor fusion algorithm for the vehicular cloud, capable of providing robust state estimation of all vehicles under the condition that the number of malicious information is sufficiently small. Using the proposed estimator, we provide an algorithm for isolating malicious vehicles. We use numerical examples to illustrate the effectiveness of our methods.