Kalman Filter-based Sensor Fusion Research Articles

Kalman filter-based sensor fusion for Ackermann steering mobile robots

Abstract Autonomous navigation technologies are continuously improving in terms of performance and safety. A key subfield of autonomous navigation for mobile robots is localization or precise positioning, involving the robot understanding its position relative to key points of interest in the environment. Traditionally, localization is purely dependent on the quality of input received from onboard sensors and therefore localization quality degrades in the presence of high sensor error or other faults. Sensor fusion is useful to combat this. Sensor fusion is used to combine various data types from different sensors to gain a lower overall uncertainty or error in the resulting data, when compared to the separate data from each individual sensor. In this study a four-wheel mobile robot with front-wheel driving and steering is localized during navigation. The investigation is performed in simulation and with a physical prototype. The mobile robot has several sensors including a wheel encoder on each wheel, an inertial measurement unit as well as an indoor GPS system. A designed Kalman Filter called The Combined Kalman Filter has been designed to fuse the sensor input data in order to output the position data required to localize the robot in (X, Y) cartesian coordinates. This is completed in the presence of sensor bias error and noise with the output path compared against a pre-selected ground truth trajectory. The proposed filter is based on the Extended Kalman Filter to contend with the non-linear model arising from the four-wheel steerable robot with the steering system following the Ackermann steering condition. The robot model and sensors are simulated in MATLAB for a chosen ground truth trajectory with results highlighting any detected faults and final accurate position information. It was seen that the estimated filter output had high accuracy when compared with the ground truth. Compared with other investigations the proposed filter has reduced mean error and minimal deviations compared with the ground truth path. In addition, this filtering model can be used in general for the localization of other actuated steering vehicles and aims to widen the research in this field as the majority of research in this area leverages differential drive robots which require less complex modeling.

Estimation of brake pad wear and remaining useful life from fused sensor system, statistical data processing, and passenger car longitudinal dynamics

The problem of estimating brake pad wear of cars has been considered, using friction work approximated from the product of longitudinal braking force and distance travelled during braking. The braking force is estimated from a longitudinal dynamics model with inputs from an inertial measurement sensing unit with Kalman filter based sensor fusion, and standard vehicle on-board diagnostics data. Wear coefficients are fitted with Bayesian linear regression on measurements from two series: (I) 16 measurements of previously worn pads over approx. 4,800 km with average wear of approx. -0.5 mm and (II) seven measurements of almost new pads over approx. 1,850 km and approx. -0.15 mm of wear. The approximated friction work appears to correlate well with the wear. The wear coefficients converge quickly after just 4-5 measurements. Coefficients ranging from −4.95⋅10−5mm3/J to −2.08⋅10−5mm3/J are obtained across the cases, when assuming a coefficient of friction μ=0.6 obtained through pin-on-disc (POD) tests. The wear coefficients are validated using POD tests, showing good agreements with the indirectly estimated wear coefficients from vehicle longitudinal dynamics. Data from series (II) is used for prediction of remaining useful life (RUL), where one additional measurement is carried out after 16,552 km. The RUL prediction requires knowledge of the relationship between usage feature and distance travelled. Different approaches are investigated for the predictions: wear based on only distance travelled, linear relationship between usage feature and distance, and three levels of braking severity based on the usage data. Four of the approaches underestimate the distance significantly at around 4,000 to 8,000 km against the actual distance of 16,552 km, while the low severity estimate is closest at 20,460 km, agreeing with expectations due to the driving being primarily on highways.

Design and Implementation of Sensor Systems for Localization of the Autonomous Robot in a Building Area

One of the popular studies recently is about social robots that have been implemented in several public areas such as offices. The robot is an employee or worker assistant robot in the Telkom Surabaya Institute of Technology building to help carry out the work of delivering packages to the destination according to the tasks given. The problem that often occurs is an error in the robot's localization system causing the robot's movement to the target point to experience a position error. This research contributes to the comparative evaluation of 2 localization methods on mobile robots, namely the first is the use of a rotary encoder sensor and the second is the use of sensor fusion based on the extended Kalman filter implemented on the robot prototype. This study aims to develop a sensor system that is adapted to the design of the robot and the environment in which the robot is tested and to find out the comparison of the two methods. The use of extended Kalman filter-based sensor fusion can provide more accurate results in robot localization, especially when moving on complex paths. Sensor fusion enables the combination of several sensors such as rotary encoders and IMU (Inertial Measurement Unit) sensors to provide more complete and accurate information about the position and orientation of the robot. In this study, sensor fusion successfully reduced the localization error of the robot to 0.63 m when moving straight and 0.29 m when moving on a complex path, compared to the use of a single sensor which resulted in a larger error of 0.89 m. Based on the study that has been conducted, it can be considered as a potential solution in the development of other social robots to improve the accuracy and performance of the robots when performing certain tasks in the future.

A Sensor Fusion-Based Cutting Device Attitude Control to Improve the Accuracy of Korean Cabbage Harvesting

Highlights Developed the attitude control system of the cutting device to improve the accuracy of Korean cabbage harvesting. The cutting device provides attitude control, allowing precise harvesting. The attitude control algorithm was designed based on sensor fusion. Field experiments were conducted to compare attitude maintenance and harvest success rates as the attitude control system of cutting device was with/without. Abstract. Harvesting Korean cabbage requires precise cutting since the cutting method determines its quality. However, Korean cabbage fields have uneven surfaces and severe slopes, making it challenging to automatically harvest the crops. This unstructured environment hindered precise cutting, delaying the commercialization of the Korean cabbage harvester. Therefore, we designed an attitude control system for the cutting machine to improve the accuracy of cabbage harvesting. The proposed system controls the cutting level, angle, and height according to the field to improve the harvest performance. This study applied the Kalman filter-based sensor fusion to measure the cutting angle and height for pose estimation and attitude control. The attitude of the cutting device is estimated even in soil and mud by fusing a gyroscope, accelerometer, and linear potentiometer. Subsequently, the attitude is controlled by two hydraulic cylinders so that the cutting machine has the desired cutting angle and height. For the validation of the attitude control, the root mean square error (RMSE) of the cutting angle and height was calculated through experiments in a sloped environment. As a result, the proposed system reduced the RMSE for the cutting angle and height by 92% and 72%, respectively. Furthermore, field tests were performed on farmland with a Korean cabbage harvester to evaluate the harvesting performance of the attitude control system. The harvesting performance was quantitatively computed by scoring the cutting surface of the cabbage, and the performance improved from 56 to 89 points when the attitude control was applied to the cutting machine. Keywords: Attitude control, Automatic cabbage harvesting, Kalman filter, Korean cabbage harvester, Sensor fusion.

Formation control of nonholonomic unmanned ground vehicles via unscented Kalman filter-based sensor fusion approach

This paper investigates the formation control of nonholonomic unmanned ground vehicles via unscented Kalman filter-based sensor fusion approach. According to the kinematic model of single unmanned ground vehicle, the formation model of multiple unmanned ground vehicles is established. Note that the physical leader is considered instead of a virtual leader, which is more realistic. The formation control problem is converted to the stability problem of an error dynamic system. An asymptotic stability condition of the error dynamic system is derived by designing an appropriate Lyapunov function. The leader-following formation is well formed through designing effective control vectors and utilizing unscented Kalman filter-based state estimation algorithm for each follower. Some simulation examples are provided to verify the effectiveness of the proposed formation control algorithm.

Sensor Reduction, Estimation, and Control of an Upper-Limb Exoskeleton

A multi-degree-of-freedom (multi-DOF) exoskeleton relies on an array of sensors to communicate its state (e.g., positions/orientations) and operator-exoskeleton contact interactions (e.g., forces/torques) to its control system. Although sensor redundancy is common in biological systems to cope with uncertainty and partial failure of sensors, in man-made systems, sensor redundancy increases the overall system's cost and control complexity. This study presents a sensor reduction technique for force/torque (F/T) sensors utilizing a Kalman filter-based sensor fusion system in the context of admittance control. The methodology is applied to the EXO-UL8 exoskeleton, which is a powered, redundant, dual-arm, upper-limb robotic system with (7 arm + 1 hand) DOFs incorporating three 6-axis F/T sensors in each arm. Motivated by improving wearability through minimizing human-exoskeleton contact interfaces, which reduces spurious contact forces due to joint misalignment; and reducing cost, the proposed strategy emulates the admittance controller's virtual dynamics with only a subset of sensors, resulting in the physical human-robot interaction feeling the same from the operator's perspective. Experimental results indicate that human-exoskeleton power exchange and actuation stresses of the operator's joints, with the proposed strategy on a subset of two sensors, are comparable to those in the full three-sensor case (p $ 0.01). The experiments verify the proposed methodology for the EXO-UL8, and support the feasibility of operating other Kalman filter-based sensor fusion systems with fewer sensors without sacrificing transparency in physical human-robot interaction.

Kalman Filter Based Sensor Fusion for Altitude Estimation of Aerial Vehicle

Stabilization of Aerial Vehicles requires altitude estimation. In this paper sensor fusion method is developed for aerial vehicles which focuses on estimation of the altitude. Altitude measurement using sensors is too noisy and biased so Kalman filter based sensor fusion is used to estimate altitude. The proposed method is divided into two parts: attitude estimation and altitude estimation. The method is implemented using IMU and barometer sensors. The results show that altitude is estimated accurately and errors are modelled in few seconds after initialization.

LOSA-X: Expandable 3D Motion Tracking System

This paper presents a novel, low-cost, expandable 3D motion tracking system with the ability to adapt to dynamic background illumination. The proposed system consists of multiple tracker modules equipped with a linear optical sensor arrays and a 9-DOF inertial measurement unit for position and orientation tracking of a wireless IR-LED illuminated active marker. This paper presents an extended Kalman filter-based sensor fusion architecture for combining data acquired from multiple trackers and marker for enhanced accuracy and operational range. The marker system adaptively adjusts the LED intensity to optimize energy consumption and measurement accuracy. The proposed system’s ability to extract background illumination allows it to be robust toward differences in scene illuminations, both static and dynamic. Detailed hardware design and performance evaluation through experimentation have been discussed in this paper.

Kalman filter-based EM-optical sensor fusion for needle deflection estimation.

In many clinical procedures such as cryoablation that involves needle insertion, accurate placement of the needle's tip at the desired target is the major issue for optimizing the treatment and minimizing damage to the neighboring anatomy. However, due to the interaction force between the needle and tissue, considerable error in intraoperative tracking of the needle tip can be observed as needle deflects. In this paper, measurements data from an optical sensor at the needle base and a magnetic resonance (MR) gradient field-driven electromagnetic (EM) sensor placed 10cm from the needle tip are used within a model-integrated Kalman filter-based sensor fusion scheme. Bending model-based estimations and EM-based direct estimation are used as the measurement vectors in the Kalman filter, thus establishing an online estimation approach. Static tip bending experiments show that the fusion method can reduce the mean error of the tip position estimation from 29.23mm of the optical sensor-based approach to 3.15mm of the fusion-based approach and from 39.96 to 6.90mm, at the MRI isocenter and the MRI entrance, respectively. This work established a novel sensor fusion scheme that incorporates model information, which enables real-time tracking of needle deflection with MRI compatibility, in a free-hand operating setup.

Augmented marker tracking for peri-acetabular osteotomy surgery.

To develop a hybrid augmented marker-based navigation system for acetabular reorientation during peri-acetabular osteotomy (PAO). The system consists of a tracking unit attached to the patient's pelvis, augmented marker attached to the acetabular fragment and a host computer to do all the computations and visualization. The augmented marker is comprised of an external planar Aruco marker facing toward the tracking unit and an internal inertial measurement unit (IMU) to measure its orientation. The orientation output from the IMU is sent to the host computer. The tracking unit streams a live video of the augmented marker to the host computer, where the planar marker is detected and its pose is estimated. A Kalman filter-based sensor fusion combines the output from marker tracking and the IMU. We validated the proposed system using a plastic bone study and a cadaver study. Every time, we compared the inclination and anteversion values measured by the proposed system to those from a previously developed optical tracking-based navigation system. Mean absolute differences for inclination and anteversion were 1.34 ([Formula: see text]) and 1.21 ([Formula: see text])[Formula: see text], respectively, for the cadaver study. Mean absolute differences were 1.63 ([Formula: see text]) and 1.55 ([Formula: see text])[Formula: see text] for inclination and anteversion for the plastic bone study. In both validation studies, very strong correlations were observed. We successfully demonstrated the feasibility of our system to measure the acetabular orientation during PAO.

Augmented marker tracking for peri-acetabular osteotomy surgery.

We developed and validated a small, easy to use and cost-effective augmented marker-based hybrid navigation system for peri-acetabular osteotomy (PAO) surgery. The hybrid system consists of a tracking unit directly placed on the patient's pelvis, an augmented marker with an integrated inertial measurement unit ('MU) attached to the patient's acetabular fragment and the host computer. The tracking unit sends a live video stream of the marker to the host computer where the marker's pose is estimated. The augmented marker with the 'MU sends its pose estimate to the host computer where we apply sensor fusion to compute the final marker pose estimate. The host computer then tracks the orientation of the acetabular fragment during peri-acetabular osteotomy surgery. Anatomy registration is done using a previously developed registration device. A Kalman filter-based sensor fusion was added to complete the system. A plastic bone study was performed for validation between an optical tracking-based navigation system and our proposed system. Mean absolute difference for inclination and anteversion was 1.63 degrees and 1.55 degrees, respectively. The results show that our system is able to accurately measure the orientation of the acetabular fragment.

Teleoperation control of Baxter robot using Kalman filter-based sensor fusion

ABSTRACTKalman filter has been successfully applied to fuse the motion capture data collected from Kinect sensor and a pair of MYO armbands to teleoperate a robot. A new strategy utilizing the vector approach has been developed to accomplish a specific motion capture task. The arm motion of the operator is captured by a Kinect sensor and programmed with Processing software. Two MYO armbands with the inertial measurement unit embedded are worn on the operator's arm, which is used to detect the upper arm motion of the human operator. This is utilized to recognize and to calculate the precise speed of the physical motion of the operator's arm. User Datagram Protocol is employed to send the human movement to a simulated Baxter robot arm for teleoperation. In order to obtain joint angles for human limb utilizing vector approach, RosPy and Python script programming has been utilized. A series of experiments have been conducted to test the performance of the proposed technique, which provides the basis for the teleoperation of simulated Baxter robot.

모바일 로봇 자세 안정화를 위한 칼만 필터 기반 센서 퓨전

로보틱스 연구에서, 모바일 로봇의 모션 제어를 위해서는 로봇의 실제 위치를 정확히 추정하는 것이 중요하다. 이를 위해 본 연구에서는, 두 개의 서로 다른 센서 데이터를 칼만필터로 융합하여 로봇의 위치인식을 개선하는 연구를 진행한다. 칼만필터로 융합한 두 개의 센서 측정값은 카메라 영상으로부터 측정된 모바일 로봇의 전역(global) 위치 좌표(x, y)값과 모바일 로봇 바퀴에 부착된 엔코더로부터 측정된 로봇의 직선 및 각속도 값이다. 다음으로 칼만필터로부터 계산된 모바일 로봇의 위치값을 모바일 로봇의 자세 안정화에 피드백하여 모션 제어의 퍼포먼스를 향상시켰다. 최종적으로 논문에서 제안한 센서융합 위치인식 기술과 모션제어기를 실제 로봇에 적용하여 실험적으로 검증하였다. 또한 모션제어에 단일 센서를 피드백으로 사용한 경우와 칼만필터로 융합한 위치 값을 사용한 경우를 비교하므로 칼만필터 기반 센서 융합 기술을 사용한 경우의 퍼포먼스 향상을 확인하였다.

Gyro Drift Correction for An Indirect Kalman Filter Based Sensor Fusion Driver.

Sensor fusion techniques have made a significant contribution to the success of the recently emerging mobile applications era because a variety of mobile applications operate based on multi-sensing information from the surrounding environment, such as navigation systems, fitness trackers, interactive virtual reality games, etc. For these applications, the accuracy of sensing information plays an important role to improve the user experience (UX) quality, especially with gyroscopes and accelerometers. Therefore, in this paper, we proposed a novel mechanism to resolve the gyro drift problem, which negatively affects the accuracy of orientation computations in the indirect Kalman filter based sensor fusion. Our mechanism focuses on addressing the issues of external feedback loops and non-gyro error elements contained in the state vectors of an indirect Kalman filter. Moreover, the mechanism is implemented in the device-driver layer, providing lower process latency and transparency capabilities for the upper applications. These advances are relevant to millions of legacy applications since utilizing our mechanism does not require the existing applications to be re-programmed. The experimental results show that the root mean square errors (RMSE) before and after applying our mechanism are significantly reduced from to , respectively.

UWB-Aided Inertial Motion Capture for Lower Body 3-D Dynamic Activity and Trajectory Tracking

This paper introduces a novel method for simultaneous 3-D trajectory tracking and lower body motion capture (MoCap) under various dynamic activities such as walking and jumping. The proposed method uses wearable inertial sensors fused with an ultrawideband localization system using a cascaded Kalman filter-based sensor fusion algorithm, which consists of a separate orientation filter cascaded with a position/velocity filter. In addition, to further improve the joint angle tracking accuracy, anatomical constraints are applied to the knee joint. Currently, available self-contained inertial tracking methods are not only drift-prone over time but also their accuracy is degraded under fast movements with unstable ground contact states due to the lack of external references. However, our experimental results, which benchmark the system against a reference camera-based motion tracking system, show that the proposed fusion method can accurately capture the dynamic activities of a subject without drift. The results show that the proposed system can maintain similar accuracies between fast and slow motions in lower body MoCap and 3-D trajectory tracking. The obtained accuracies of the system for 3-D body localization and knee joint angle tracking for fast motions were less than 5 cm and 2.1°, respectively.

사물인터넷 시스템을 위한 센서 융합 FPGA 구현

In this paper, a Kalman filter-based sensor fusion filter that measures posture by calibrating and combining information obtained from acceleration and gyro sensors was proposed. Recent advancements in sensor network technology have required sensor fusion technology. In the proposed approach, the nonlinear system model of the filter is converted to a linear system model through a Jacobian matrix operation, and the measurement value predicted via Euler integration. The proposed filter was implemented at an operating frequency of 74 MHz using a Virtex-6 FPGA Board from Xilinx Inc. Further, the accuracy and reliability of the measured posture were validated by comparing the values obtained using the implemented filters with those from existing filters.

Unscented Kalman Filter Based Sensor Fusion for Robust Optical and Electromagnetic Tracking in Surgical Navigation

Computer-aided surgery systems provide visual guidance to surgeons by showing the real-time pose of surgical instruments overlaid on preoperative medical images of the patient. Surgical instrument poses are localized in space using mainly optical tracking systems (OTS) and electromagnetic tracking systems (EMTS). OTS systems require clear line-of-sight, which is difficult to ensure in current overcrowded operating rooms. On the other hand, EMTSs provide less accuracy and suffer from magnetic field distortion in the presence of metal objects. In this paper, we propose a sensor fusion algorithm to compensate for the drawbacks of OTS and EMTS, and achieve robust tracking of surgical instruments. Spatial alignment of OTS and EMTS data will be achieved through a calibration procedure. The proposed sensor fusion algorithm uses an unscented Kalman filter (UKF), an extension of the standard Kalman filter based on a deterministic sampling of nonlinear functions. Quaternion representation for rotations is used to avoid singularities of other parameterizations (e.g., Euler angles). In cases of optical marker occlusion, our algorithm will take advantage of the EMTS to estimate the position of the hidden marker(s) and feed it in the UKF. The fusion algorithm will also keep track of an error matrix between OTS and EMTS measured poses, providing an up-to-date estimate of the electromagnetic distortion. This will be used to correct EMTS measurement before using it to compensate for possible optical line-of-sight occlusions. The proposed sensor fusion (SF) method is compared to a predictive UKF based on optical data (NSF). Our results show that the SF effectively compensates for short marker occlusion (nine samples) providing a continuous estimate of the instrument pose with an error significantly lower than the NSF method, and reaching a clinically acceptable accuracy. Furthermore, the proposed algorithm increases the accuracy of EMTS in the presence of magnetic field distortion.

Rider Trunk and Bicycle Pose Estimation With Fusion of Force/Inertial Sensors

Estimation of human pose in physical human-machine interactions such as bicycling is challenging because of highly-dimensional human motion and lack of inexpensive, effective motion sensors. In this paper, we present a computational scheme to estimate both the rider trunk pose and the bicycle roll angle using only inertial and force sensors. The estimation scheme is built on a rider-bicycle dynamic model and the fusion of the wearable inertial sensors and the bicycle force sensors. We take advantages of the attractive properties of the robust force measurements and the motion-sensitive inertial measurements. The rider-bicycle dynamic model provides the underlying relationship between the force and the inertial measurements. The extended Kalman filter-based sensor fusion design fully incorporates the dynamic effects of the force measurements. The performance of the estimation scheme is demonstrated through extensive indoor and outdoor riding experiments.

Kalman Filter-Based Sensor Fusion for Improving Localization of AGV

This paper proposes localization using sensor fusion with a laser navigation and an inertial navigation system for indoor mobile. The laser navigation is a device that measures angle and distance between the robot and the reflectors. Although it is the high-precision device for indoor global positioning, there is a problem that the accuracy of laser navigation significantly drops while moving at high speed and rapid turning. To solve this problem, the laser navigation was fused to inertial navigation system through Kalman filter. For experiment, we use omnidirectional robot with Mecanum Wheels and analyze the positioning accuracy according to driving direction of the robot.

A Kalman Filter based Sensor Fusion Technique for Balancing a 2-Wheel System

The objective of the presented work is to design and implement a sensor fusion algorithm using Kalman Filter for balancing the system on 2-wheels. The system uses inertial sensors such as 3-axis linear accelerometer and dual-axis gyroscope to calculate the tilt for balancing. The estimation algorithm i.e. Kalman filter continuously and recursively corrects the values obtained by mathematical integration of the velocity, measured using gyroscope at the rate of 20Hz. The correction is performed using the inclination value obtained from accelerometer. This reduces the integration drift that originates from errors in the angular velocity signal. In addition, the gyroscope offset is continuously calibrated. The tilt estimated by the Kalman filter is given to PID algorithm with a reference of 0 radian, to balance the system. The result shows the need of Kalman filter to remove sensor noise. The control and filter algorithm are implemented on Atmega32 microcontroller. This study reinforces the significance of sensor fusion for optimum performance. The conception presented in this paper will be of assistance in existing applications and in new designs.