Sensor Fusion Scheme Research Articles

Machine Vision/GPS Integration Using EKF for the UAV Aerial Refueling Problem

The purpose of this paper is to propose the application of an extended Kalman filter (EKF) for the sensors fusion task within the problem of aerial refueling for unmanned aerial vehicles (UAVs). Specifically, the EKF is used to combine the position data from a global positioning system (GPS) and a machine vision (MV)-based system for providing a reliable estimation of the tanker-UAV relative position throughout the docking and the refueling phase. The performance of the scheme has been evaluated using a virtual environment specifically developed for the study of the UAV aerial refueling problem. Particularly, the EKF-based sensor fusion scheme integrates GPS data with MV-based estimates of the tanker-UAV position derived through a combination of feature extraction, feature classification, and pose estimation algorithms. The achieved results indicate that the accuracy of the relative position using GPS or MV estimates can be improved by at least one order of magnitude with the use of EKF in lieu of other sensor fusion techniques.

Sonar-Based Rover Navigation for Single or Multiple Platforms: Forward Safe Path and Target Switching Approach

In this paper, we have proposed a sensor fusion scheme along with the geometrical modeling of mobile robot navigation path in an unknown environment. In this scheme, the physical placement of sonars, their ranging limits and beam opening angles are considered. A simple 2-D axis transformation is proposed to relate local robot frame with the actual navigation environment. forward safe path (FSP) and target switching approach (TSA) are proposed for efficient obstacle avoidance and target tracking of mobile robot. FSP greatly simplifies the environment conditions as sensed by the robot and also provides minimum turning path during avoidance of obstacles. This method also removes the ldquooscillationrdquo in the mobile robot navigation path. TSA technique gives highest priority on the target tracking during the obstacle avoidance and seeks minimum distance path towards the target. These methods remove unnecessary turning of mobile robot during navigation. A scheme for target directional motion is also proposed. So, mobile robot takes the minimum turning path required towards the target. These methods also ensure the avoidance of ldquodead cycle problemrdquo. These schemes are successfully implemented on a model of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PatrolBot</i> mobile robot from <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ActivMedia</i> Robotics. The overview of current research work on multi-domain robotic system namely system-of-systems is also presented. This paper also describes the Global Positioning System-based navigation of rovers. Results of real-time experiments with Pioneer II P2AT-8 from <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ActivMedia</i> are included in this paper to show the future aspect of this research work.

Sensor Fusion for Compliant Robot Motion Control

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Force feedback is necessary for accurate force control in robotic manipulators, and thus far, wrist force/torque (F/T) sensors have been used. But an important problem arises when only these types of sensors are used. In a dynamic situation where the manipulator moves in either free or constrained space, the interaction forces and moments at the contact point and also the noncontact ones are measured by the mentioned sensor. In this paper, an estimator based on a sensor fusion strategy integrating the measurements of three different sensors (a wrist F/T sensor, an inertial sensor, and joint sensors) was developed to determine the contact force and torque exerted by the manipulator to its environment. The resulting observer helps to overcome some difficulties of uncertain world models and unknown environments since it reduces the high-frequency and low-frequency spectral contents, i.e., the low-frequency component due to inertia of a heavy tool mass and the high-frequency component due to impacts. The new improvement was experimentally validated in a force/position impedance control loop applied to a StÄubli RX60 industrial robotic platform. </para>

Sensor fusion for online tool condition monitoring in milling

The objective of this paper is to combine a direct sensor (vision) and an indirect sensor (force) to create an intelligent integrated tool condition monitoring (TCM) system for online monitoring of flank wear and breakage in milling, using the complementary strengths of the two types of sensors. For flank wear, images of the tool are captured and processed in-cycle using successive moving-image analysis. Two features of the cutting force, which closely indicate flank wear, are extracted in-process and appropriately pre-processed. A self-organizing map (SOM) network is trained in a batch mode after each cutting pass, using the two features derived from the cutting force, and measured wear values obtained by interpolating the vision-based measurement. The trained SOM network is applied to the succeeding machining pass to estimate the flank wear in-process. The in-cycle and in-process procedures are employed alternatively for the online monitoring of the flank wear. To detect breakage, two features in time domain derived from cutting force are used, and the thresholds for them are determined dynamically. Again, vision is used to verify any breakage identified in-process through the cutting force monitoring. Experimental results show that this sensor fusion scheme is feasible and effective for the implementation of online tool condition monitoring in milling, and is independent of the cutting conditions used.

A Method for Judicious Fusion of Inconsistent Multiple Sensor Data

One of the major problems in sensor fusion is that sensors frequently provide spurious observations which are difficult to predict and model. The spurious measurements from sensors must be identified and eliminated since their incorporation in the fusion pool might lead to inaccurate estimation. This paper presents a unified sensor fusion strategy based on a modified Bayesian approach that can automatically identify the inconsistency in sensor measurements so that the spurious measurements can be eliminated from the data fusion process. The proposed method adds a term to the commonly used Bayesian formulation. This term is an estimate of the probability that the data is not spurious, based upon the measured data and the unknown value of the true state. In fusing two measurements, it has the effect of increasing the variance of the posterior distribution when measurement from one of the sensors is inconsistent with respect to the other. The increase or decrease in variance can be estimated using the information theoretic measure entropy. The proposed strategy was verified with the help of extensive computations performed on simulated data from three sensors. A comparison was made between two different fusion schemes: centralized fusion in which data obtained from all sensors were fused simultaneously, and a decentralized or sequential Bayesian scheme that proved useful for identifying and eliminating spurious data from the fusion process. The simulations verified that the proposed strategy was able to identify spurious sensor measurements and eliminate them from the fusion process, thus leading to a better overall estimate of the true state. The proposed strategy was also validated with the help of experiments performed using stereo vision cameras, one infrared proximity sensor, and one laser proximity sensor. The information from these three sensing sources was fused to obtain an occupancy profile of the robotic workspace

Fusion of color and infrared video for moving human detection

We approach the task of human silhouette extraction from color and thermal image sequences using automatic image registration. Image registration between color and thermal images is a challenging problem due to the difficulties associated with finding correspondence. However, the moving people in a static scene provide cues to address this problem. In this paper, we propose a hierarchical scheme to automatically find the correspondence between the preliminary human silhouettes extracted from synchronous color and thermal image sequences for image registration. Next, we discuss strategies for probabilistically combining cues from registered color and thermal images for improved human silhouette detection. It is shown that the proposed approach achieves good results for image registration and human silhouette extraction. Experimental results also show a comparison of various sensor fusion strategies and demonstrate the improvement in performance over non-fused cases for human silhouette extraction.

Sensor Fusion Strategy in the Monitoring of Cutting Tool Wear

Cutting tool wear is a major problem in machining processes. It has a great effect on the quality of a workpiece. Thus, monitoring cutting tool wear is very important in order to maintain the workpiece quality as well to reduce production rate and production time. The use of a single sensor in a monitoring system may not be accurate to detect cutting tool wear. In this paper, sensor fusion technology is introduced for monitoring cutting tool wear.

비선형 변환의 비젼센서 데이터융합을 이용한 이동로봇 주행제어

The robots that will be needed in the near future are human-friendly robots that are able to coexist with humans and support humans effectively. To realize this, robot need to recognize his position and direction for intelligent performance in an unknown environment. And the mobile robots may navigate by means of a number of monitoring systems such as the sonar-sensing system or the visual-sensing system. Notice that in the conventional fusion schemes, the measurement is dependent on the current data sets only. Therefore, more of sensors are required to measure a certain physical parameter or to improve the accuracy of the measurement. However, in this research, instead of adding more sensors to the system, the temporal sequence of the data sets are stored and utilized for the accurate measurement. As a general approach of sensor fusion, a UT -Based Sensor Fusion(UTSF) scheme using Unscented Transformation(UT) is proposed for either joint or disjoint data structure and applied to the landmark identification for mobile robot navigation. Theoretical basis is illustrated by examples and the effectiveness is proved through the simulations and experiments. The newly proposed, UT-Based UTSF scheme is applied to the navigation of a mobile robot in an unstructured environment as well as structured environment, and its performance is verified by the computer simulation and the experiment.

지능형 이족보행로봇을 위한 센서시스템 연구

In this paper, An analysis on the intelligence system for a biped walking robot(BWR) was made and its results were applied to the BWR. Various sensors were applied to the developed BWR for autonomous and intelligent walk in unknown environments. To measure the distance between the object and BWR, ultrasonic sensor and infrared-rays sensor were used. To identity surrounding environments, vision system was used. Gyro sensor was used to control the posture of BWR. Also, piezoelectricity sensor was used to identity the pressure of foot landing on the surface. Sensors applied to the robot have measurement errors according to noises or walking environments. To improve the function of these sensors, influences of noise or sensing errors were minimized using a sensor fusion scheme. A gait test using the sensor fusion system was performed, and its results are presented.

Space and time sensor fusion using an active camera for mobile robot navigation

We propose a sensor-fusion technique where the data sets for previous moments are properly transformed and fused into the current data sets to allow accurate measurements, such as the distance to an obstacle or the location of the service robot itself. In conventional fusion schemes, measurements are dependent on the current data sets. As a result, more sensors are required to measure a certain physical parameter or to improve the accuracy of a measurement. However, in this approach, instead of adding more sensors to the system, the temporal sequences of the data sets are stored and utilized to improve the measurements. The theoretical basis is illustrated by examples, and the effectiveness is proved through simulations. Finally, the new space and time sensor fusion (STSF) scheme is applied to the control of a mobile robot in an unstructured environment and a structured environment.

Modeling and control issues for autonomous aerial refueling for UAVs using a probe–drogue refueling system

A critical limitation for the current use of Unmanned Aerial Vehicles (UAVs) is represented by the lack of aerial refueling capabilities. This paper describes the results of an effort on the modeling of the UAV aerial refueling problem and on the design of the docking control scheme. The control of the docking maneuver is based on a fuzzy sensor fusion strategy featuring GPS and Machine Vision (MV) data. The design for a smooth docking maneuver under the presence of wake effects is performed; desirable performances were achieved with LQR-based control laws for the docking of the UAV to the probe–drogue refueling system. Simulations of the proposed docking scheme are presented and discussed.

Control of the Mobile Robot Navigation Using a New Time Sensor Fusion

This paper proposes a sensor-fusion technique where the data sets for the previous moments are properly transformed and fused into the current data sets to enable accurate measurement, such as, distance to an obstacle and location of the service robot itself. In the conventional fusion schemes, the measurement is dependent on the current data sets. As the results, more of sensors are required to measure a certain physical parameter or to improve the accuracy of the measurement. However, in this approach, instead of adding more sensors to the system, the temporal sequence of the data sets are stored and utilized for the measurement improvement. Theoretical basis is illustrated by examples and the effectiveness is proved through the simulations. Finally, the new space and time sensor fusion(STSF) scheme is applied to the control of a mobile robot in an unstructured environment as well as structured environment.

Adaptive, integrated sensor processing to compensate for drift and uncertainty: a stochastic ‘neural’ approach

An adaptive stochastic classifier based on a simple, novel neural architecture--the Continuous Restricted Boltzmann Machine (CRBM) is demonstrated. Together with sensors and signal conditioning circuits, the classifier is capable of measuring and classifying (with high accuracy) the H+ ion concentration, in the presence of both random noise and sensor drift. Training on-line, the stochastic classifier is able to overcome significant drift of real incomplete sensor data dynamically. As analogue hardware, this signal-level sensor fusion scheme is therefore suitable for real-time analysis in a miniaturised multisensor microsystem such as a Lab-in-a-Pill (LIAP).

AGV Navigation Using a Space and Time Sensor Fusion of an Active Camera

This paper proposes a sensor-fusion technique where rho data sets for the previous moments are properly transformed and fused into the current data sets to enable accurate measurement, such as, distance to an obstacle and location of the service robot itself. In the conventional fusion schemes, the measurement is dependent only on the current data sets. As the results, more of sensors are required to measure a certain physical promoter or to improve the accuracy of the measurement. However, in this approach, intend of adding more sensors to the system, the temporal sequence of the data sets are stored and utilized for the measurement improvement. Theoretical basis is illustrated by examples md the effectiveness is proved through the simulation. Finally, the new space and time sensor fusion (STSF) scheme is applied to the control of a mobile robot in the indoor environment and the performance was demonstrated by the real experiments.

Development of a Sensor Fusion Strategy for Robotic Application Based on Geometric Optimization

Fusion of multi-sensor information is an important technology, which is growing exponentially due to its tremendous application potential in many areas. Effective fusion of data from sensors is very critical in increasing an intelligent system's capability to accomplish complex tasks. Appropriate fusion technologies are needed to be developed specially when a system requires redundant sensors to be used. More the redundancy in sensors, more is the computational complexity for controlling the system and more is its intelligence level. This research presents a strategy developed for multiple sensor fusion, based on geometric optimization. Each sensor's uncertainty has been modeled using classical Lagrangian optimization techniques. However, the uniqueness and effectiveness of the present technique lies on the fact that starting from the optimized value as initial estimate the accuracy of the sensory information has further been improved up to any pre defined bounded range, by developing two architectures – FFA (fission–fusion architecture) and FDD (fusion in differential domain). Sufficient evidences and analyses have been provided in the paper to show its effectiveness in various applications.

A Sensor Fusion Scheme for Autonomous Undetwater Vehicle Localization in Cable Following

In this paper, a sensor fusion technique is proposed for autonomous underwater cable following. The work presented here is an extension of the vision based tracking system proposed earlier (Balasuriya et al., 1998). Two practical problems, encountered in vision based localizing, are considered and a solution based on a sensor fusion technique is proposed. The two practical problems considered are; 1) situation when the cable is totally invisible in the image; and 2) situation when there are many similar cables appearing in the image. An experiment is conducted by setting an underwater cable with real world situations, especially with the above mentioned cases, and an Autonomous Underwater Vehicle (AUV), the "Twin-Burger 2" is used to track the cable. The experimental results demonstrate that the proposed method is quite stable in handling the practical problems related to vision based tracking systems.

Multiple Hypotheses Testing Method for Distributed Multisensor Systems

In this paper, we propose a two-layer sensor fusion scheme for multiple hypotheses multisensor systems. To reflect reality in decision making, uncertain decision regions are introduced in the hypotheses testing process. The entire decision space is partitioned into distinct regions of “correct”, “uncertain” and “incorrect” regions. The first layer of decision is made by each sensor indepedently based on a set of optimal decision rules. The fusion process is performed by treating the fusion center as an additional “virtual” sensor to the system. This “virtual” sensor makes decision based on the decisions reached by the set of sensors in the system. The optimal decision rules are derived by minimizing the Bayes risk function. As a consequence, the performance of the system as well as individual sensors can be quantified by the probabilities of correct, incorrect and uncertain decisions. Numerical examples of three hypotheses, two and four sensor systems are presented to illustrate the proposed scheme.

Improved Sensor Selection Technique by Integrating Sensor Fusion in Robot Position Estimation

Modelling and reducing uncertainty are two essential problems with mobile robot localisation. Previously we developed a robot localisation system, namely, the Gaussian Mixture of Bayes with Regularised Expectation Maximisation (GMB-REM), which introduced the sensor selection technique. GMB-REM allows a robot"s position to be modelled as a probability distribution and uses Bayes" theorem to reduce the uncertainty of its location. A new sensor selection technique incorporated with sensor fusion is introduced in this paper. Actually the new technique is realised by incorporating with the sensor fusion scheme. Empirical results show that the new system outperforms the previous GMB-REM with sensor selection alone. More specifically, we illustrate that the new technique is able to considerably constrain the error of a robot"s position.

Triangulation-based fusion of sonar data with application in robot pose tracking

In this paper a sensor fusion scheme, called triangulation-based fusion (TBF) of sonar data, is presented. This algorithm delivers stable natural point landmarks, which appear in practically all indoor environments, i.e., vertical edges like door posts, table legs, and so forth. The landmark precision is in most cases within centimeters. The TBF algorithm is implemented as a voting scheme, which groups sonar measurements that are likely to have hit the same object in the environment. The algorithm has low complexity and is sufficiently fast for most mobile robot applications. As a case study, we apply the TBF algorithm to robot pose tracking. The pose tracker is implemented as a classic extended Kalman filter, which use odometry readings for the prediction step and TBF data for measurement updates. The TBF data is matched to pre-recorded reference maps of landmarks in order to measure the robot pose. In corridors, complementary TBF data measurements from the walls are used to improve the orientation and position estimate. Experiments demonstrate that the pose tracker is robust enough for handling kilometer distances in a large scale indoor environment containing a sufficiently dense landmark set.

On-Line Detection of Drill Wear

An on-line system for drill wear detection has been developed by using a sensor fusion strategy. Both acceleration and thrust signals were analyzed. Flank wear area was used to evaluate drill wear states. The drill wear area was measured by a vision system and classified into two groups: usable and worn-out. The wear prediction model was obtained by a two-category linear classifier. On-line detection tests indicate that the prediction model has over a 90 percent success rate.