Multi-sensor Fusion Algorithm Research Articles

Multi-sensor Image Fusion Algorithm Based on Multiresolution Analysis

To solve the fusion problem of visible and infrared images, based on image fusion algorithm such as region fusion, wavelet transform, spatial frequency, Laplasse Pyramid and principal component analysis, the quality evaluation index of image fusion was defined. Then, curve-let transform was used to replace the wavelet change to express the superiority of the curve. It integrated the intensity channel and the infrared image, and then transformed it to the original space to get the fused color image. Finally, two groups of images at different time intervals were used to carry out experiments, and the images obtained after fusion were compared with the images obtained by the first five algorithms, and the quality was evaluated. The experiment showed that the image fusion algorithm based on curve-let transform had good performance, and it can well integrate the information of visible and infrared images. It is concluded that the image fusion algorithm based on curve-let change is a feasible multi-sensor image fusion algorithm based on multi-resolution analysis.

Accurate and Real-Time 3-D Tracking for the Following Robots by Fusing Vision and Ultrasonar Information

Acquiring the accurate three-dimensional (3-D) position of a target person around a robot provides valuable information that is applicable to a wide range of robotic tasks, especially for promoting the intelligent manufacturing processes of industries. This paper presents a real-time robotic 3-D human tracking system that combines a monocular camera with an ultrasonic sensor by an extended Kalman filter (EKF). The proposed system consists of three submodules: a monocular camera sensor tracking module, an ultrasonic sensor tracking module, and the multisensor fusion algorithm. An improved visual tracking algorithm is presented to provide 2-D partial location estimation. The algorithm is designed to overcome severe occlusions, scale variation, target missing, and achieve robust redetection. The scale accuracy is further enhanced by the estimated 3-D information. An ultrasonic sensor array is employed to provide the range information from the target person to the robot, and time of flight is used for the 2-D partial location estimation. EKF is adopted to sequentially process multiple, heterogeneous measurements arriving in an asynchronous order from the vision sensor, and the ultrasonic sensor separately. In the experiments, the proposed tracking system is tested in both a simulation platform and actual mobile robot for various indoor and outdoor scenes. The experimental results show the persuasive performance of the 3-D tracking system in terms of both the accuracy and robustness.

Superpixel-Based Causal Multisensor Video Fusion

Video surveillance systems have become extremely important recently. It has been observed that information extracted from a single spectrum video is often insufficient in adverse conditions like low illumination, shadowing, smoke, dust, unstable background, and camouflage. Real-time video processing systems also need to be very fast where future frames are often unavailable at the time of processing the current frame. In this paper, we propose a superpixel-based causal multisensor video fusion algorithm suitable for real-time surveillance tasks. We develop new superpixel level spatial and temporal saliency models. Novel superpixel level multiple fusion rules are also designed to obtain the fused output. Comprehensive comparisons with several existing works clearly indicate the benefit of our solution.

An event based multi-sensor fusion algorithm with deadzone like measurements

Event based strategy has received increasing attention recent years since it can provide a useful compromise between estimation performance and constrained energy or communication resources. In this paper, we propose an event based multi-sensor fusion algorithm with deadzone like measurements, where every sensor compares its measurements with an interval, and only elements beyond the thresholds, i.e., outside the deadzone, are sent to the fusion centre. To fuse this kind of deadzone like measurement, we firstly derive a modified Kalman filter (KF), which is based on the statistical properties of measurements. Then we obtain its information form, which is utilised in our event based information fusion algorithm to further release the computation burden caused by multi-sensor. Existing standard Tobit KF and KF are special cases of our modified KF, and simulation results demonstrate the advantages of the proposed event based algorithm as compared with several existing methods.

Unsupervised Residential Power Usage Monitoring Using a Wireless Sensor Network

Appliance-level power usage monitoring may help conserve electricity in homes. Several existing systems achieve this goal by exploiting appliances’ power usage signatures identified in labor-intensive in situ training processes. Recent work shows that autonomous power usage monitoring can be achieved by supplementing a smart meter with distributed sensors that detect the working states of appliances. However, sensors must be carefully installed for each appliance, resulting in a high installation cost. This article presents Supero —the first ad hoc sensor system that can monitor appliance power usage without supervised training. By exploiting multisensor fusion and unsupervised machine learning algorithms, Supero can classify the appliance events of interest and autonomously associate measured power usage with the respective appliances. Our extensive evaluation in five real homes shows that Supero can estimate the energy consumption with errors less than 7.5%. Moreover, nonprofessional users can quickly deploy Supero with considerable flexibility.

Oil exploration oriented multi-sensor image fusion algorithm

Abstract In order to accurately forecast the fracture and fracture dominance direction in oil exploration, in this paper, we propose a novel multi-sensor image fusion algorithm. The main innovations of this paper lie in that we introduce Dual-tree complex wavelet transform (DTCWT) in data fusion and divide an image to several regions before image fusion. DTCWT refers to a new type of wavelet transform, and it is designed to solve the problem of signal decomposition and reconstruction based on two parallel transforms of real wavelet. We utilize DTCWT to segment the features of the input images and generate a region map, and then exploit normalized Shannon entropy of a region to design the priority function. To test the effectiveness of our proposed multi-sensor image fusion algorithm, four standard pairs of images are used to construct the dataset. Experimental results demonstrate that the proposed algorithm can achieve high accuracy in multi-sensor image fusion, especially for images of oil exploration.

An algorithm for multi-sensor image fusion using maximum a posteriori and nonsubsampled contourlet transform

Multi-sensor image fusion draws an inference based on the information obtained from different sensors. Recently, wavelet and contourlet transforms have been widely used in multi-sensor image fusion. But these transforms have been inadequate in the representation of images due to their subsampling. Hence, a fusion algorithm based on Synthetic Aperture Radar (SAR) and Panchromatic (PAN) images in Nonsubsampled Contourlet Transform (NSCT) domain is proposed. NSCT gives flexible multiscale, multidirectional expansion for images. A high fusion accuracy is achieved by ‘Maximum A Posteriori (MAP)’ estimation based on Rayleigh and Laplacian probabilities for despeckling of SAR higher frequency coefficients. Subsequently, the despeckled SAR coefficient is directly fused with PAN coefficients using the newly developed Edge-based fusion rule. The combination of NSCT, MAP and Edge-fusion rule facilitates maximum preservation of the edge and the texture information. The performance of the proposed fusion algorithm is evaluated using reference and non-reference quality metrics. The results prove that the proposed method outperforms the existing NSCT methods by preserving maximum features.

An Improved Multi-Sensor Fusion Navigation Algorithm Based on the Factor Graph.

An integrated navigation system coupled with additional sensors can be used in the Micro Unmanned Aerial Vehicle (MUAV) applications because the multi-sensor information is redundant and complementary, which can markedly improve the system accuracy. How to deal with the information gathered from different sensors efficiently is an important problem. The fact that different sensors provide measurements asynchronously may complicate the processing of these measurements. In addition, the output signals of some sensors appear to have a non-linear character. In order to incorporate these measurements and calculate a navigation solution in real time, the multi-sensor fusion algorithm based on factor graph is proposed. The global optimum solution is factorized according to the chain structure of the factor graph, which allows for a more general form of the conditional probability density. It can convert the fusion matter into connecting factors defined by these measurements to the graph without considering the relationship between the sensor update frequency and the fusion period. An experimental MUAV system has been built and some experiments have been performed to prove the effectiveness of the proposed method.

Multi-Sensor Fusion with Interaction Multiple Model and Chi-Square Test Tolerant Filter.

Motivated by the key importance of multi-sensor information fusion algorithms in the state-of-the-art integrated navigation systems due to recent advancements in sensor technologies, telecommunication, and navigation systems, the paper proposes an improved and innovative fault-tolerant fusion framework. An integrated navigation system is considered consisting of four sensory sub-systems, i.e., Strap-down Inertial Navigation System (SINS), Global Navigation System (GPS), the Bei-Dou2 (BD2) and Celestial Navigation System (CNS) navigation sensors. In such multi-sensor applications, on the one hand, the design of an efficient fusion methodology is extremely constrained specially when no information regarding the system’s error characteristics is available. On the other hand, the development of an accurate fault detection and integrity monitoring solution is both challenging and critical. The paper addresses the sensitivity issues of conventional fault detection solutions and the unavailability of a precisely known system model by jointly designing fault detection and information fusion algorithms. In particular, by using ideas from Interacting Multiple Model (IMM) filters, the uncertainty of the system will be adjusted adaptively by model probabilities and using the proposed fuzzy-based fusion framework. The paper also addresses the problem of using corrupted measurements for fault detection purposes by designing a two state propagator chi-square test jointly with the fusion algorithm. Two IMM predictors, running in parallel, are used and alternatively reactivated based on the received information form the fusion filter to increase the reliability and accuracy of the proposed detection solution. With the combination of the IMM and the proposed fusion method, we increase the failure sensitivity of the detection system and, thereby, significantly increase the overall reliability and accuracy of the integrated navigation system. Simulation results indicate that the proposed fault tolerant fusion framework provides superior performance over its traditional counterparts.

Research on Target Information Fusion Identification Algorithm in Multi-Sky-Screen Measurement System

To solve the false targets influence identification problems in multi-sky-screen measurement system, this paper proposes multi-sensors fuzzy data fusion identification algorithm by using D–S data fusion recognition theory and the space and time relation that target pass through multi-sky-screen measurement system. According to space optical geometry of eight-sky-screen, this paper gives the calculation function of flight target location and target flight azimuth parameters, analyzes eight-sky-screens detection sensor output signal peak and signal pulse width characteristics, sets up target recognition propositional structures by using each detection screen’s output signal as a unit, researches eight-sky-screen detection target signal fusion identification processing algorithm, and derives the detection target signal fusion recognition calculation function based on support and weight values of multiple photoelectric detection sensor output signal characteristic. By using two target signals reasoning and calculation, the results verify the feasible and scientific of multi-sensors fuzzy data fusion algorithm and give the calculation process.

A noncontact positioning measuring system based on distributed wireless networks

A new type of distributed wireless network which combines a laser range finder with binocular vision sensors is developed to improve the accuracy of measurement along the direction of optical axis. By obtaining the coordinate of the target by the binocular vision sensor, the laser range finder which is installed at a two-axes rotary table is able to measure the distance between the target and the turntable of the current position. Then, an adaptive weighted fusion algorithm of multi-sensor information fusion is proposed to improve the utilization efficiency of the multi-sensor information and to make the results more accurate. Finally, the parameters of the system are calibrated through the simulations and the experiments show that the system is feasible and effective.

Multi-sensor Information Fusion Method Based on BP Neural Network

This study aims to solve the problem of multi-sensor information fusion, which is a key issue in the multi-sensor system development. The main innovation of this study is to propose a novel multi-sensor information fusion algorithm based on back propagation neural network and Bayesian inference. In the proposed algorithm, a triple is defined to represent a probability space; thereafter, the Bayesian inference is used to estimate the posterior expectation. Finally, we construct a simulation environment to test the performance of the proposed algorithm. Experimental results demonstrate that the proposed algorithm can significantly enhance the accuracy of temperature detection after fusing the data obtained from different sensors.

A Knowledge-Based Step Length Estimation Method Based on Fuzzy Logic and Multi-Sensor Fusion Algorithms for a Pedestrian Dead Reckoning System

The demand for pedestrian navigation has increased along with the rapid progress in mobile and wearable devices. This study develops an accurate and usable Step Length Estimation (SLE) method for a Pedestrian Dead Reckoning (PDR) system with features including a wide range of step lengths, a self-contained system, and real-time computing, based on the multi-sensor fusion and Fuzzy Logic (FL) algorithms. The wide-range SLE developed in this study was achieved by using a knowledge-based method to model the walking patterns of the user. The input variables of the FL are step strength and frequency, and the output is the estimated step length. Moreover, a waist-mounted sensor module has been developed using low-cost inertial sensors. Since low-cost sensors suffer from various errors, a calibration procedure has been utilized to improve accuracy. The proposed PDR scheme in this study demonstrates its ability to be implemented on waist-mounted devices in real time and is suitable for the indoor and outdoor environments considered in this study without the need for map information or any pre-installed infrastructure. The experiment results show that the maximum distance error was within 1.2% of 116.51 m in an indoor environment and was 1.78% of 385.2 m in an outdoor environment.

AUV Positioning Method Based on Tightly Coupled SINS/LBL for Underwater Acoustic Multipath Propagation.

This paper researches an AUV (Autonomous Underwater Vehicle) positioning method based on SINS (Strapdown Inertial Navigation System)/LBL (Long Base Line) tightly coupled algorithm. This algorithm mainly includes SINS-assisted searching method of optimum slant-range of underwater acoustic propagation multipath, SINS/LBL tightly coupled model and multi-sensor information fusion algorithm. Fuzzy correlation peak problem of underwater LBL acoustic propagation multipath could be solved based on SINS positional information, thus improving LBL positional accuracy. Moreover, introduction of SINS-centered LBL locating information could compensate accumulative AUV position error effectively and regularly. Compared to loosely coupled algorithm, this tightly coupled algorithm can still provide accurate location information when there are fewer than four available hydrophones (or within the signal receiving range). Therefore, effective positional calibration area of tightly coupled system based on LBL array is wider and has higher reliability and fault tolerance than loosely coupled. It is more applicable to AUV positioning based on SINS/LBL.

An Innovative Asynchronous, Multi-rate, Multi-sensor State Vector Fusion Algorithm for Air Defence Applications

Abstract This paper presents an innovative approach for data fusion of asynchronous, multi-rate, multi-sensor data in real time. This technique is based on state vector data fusion. For air defence applications, the target information from multi sensors is sent to a central control centre to create a complete threat scenario which would be used for taking appropriate decision. This target data needs to be sent in real time, hence there is a need for low bandwidth consuming data fusion algorithms but at the same time there should be minimal degradation in data accuracy. The optimal fusion algorithm requires the entire state covariance information matrix to be transmitted to the data fusion centre, however this requires huge bandwidth for transmission of data. To avoid this generally, only diagonal elements are transmitted at the cost of considerable degradation in performance. The authors present an innovative approach by considering the only covariance of position, velocity and acceleration but by neglecting the cross covariance terms among position, velocity and acceleration. The accuracy of this proposed algorithm is about 95% of that of optimal method but the reduction in bandwidth requirement is of the order of 60%.

Decision-Making Algorithm for Multisensor Fusion Based on Grey Relation and DS Evidence Theory

Decision-making algorithm, as the key technology for uncertain data fusion, is the core to obtain reasonable multisensor information fusion results. DS evidence theory is a typical and widely applicable decision-making method. However, DS evidence theory makes decisions without considering the sensors’ difference, which may lead to illogical results. In this paper, we present a novel decision-making algorithm for uncertain fusion based on grey relation and DS evidence theory. The proposed algorithm comprehensively takes consideration of sensor’s credibility and evidence’s overall discriminability, which can solve the uncertainty problems caused by inconsistence of sensors themselves and complexity of monitoring environment and simultaneously ensure the validity and accuracy of fusion results. The innovative decision-making algorithm firstly obtains the sensor’s credibility through the introduction of grey relation theory and then defines two impact factors as sensor’s credibility and evidence’s overall discriminability according to the focal element analyses and evidence’s distance analysis, respectively; after that, it uses the impact factors to modify the evidences and finally gets more reasonable and effective results through DS combination rule. Simulation results and analyses demonstrate that the proposed algorithm can overcome the trouble caused by large evidence conflict and one-vote veto, which indicates that it can improve the ability of target judgment and enhance precision of uncertain data fusion. Thus the novel decision-making method has a certain application value.

AUV Underwater Positioning Algorithm Based on Interactive Assistance of SINS and LBL.

This paper studies an underwater positioning algorithm based on the interactive assistance of a strapdown inertial navigation system (SINS) and LBL, and this algorithm mainly includes an optimal correlation algorithm with aided tracking of an SINS/Doppler velocity log (DVL)/magnetic compass pilot (MCP), a three-dimensional TDOA positioning algorithm of Taylor series expansion and a multi-sensor information fusion algorithm. The final simulation results show that compared to traditional underwater positioning algorithms, this scheme can not only directly correct accumulative errors caused by a dead reckoning algorithm, but also solves the problem of ambiguous correlation peaks caused by multipath transmission of underwater acoustic signals. The proposed method can calibrate the accumulative error of the AUV position more directly and effectively, which prolongs the underwater operating duration of the AUV.

A situation assessment method for rock burst based on multi-agent information fusion

An assessment approach to assess the likelihood of rock burst in coal mines by integrating the Multi-Agent System with data fusion techniques is proposed in this paper. We discuss an optimal algorithm for multi-sensor data fusion to improve the accuracy and reliability of the source data. Some model-based situation quantization methods are described and a rock burst situation quantitative assessment model incorporating improved Dempster–Shafer theory is presented. The Auto-Regressive, Moving Average and Holt–Winters models are used to address indefinability and inaccuracy of the prediction. A case study demonstrates that the proposed situation assessment model is capable of producing relatively accurate forecasts, and thereby it can provide coal mine decision-makers with an overview of the development of rock bursts.

DEVELOPMENT OF A PEDESTRIAN INDOOR NAVIGATION SYSTEM BASED ON MULTI-SENSOR FUSION AND FUZZY LOGIC ESTIMATION ALGORITHMS

Abstract. This paper presents a pedestrian indoor navigation system based on the multi-sensor fusion and fuzzy logic estimation algorithms. The proposed navigation system is a self-contained dead reckoning navigation that means no other outside signal is demanded. In order to achieve the self-contained capability, a portable and wearable inertial measure unit (IMU) has been developed. Its adopted sensors are the low-cost inertial sensors, accelerometer and gyroscope, based on the micro electro-mechanical system (MEMS). There are two types of the IMU modules, handheld and waist-mounted. The low-cost MEMS sensors suffer from various errors due to the results of manufacturing imperfections and other effects. Therefore, a sensor calibration procedure based on the scalar calibration and the least squares methods has been induced in this study to improve the accuracy of the inertial sensors. With the calibrated data acquired from the inertial sensors, the step length and strength of the pedestrian are estimated by multi-sensor fusion and fuzzy logic estimation algorithms. The developed multi-sensor fusion algorithm provides the amount of the walking steps and the strength of each steps in real-time. Consequently, the estimated walking amount and strength per step are taken into the proposed fuzzy logic estimation algorithm to estimates the step lengths of the user. Since the walking length and direction are both the required information of the dead reckoning navigation, the walking direction is calculated by integrating the angular rate acquired by the gyroscope of the developed IMU module. Both the walking length and direction are calculated on the IMU module and transmit to a smartphone with Bluetooth to perform the dead reckoning navigation which is run on a self-developed APP. Due to the error accumulating of dead reckoning navigation, a particle filter and a pre-loaded map of indoor environment have been applied to the APP of the proposed navigation system to extend its usability. The experiment results of the proposed navigation system demonstrate good navigation performance in indoor environment with the accurate initial location and direction.

Particle filter robot localisation through robust fusion of laser, WiFi, compass, and a network of external cameras

In this paper, we propose a multi-sensor fusion algorithm based on particle filters for mobile robot localisation in crowded environments. Our system is able to fuse the information provided by sensors placed on-board, and sensors external to the robot (off-board). We also propose a methodology for fast system deployment, map construction, and sensor calibration with a limited number of training samples. We validated our proposal experimentally with a laser range-finder, a WiFi card, a magnetic compass, and an external multi-camera network. We have carried out experiments that validate our deployment and calibration methodology. Moreover, we performed localisation experiments in controlled situations and real robot operation in social events. We obtained the best results from the fusion of all the sensors available: the precision and stability was sufficient for mobile robot localisation. No single sensor is reliable in every situation, but nevertheless our algorithm works with any subset of sensors: if a sensor is not available, the performance just degrades gracefully.