Multi-sensor Data Fusion Method Research Articles

Multisensor data fusion of operational sea ice observations

Multisensor data fusion (MDF) is a process/technique of combining observations from multiple sensors to provide a more robust, accurate and complete description of the concerned object, environment or process. In this paper we introduce a new MDF method, multisensor optimal data fusion (MODF), to fuse different operational sea ice observations around Svalbard. The overall MODF includes regridding, univariate multisensor optimal data merging (MODM), multivariate check of consistency, and generation of new variables. For MODF of operational sea ice observations around Svalbard, the AMSR2 sea ice concentration (SIC) is firstly merged with the Norwegian Meteorological Institute ice chart. Then the daily SMOS sea ice thickness (SIT) is merged with the weekly CS2SMOS SIT to form a daily CS2SMOS SIT, which is further refined to be consistent with the SIC through consistency check. Finally sea ice volume (SIV) and its uncertainty are calculated based on the merged SIC and fused SIT. The fused products provide an improved, united, consistent and multifaceted description for the operational sea ice observations, they also provide consistent descriptions of sea ice edge and marginal ice zone. We note that uncertainties may vary during the regridding process, and therefore correct determination of the observation uncertainties is critically important for MDF. This study provides a basic framework for managing multivariate multisensor observations.

Bearing fault diagnosis based on a Multiple-Constraint Modal-invariant Graph Convolutional Fusion Network

Multisensor data fusion method can improve the accuracy of bearing fault diagnosis, in order to address the problems of single-sensor data types and the insufficient exploration of redundancy and complementarity between different modal data in most existing multisensor data fusion methods for bearing fault diagnosis, a bearing fault diagnosis method based on a Multiple-Constraint Modal-Invariant Graph Convolutional Fusion Network (MCMI-GCFN) is proposed in this paper. Firstly, a Convolutional Autoencoder (CAE) and Squeeze-and-Excitation Block (SE block) are used to extract features of raw current and vibration signals. Secondly, the model introduces source domain classifiers and domain discriminators to capture modal invariance between different modal data based on domain adversarial training, making use of the redundancy and complementarity between multimodal data. Then, the spatial aggregation property of Graph Convolutional Neural Networks (GCN) is utilized to capture the dependency relationship between current and vibration modes with similar time step features for accurately fusing contextual semantic information. Finally, the validation is conducted on the public bearing damage current and vibration dataset from Paderborn University. The experimental results showed that the delivered fusion method achieved a bearing fault diagnosis accuracy of 99.6 %, which was about 9–11.4 % better than that with nonfusion methods.

Deep Learning Enhanced Multisensor Data Fusion for Building Assessment Using Multispectral Voxels and Self-Organizing Maps

Efforts in the domain of building studies involve the use of a diverse array of geomatic sensors, some providing invaluable information in the form of three-dimensional point clouds and associated registered properties. However, managing the vast amounts of data generated by these sensors presents significant challenges. To ensure the effective use of multisensor data in the context of cultural heritage preservation, it is imperative that multisensor data fusion methods be designed in such a way as to facilitate informed decision-making by curators and stakeholders. We propose a novel approach to multisensor data fusion using multispectral voxels, which enable the application of deep learning algorithms as the self-organizing maps to identify and exploit the relationships between the different sensor data. Our results indicate that this approach provides a comprehensive view of the building structure and its potential pathologies, and holds great promise for revolutionizing the study of historical buildings and their potential applications in the field of cultural heritage preservation.

The Multisensor Data Fusion Method Based on Improved Fuzzy Evidence Theory in the Coal Mine Environment

An enhanced evidence theory-based multisensor data fusion technique is presented to address the problem of poor data fusion caused by an unknown interference in the fully automated mining face multisensor system of a coal mine. Initially, the set of all measurement values is considered as the identification framework, and the principles of fuzzy mathematics are applied to introduce the membership function. This leads to the proposal of a novel method for calculating mutual support among multiple sensors. Furthermore, the basic belief assignment (BBA) in evidence theory is determined by measuring the confidence distance between sensors. Subsequently, a divergence measure is employed to assess the level of conflict and difference between BBA functions, which serves as an indicator of their credibility. The credibility of BBA functions is further adjusted by calculating their information volume using Shannon entropy. This adjustment aims to increase the weight of BBA functions that exhibit less conflict with other BBA functions. Ultimately, the fusion result is obtained through an evidence combination rule based on a conflict allocation. The numerical experimental results demonstrate that the proposed approach achieves higher accuracy, better robustness, and generality compared to the existing methods.

Perception Methods for Adverse Weather Based on Vehicle Infrastructure Cooperation System: A Review.

Environment perception plays a crucial role in autonomous driving technology. However, various factors such as adverse weather conditions and limitations in sensing equipment contribute to low perception accuracy and a restricted field of view. As a result, intelligent connected vehicles (ICVs) are currently only capable of achieving autonomous driving in specific scenarios. This paper conducts an analysis of the current studies on image or point cloud processing and cooperative perception, and summarizes three key aspects: data pre-processing methods, multi-sensor data fusion methods, and vehicle-infrastructure cooperative perception methods. Data pre-processing methods summarize the processing of point cloud data and image data in snow, rain and fog. Multi-sensor data fusion methods analyze the studies on image fusion, point cloud fusion and image-point cloud fusion. Because communication channel resources are limited, the vehicle-infrastructure cooperative perception methods discuss the fusion and sharing strategies for cooperative perception information to expand the range of perception for ICVs and achieve an optimal distribution of perception information. Finally, according to the analysis of the existing studies, the paper proposes future research directions for cooperative perception in adverse weather conditions.

Multi-Sensor Data Fusion Method Based on Self-Attention Mechanism

In 3D reconstruction tasks, single-sensor data fusion based on deep learning is limited by the integrity and accuracy of the data, which reduces the accuracy and reliability of the fusion results. To address this issue, this study proposes a multi-sensor data fusion method based on a self-attention mechanism. A multi-sensor data fusion model for acquiring multi-source and multi-modal data is constructed, with the core component being a convolutional neural network with self-attention (CNN-SA), which employs CNNs to process multi-source and multi-modal data by extracting their features. Additionally, it introduces an SA mechanism to weigh and sum the features of different modalities, adaptively focusing on the importance of different modal data. This enables mutual support, complementarity, and correction among the multi-modal data. Experimental results demonstrate that the accuracy of the CNN-SA network is improved by 72.6%, surpassing the improvements of 29.9% for CNN-CBAM, 23.6% for CNN, and 11.4% for CNN-LSTM, exhibiting enhanced generalization capability, accuracy, and robustness. The proposed approach will contribute to the effectiveness of multi-sensor data fusion processing.

A novel collaborative bearing fault diagnosis method based on multi-scale dynamic fusion network under speed fluctuating condition

Improving bearing fault diagnosis accuracy under speed fluctuation is a challenge in engineering applications. With the development of big data processing technology, a new solution, multi-sensor complementary information, has emerged. However, single-scale dimension compression, which is adopted in most multi-sensor data fusion methods, captures only a small amount of valuable information. To deal with this deficiency, a multi-scale dynamic fusion network (MSDFN) is proposed. First, considering the existence of non-stationary features in the fluctuating speed signal, the FReLU function is adopted to activate the features after considering contextual information. Then, multi-sensor features are fused by multiple scales to obtain richer feature information, and fusion features at different scales are weighted by using the attention mechanism. Finally, batch normalization is employed to standardize the variable speed feature distribution. The validity of the MSDFN is proved by conducting fault diagnosis experiments on two bearings under speed fluctuating conditions. Experimental results indicate that the MSDFN is not only effective in identifying various types of fault samples, but also shows higher stability in multiple trials when compared with other methods.

A Fuzzy Dempster–Shafer Evidence Theory Method with Belief Divergence for Unmanned Surface Vehicle Multi-Sensor Data Fusion

The safe navigation of unmanned surface vehicles in the marine environment requires multi-sensor collaborative perception, and multi-sensor data fusion technology is a prerequisite for realizing the collaborative perception of different sensors. To address the problem of poor fusion accuracy for existing multi-sensor fusion methods without prior knowledge, a fuzzy evidence theory multi-sensor data fusion method with belief divergence is proposed in this paper. First of all, an adjustable distance for measuring discrepancies between measurements is devised to evaluate the degree of measurement closeness to the true value, which improves the adaptability of the method to different classes of sensor data. Furthermore, an adaptive multi-sensor measurement fusion strategy is designed for the case where the sensor accuracy is known in advance. Secondly, the affiliation function of the fuzzy theory is introduced into the evidence theory approach to assign initial evidence of measurements in terms of defining the degree of fuzzy support between measurements, which improves the fusion accuracy of the method. Finally, the belief Jensen–Shannon divergence and the Rényi divergence are combined for measuring the conflict between the evidence pieces to obtain the credibility degree as the reliability of the evidence, which solves the problem of high conflict between evidence pieces. Three examples of multi-sensor data fusion in different domains are employed to validate the adaptability of the proposed method to different kinds of multi-sensors. The maximum relative error of the proposed method for multiple sensor experiments is greater than or equal to 0.18%, and its error accuracy is much higher than the best result of 0.46% among other comparative methods. The experimental results verify that the proposed data fusion method is more accurate than other existing methods.

Multi-sensor data fusion method based on divergence measure and probability transformation belief factor

Dempster–Shafer evidence theory is widely used in multi-sensor data fusion. However, how to manage the counterintuitive result generated by the highly conflicting evidence remains an open question. To solve the problem, a novel multi-sensor data fusion method is proposed, which analyses the credibility of evidence from both the discrepancy between evidences and the factors of evidence itself. Firstly, a new Belief Kullback–Leibler divergence is put forward, which evaluates the credibility of evidence from the discrepancy between evidences. Secondly, another credibility measure called the Probability Transformation Belief Factor is defined, which assesses the credibility of evidence from the evidence itself. These two credibilities are combined as the comprehensive credibility of evidence. Furthermore, considering the uncertainty of evidence, a new belief entropy based on the cross-information within the evidence is presented, which is applied to quantify the information volume of evidence and to adjust the comprehensive credibility of evidence. The adjusted comprehensive credibility is regarded as the final weight to modify the body of evidence. Finally, the Dempster’s combination rule is applied for fusion. Experiment and applications show that the proposed method is effective and superior.

Multi-Sensor Data Fusion Method Based on Improved Evidence Theory

To achieve autonomous navigation in complex marine environments, unmanned surface vehicles are equipped with a variety of sensors for sensing the surrounding environment and their own state. To address the issue of unsatisfactory multi-sensor information fusion in stochastic uncertain systems with unknown disturbances, an improved evidence theory multi-sensor data fusion method is proposed in this article. First, the affiliation function in fuzzy set theory is introduced as a support function to assign initial evidence for multi-sensor data, and the initial evidence is corrected according to the degree of data bias. Second, a divergence measure is employed to measure the degree of conflict and discrepancy among the evidence, and each piece of evidence is allocated proportional weight based on the conflict allocation principle. Finally, the evidence is synthesized through the evidence combination rule, and the data are weighted and summed to obtain the data fusion results. Since it is difficult to obtain dynamic information from multiple sensors carried by unmanned surface vehicles in practical applications, and considering that the proposed method has universal applicability, practical application experiments using previous research demonstrate that the proposed method has higher fusion accuracy than other existing data fusion methods.

Multi-sensor InSAR time series fusion for long-term land subsidence monitoring

ABSTRACT Satellite Interferometric Synthetic Aperture Radar (InSAR) is widely used for topographic, geological and natural resource investigations. However, most of the existing InSAR studies of ground deformation are based on relatively short periods and single sensors. This paper introduces a new multi-sensor InSAR time series data fusion method for time-overlapping and time-interval datasets, to address cases when partial overlaps and/or temporal gaps exist. A new Power Exponential Knothe Model (PEKM) fits and fuses overlaps in the deformation curves, while a Long Short-Term Memory (LSTM) neural network predicts and fuses any temporal gaps in the series. Taking the city of Wuhan (China) as experiment area, COSMO-SkyMed (2011–2015), TerraSAR-X (2015–2019) and Sentinel-1 (2019–2021) SAR datasets were fused to map long-term surface deformation over the last decade. An independent 2011–2020 InSAR time series analysis based on 230 COSMO-SkyMed scenes was also used as reference for comparison. The correlation coefficient between the results of the fusion algorithm and the reference data is 0.87 in the time overlapping region and 0.97 in the time-interval dataset. The correlation coefficient of the overall results is 0.78, which fully demonstrates that the algorithm proposed in our paper achieves a similar trend as the reference deformation curve. The experimental results are consistent with existing studies of surface deformation at Wuhan, demonstrating the accuracy of the proposed new fusion method to provide robust time series for the analysis of long-term land subsidence mechanisms.

The Application Of Multi-Sensor Data Fusion Method with Fuzzy Time Series Model to Improve Indoor Water Prediction Accuracy Quality

There is a lot of indoor air pollution, especially from cigarette smoke, wall paint, air fresheners and gas. With this situation, the room uses Air Box WP6003 air quality detection device by transmitting information about air quality through visualization index. This study aims to improve prediction accuracy with fuzzy time series methods processed through 2 naïve and moving average models using forecast transformers and without transformers. The level of prediction accuracy is calculated through several metrics, namely Mean Absolute Percentage Error (MAPE), Sum of Squares Error (SSE), Mean Square Error (MSE), Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). These results can be calculated between the actual value and the predicted value. The data used is 204584 data from 4 parameters including Temperature, TVOC, HCHO and CO2. The test results with the difference from the forecast transformer and without transformer are comparable. Temperature value obtained using naïve with transformer from RMSE of 0.158866 and naïve without transformer of 0.782397, data using moving average with transformer obtained by 0.147546 and moving average without transformer of 0.772570. This can be explained by the error analysis that was tried, where the error rate continued to increase so that the experimental results continued to be far from the actual number. From the test results it can be concluded that the accuracy of air quality prediction using naïve forecast transformer is pretty accurate.

An Innovative multi-sensor data fusion method based on thrice deeply-fusion architecture for multi-object tracking

For the purpose of track maintenance in a cluttered environment with some false alarms, false dismissals and measurement disturbances, this paper presents a new multi-object tracking procedure with uncertainty of the source of the measurement data returned from multi-sensor. The tracking method is a thrice deeply-fusion approach constructed by a primary fusion based on an improved probabilistic data association filter (IPDAF), a secondary fusion with historical motion trajectories, and a thrice fusion with road-markings. It incorporates the existence probabilities of the individual tracks and the variable number of objects based on the Bayesian estimation theory, which can improve the tracking performances effectively in an environment with high clutter density. A binary 2-D assignment is adopted for the optimal data association, which is established as a nonlinear optimization problem. In the motion modeling, it introduces multiple measurement models for different sensors into the method. Then, the estimation could be performed with greater reliability. The computational efficiency is satisfying and it can be used for real-time application, which is verified by two real test scenarios.

A Multisensor Data Fusion Based Anomaly Detection (Ammonia Nitrogen) Approach for Ensuring Green Coastal Environment

Great changes have been brought about by the coastal environment when the economy develops rapidly. Coastal environmental monitoring is the basis and technical guarantee for coastal environmental protection supervision and management. It is one of the important tasks to detect and timely discover coastal seawater anomalies. Usually, a single sensor cannot determine whether the coastal environment or ship operation is an anomaly. Recently, an unmanned surface vehicle for coastal environment monitoring was developed, and stacked autoencoders are used for seawater anomaly detection using multisensor data fusion methods. The multisensor data of pH, conductivity, and ammonia nitrogen are employed to judge the anomaly of seawater. The mean, standard deviation, mean square root, and normalized power spectrum features of multisensor data are extracted, and a stacked autoencoder is employed to fuse these features for anomaly detection. The proposed method is feasible and effective for anomaly detection of coastal water quality and ship operation. Compared with other commonly used methods, the proposed method has a higher recall, precision, and F1 score performance.

Karst Collapse Monitoring and Early Warning Evaluation Method Based on Multisensor Internet of Things.

The international community has paid extensive attention to the numerous engineering problems faced by karst areas caused by the increasingly frequent human activities. China has a wide variety of karst forms. Among them, carbonate karst is the most widely distributed, and the development of carbonate karst is relatively strong in many areas. Countless property losses are caused by karst disasters every year. This article aims to study the real-time monitoring and timely early warning of karst collapse through the use of multisensor Internet of Things technology. To this end, this article proposes an improved method for multisensor data fusion. It optimizes and improves the transmission and delivery efficiency of its data. This makes the improved multisensor more in line with the research content of this article in terms of monitoring efficiency. At the same time, related experiments and analyses are designed to compare and analyze the karst collapse and the monitoring efficiency of the sensor. The experimental results of this article show that after the improvement, the anti-interference ability of the monitoring system is increased by 34%. The frequency of early warning has also been improved by 24%, which has high practical application value.

Multi-Sensor Data Fusion Approach for Kinematic Quantities

A theoretical framework to implement multi-sensor data fusion methods for kinematic quantities is proposed. All methods defined through the framework allow the combination of signals obtained from position, velocity and acceleration sensors addressing the same target, and improvement in the observation of the kinematics of the target. Differently from several alternative methods, the considered ones need no dynamic and/or error models to operate and can be implemented with low computational burden. In fact, they gain measurements by summing filtered versions of the heterogeneous kinematic quantities. In particular, in the case of position measurement, the use of filters with finite impulse responses, all characterized by finite gain throughout the bandwidth, in place of straightforward time-integrative operators, prevents the drift that is typically produced by the offset and low-frequency noise affecting velocity and acceleration data. A simulated scenario shows that the adopted method keeps the error in a position measurement, obtained indirectly from an accelerometer affected by an offset equal to 1 ppm on the full scale, within a few ppm of the full-scale position. If the digital output of the accelerometer undergoes a second-order time integration, instead, the measurement error would theoretically rise up to 12n(n+1) ppm in the full scale at the n-th discrete time instant. The class of methods offered by the proposed framework is therefore interesting in those applications in which the direct position measurements are characterized by poor accuracy and one has also to look at the velocity and acceleration data to improve the tracking of a target.

Bearing Fault Diagnosis of Switched Reluctance Motor in Electric Vehicle Powertrain via Multisensor Data Fusion

A multisensor data fusion method is investigated for bearing fault diagnosis of a switched reluctance motor (SRM) of an electric vehicle (EV) powertrain under varying speed conditions. The accumulative rotation angle of the SRM rotor is estimated by fusing the synchronous sampled current and vibration signals. The time-domain vibration signal is then resampled on an angular domain, and the bearing fault type is identified on the envelope spectrum of the resampled signal. In this article, an experimental setup is designed to validate the performance of the proposed method compared with the traditional ones. The practical EV working conditions including driving, coasting, and braking are considered in the experiments. Results indicated that the proposed method successfully diagnoses the SRM bearing faults under random and complex conditions. The method is promising for online SRM fault diagnosis under varying speed conditions as it requires no extra tachometer, specifically when the sensorless control strategy is adopted.

A Study of Weighted Average Method for Multi-sensor Data Fusion

A Study of Weighted Average Method for Multi-sensor Data Fusion

Indirect tool wear measurement and prediction using multi-sensor data fusion and neural network during machining

The quality produced of the machined workpiece is affected by various machining parameters including the tool wear. The condition of tool can be monitored by analyzing various machining responses through sensors. The use of multiple sensors allows for a comparison of data and information gathered from various sources, assisting in deciding the state of the tool and the workpiece. In current work, multiple sensors are attached to a lathe during the machining of AISI 4140. This paper presents a multi-sensor data fusion method for flank wear measurement and prediction using various parameters such as vibration, power, temperature, force, and surface roughness. Dry machining is performed by using PVD coated tool insert, and the flank wear is analyzed. Experiments are performed with a full factorial experimental design L27 orthogonal array. Taguchi approach has been utilized to evaluate the effect of machining parameters on tool wear. Also, the tool wear is predicted through an artificial neural network (ANN) model, and the results are compared with manually measured data. The results showcased the effectiveness of the proposed tool wear prediction model with the least measurement error. The work presents a novel neural network approach for the accurate prediction of tool wear and hence eliminates the necessity of manual measurement of tool wear and leading towards the increased productivity and quality of the work part.

Multi-sensor fusion method for roadheader pose detection

Pose detection of roadheader has become an important topic in the field of mine automation. This paper presents a method of multi sensor data fusion for the detection of roadheader fuselage pose. The pose detection system consists of a cross laser transmitter, several photoelectric sensors and an inclination sensor. The laser transmitter is used to create reference coordinate frame. The photoelectric sensors and inclination sensor are used to acquire data for the calculation of roadheader pose. The spatial algorithm for the pose parameters of roadheader is derived by fusing the measurement data of photoelectric sensors and inclination sensor. A test platform is built to verify the performance of the pose estimation system. The experimental results show that by using multi-sensor data fusion method, the pose estimation system has higher measurement accuracy and can meet the requirements of engineering application.