Multi-sensor Data Fusion Method Research Articles

Exploring Trends and Clusters in Human Posture Recognition Research: An Analysis Using CiteSpace

This study delves into interdisciplinary research directions in human posture recognition, covering vision-based and non-vision-based methods. Visually analyzing 3066 core research papers published from 2011 to 2024 with CiteSpace software reveals knowledge structures, research topics, key documents, trends, and institutional contributions. In-depth citation analysis identified 1200 articles and five significant research clusters. Findings show that in recent years, deep learning and sensor-based methods have dominated, significantly improving recognition accuracy, like the deep learning-based posture recognition method achieving 99.7% verification set accuracy with a 20-ms delay in a controlled environment. Logarithmic growth analysis of annual publications, supported by logistic model fitting, indicates the field’s maturation since 2011, with a shift from early simple applications of traditional and deep learning algorithms to integrating interdisciplinary approaches for problem-solving as the field matures and a predicted decline in future breakthroughs. By integrating indicators like citation bursts, degree centrality, and sigma, the research identifies interdisciplinary trends and key innovation directions, showing a transition from traditional to deep learning and multi-sensor data fusion methods. The integration of biomechanics principles with engineering technologies highlights new research paths. Overall, this study offers a systematic overview to identify gaps, trends, and innovation directions, facilitating future research and providing a roadmap for innovation in human posture recognition.

A novel multi-sensor data fusion enabled health indicator construction and remaining useful life prediction of aero-engine

Remaining useful life (RUL) prediction is vital to formulate a suitable maintenance strategy in manufacturing systems health management. Multisensor data fusion of complex engineering systems has attracted substantial attention due to the fact that a single sensor can only collect partial information. Health indicator (HI) construction plays a crucial role in multisensor data fusion and machinery prognostic, mainly because it attempts to quantify a history and ongoing degradation process by fusing the advantages of multiple sensors. However, large numbers of coefficients are involved for most of the existing HIs. Additionally, simplifications during modeling may inhibit the wide application of the constructed HI. To address these two challenges, a new multisensor data fusion method is proposed in this paper by constructing a HI for the characterization of the degradation process. Firstly, the sensors that collect invalid data or conflicting data are removed through a correlation coefficient operation. Then, principal component analysis (PCA) is adopted to reduce the number of coefficient before constructing the HI. Furthermore, the objective function is constructed under the comprehensive consideration of the three factors of the HI, that is, monotonicity, trendability, and fitting errors. The effectiveness of the proposed method is verified using the C-MAPSS dataset. Multiple comparison results show that the HI possesses excellent performance in both degradation characterization and remaining useful life prediction.

Real-time multi-sensor fusion for object detection and localization in self-driving cars: A Carla simulation

Research on integrating camera and LiDAR in self-driving car systems has important scientific significance in the context of developing 4.0 technology and applying artificial intelligence. The research contributes to improving the accuracy in recognizing and locating objects in complex environments. This is an important foundation for further research on optimizing response time and improving the safety of self-driving systems. This study proposes a real-time multi-sensor data fusion method, termed "Multi-Layer Fusion," for object detection and localization in autonomous vehicles. The fusion process leverages pixel-level and feature-level integration, ensuring seamless data synchronization and robust performance under adverse conditions. Experiments conducted on the CARLA simulator. The results show that the method significantly improves environmental perception and object localization, achieving a mean detection accuracy of 95% and a mean distance error of 0.54 meters across diverse conditions, with real-time performance at 30 FPS. These results demonstrate its robustness in both ideal and adverse scenarios

Bearing fault diagnosis based on a multiple-constraint modal-invariant graph convolutional fusion network

Multisensor data fusion method can improve the accuracy of bearing fault diagnosis, in order to address the problems of single-sensor data types and the insufficient exploration of redundancy and complementarity between different modal data in most existing multisensor data fusion methods for bearing fault diagnosis, a bearing fault diagnosis method based on a Multiple-Constraint Modal-Invariant Graph Convolutional Fusion Network (MCMI-GCFN) is proposed in this paper. Firstly, a Convolutional Autoencoder (CAE) and Squeeze-and-Excitation Block (SE block) are used to extract features of raw current and vibration signals. Secondly, the model introduces source domain classifiers and domain discriminators to capture modal invariance between different modal data based on domain adversarial training, making use of the redundancy and complementarity between multimodal data. Then, the spatial aggregation property of Graph Convolutional Neural Networks (GCN) is utilized to capture the dependency relationship between current and vibration modes with similar time step features for accurately fusing contextual semantic information. Finally, the validation is conducted on the public bearing damage current and vibration dataset from Paderborn University. The experimental results showed that the delivered fusion method achieved a bearing fault diagnosis accuracy of 99.6 %, which was about 9 %–11.4 % better than that with nonfusion methods.

Multisensor data fusion of operational sea ice observations

Multisensor data fusion (MDF) is a process/technique of combining observations from multiple sensors to provide a more robust, accurate and complete description of the concerned object, environment or process. In this paper we introduce a new MDF method, multisensor optimal data fusion (MODF), to fuse different operational sea ice observations around Svalbard. The overall MODF includes regridding, univariate multisensor optimal data merging (MODM), multivariate check of consistency, and generation of new variables. For MODF of operational sea ice observations around Svalbard, the AMSR2 sea ice concentration (SIC) is firstly merged with the Norwegian Meteorological Institute ice chart. Then the daily SMOS sea ice thickness (SIT) is merged with the weekly CS2SMOS SIT to form a daily CS2SMOS SIT, which is further refined to be consistent with the SIC through consistency check. Finally sea ice volume (SIV) and its uncertainty are calculated based on the merged SIC and fused SIT. The fused products provide an improved, united, consistent and multifaceted description for the operational sea ice observations, they also provide consistent descriptions of sea ice edge and marginal ice zone. We note that uncertainties may vary during the regridding process, and therefore correct determination of the observation uncertainties is critically important for MDF. This study provides a basic framework for managing multivariate multisensor observations.

Deep Learning Enhanced Multisensor Data Fusion for Building Assessment Using Multispectral Voxels and Self-Organizing Maps

Efforts in the domain of building studies involve the use of a diverse array of geomatic sensors, some providing invaluable information in the form of three-dimensional point clouds and associated registered properties. However, managing the vast amounts of data generated by these sensors presents significant challenges. To ensure the effective use of multisensor data in the context of cultural heritage preservation, it is imperative that multisensor data fusion methods be designed in such a way as to facilitate informed decision-making by curators and stakeholders. We propose a novel approach to multisensor data fusion using multispectral voxels, which enable the application of deep learning algorithms as the self-organizing maps to identify and exploit the relationships between the different sensor data. Our results indicate that this approach provides a comprehensive view of the building structure and its potential pathologies, and holds great promise for revolutionizing the study of historical buildings and their potential applications in the field of cultural heritage preservation.

The Multisensor Data Fusion Method Based on Improved Fuzzy Evidence Theory in the Coal Mine Environment

An enhanced evidence theory-based multisensor data fusion technique is presented to address the problem of poor data fusion caused by an unknown interference in the fully automated mining face multisensor system of a coal mine. Initially, the set of all measurement values is considered as the identification framework, and the principles of fuzzy mathematics are applied to introduce the membership function. This leads to the proposal of a novel method for calculating mutual support among multiple sensors. Furthermore, the basic belief assignment (BBA) in evidence theory is determined by measuring the confidence distance between sensors. Subsequently, a divergence measure is employed to assess the level of conflict and difference between BBA functions, which serves as an indicator of their credibility. The credibility of BBA functions is further adjusted by calculating their information volume using Shannon entropy. This adjustment aims to increase the weight of BBA functions that exhibit less conflict with other BBA functions. Ultimately, the fusion result is obtained through an evidence combination rule based on a conflict allocation. The numerical experimental results demonstrate that the proposed approach achieves higher accuracy, better robustness, and generality compared to the existing methods.

Perception Methods for Adverse Weather Based on Vehicle Infrastructure Cooperation System: A Review.

Environment perception plays a crucial role in autonomous driving technology. However, various factors such as adverse weather conditions and limitations in sensing equipment contribute to low perception accuracy and a restricted field of view. As a result, intelligent connected vehicles (ICVs) are currently only capable of achieving autonomous driving in specific scenarios. This paper conducts an analysis of the current studies on image or point cloud processing and cooperative perception, and summarizes three key aspects: data pre-processing methods, multi-sensor data fusion methods, and vehicle-infrastructure cooperative perception methods. Data pre-processing methods summarize the processing of point cloud data and image data in snow, rain and fog. Multi-sensor data fusion methods analyze the studies on image fusion, point cloud fusion and image-point cloud fusion. Because communication channel resources are limited, the vehicle-infrastructure cooperative perception methods discuss the fusion and sharing strategies for cooperative perception information to expand the range of perception for ICVs and achieve an optimal distribution of perception information. Finally, according to the analysis of the existing studies, the paper proposes future research directions for cooperative perception in adverse weather conditions.

Algebraic method for multisensor data fusion.

In this contribution, we use Gaussian posterior probability densities to characterize local estimates from distributed sensors, and assume that they all belong to the Riemannian manifold of Gaussian distributions. Our starting point is to introduce a proper Lie algebraic structure for the Gaussian submanifold with a fixed mean vector, and then the average dissimilarity between the fused density and local posterior densities can be measured by the norm of a Lie algebraic vector. Under Gaussian assumptions, a geodesic projection based algebraic fusion method is proposed to achieve the fused density by taking the norm as the loss. It provides a robust fixed point iterative algorithm for the mean fusion with theoretical convergence, and gives an analytical form for the fused covariance matrix. The effectiveness of the proposed fusion method is illustrated by numerical examples.

Multi-Sensor Data Fusion Method Based on Self-Attention Mechanism

In 3D reconstruction tasks, single-sensor data fusion based on deep learning is limited by the integrity and accuracy of the data, which reduces the accuracy and reliability of the fusion results. To address this issue, this study proposes a multi-sensor data fusion method based on a self-attention mechanism. A multi-sensor data fusion model for acquiring multi-source and multi-modal data is constructed, with the core component being a convolutional neural network with self-attention (CNN-SA), which employs CNNs to process multi-source and multi-modal data by extracting their features. Additionally, it introduces an SA mechanism to weigh and sum the features of different modalities, adaptively focusing on the importance of different modal data. This enables mutual support, complementarity, and correction among the multi-modal data. Experimental results demonstrate that the accuracy of the CNN-SA network is improved by 72.6%, surpassing the improvements of 29.9% for CNN-CBAM, 23.6% for CNN, and 11.4% for CNN-LSTM, exhibiting enhanced generalization capability, accuracy, and robustness. The proposed approach will contribute to the effectiveness of multi-sensor data fusion processing.

A novel collaborative bearing fault diagnosis method based on multi-scale dynamic fusion network under speed fluctuating condition

Improving bearing fault diagnosis accuracy under speed fluctuation is a challenge in engineering applications. With the development of big data processing technology, a new solution, multi-sensor complementary information, has emerged. However, single-scale dimension compression, which is adopted in most multi-sensor data fusion methods, captures only a small amount of valuable information. To deal with this deficiency, a multi-scale dynamic fusion network (MSDFN) is proposed. First, considering the existence of non-stationary features in the fluctuating speed signal, the FReLU function is adopted to activate the features after considering contextual information. Then, multi-sensor features are fused by multiple scales to obtain richer feature information, and fusion features at different scales are weighted by using the attention mechanism. Finally, batch normalization is employed to standardize the variable speed feature distribution. The validity of the MSDFN is proved by conducting fault diagnosis experiments on two bearings under speed fluctuating conditions. Experimental results indicate that the MSDFN is not only effective in identifying various types of fault samples, but also shows higher stability in multiple trials when compared with other methods.

A Fuzzy Dempster–Shafer Evidence Theory Method with Belief Divergence for Unmanned Surface Vehicle Multi-Sensor Data Fusion

The safe navigation of unmanned surface vehicles in the marine environment requires multi-sensor collaborative perception, and multi-sensor data fusion technology is a prerequisite for realizing the collaborative perception of different sensors. To address the problem of poor fusion accuracy for existing multi-sensor fusion methods without prior knowledge, a fuzzy evidence theory multi-sensor data fusion method with belief divergence is proposed in this paper. First of all, an adjustable distance for measuring discrepancies between measurements is devised to evaluate the degree of measurement closeness to the true value, which improves the adaptability of the method to different classes of sensor data. Furthermore, an adaptive multi-sensor measurement fusion strategy is designed for the case where the sensor accuracy is known in advance. Secondly, the affiliation function of the fuzzy theory is introduced into the evidence theory approach to assign initial evidence of measurements in terms of defining the degree of fuzzy support between measurements, which improves the fusion accuracy of the method. Finally, the belief Jensen–Shannon divergence and the Rényi divergence are combined for measuring the conflict between the evidence pieces to obtain the credibility degree as the reliability of the evidence, which solves the problem of high conflict between evidence pieces. Three examples of multi-sensor data fusion in different domains are employed to validate the adaptability of the proposed method to different kinds of multi-sensors. The maximum relative error of the proposed method for multiple sensor experiments is greater than or equal to 0.18%, and its error accuracy is much higher than the best result of 0.46% among other comparative methods. The experimental results verify that the proposed data fusion method is more accurate than other existing methods.

Multi-sensor data fusion method based on divergence measure and probability transformation belief factor

Dempster–Shafer evidence theory is widely used in multi-sensor data fusion. However, how to manage the counterintuitive result generated by the highly conflicting evidence remains an open question. To solve the problem, a novel multi-sensor data fusion method is proposed, which analyses the credibility of evidence from both the discrepancy between evidences and the factors of evidence itself. Firstly, a new Belief Kullback–Leibler divergence is put forward, which evaluates the credibility of evidence from the discrepancy between evidences. Secondly, another credibility measure called the Probability Transformation Belief Factor is defined, which assesses the credibility of evidence from the evidence itself. These two credibilities are combined as the comprehensive credibility of evidence. Furthermore, considering the uncertainty of evidence, a new belief entropy based on the cross-information within the evidence is presented, which is applied to quantify the information volume of evidence and to adjust the comprehensive credibility of evidence. The adjusted comprehensive credibility is regarded as the final weight to modify the body of evidence. Finally, the Dempster’s combination rule is applied for fusion. Experiment and applications show that the proposed method is effective and superior.

Multi-Sensor Data Fusion Method Based on Improved Evidence Theory

To achieve autonomous navigation in complex marine environments, unmanned surface vehicles are equipped with a variety of sensors for sensing the surrounding environment and their own state. To address the issue of unsatisfactory multi-sensor information fusion in stochastic uncertain systems with unknown disturbances, an improved evidence theory multi-sensor data fusion method is proposed in this article. First, the affiliation function in fuzzy set theory is introduced as a support function to assign initial evidence for multi-sensor data, and the initial evidence is corrected according to the degree of data bias. Second, a divergence measure is employed to measure the degree of conflict and discrepancy among the evidence, and each piece of evidence is allocated proportional weight based on the conflict allocation principle. Finally, the evidence is synthesized through the evidence combination rule, and the data are weighted and summed to obtain the data fusion results. Since it is difficult to obtain dynamic information from multiple sensors carried by unmanned surface vehicles in practical applications, and considering that the proposed method has universal applicability, practical application experiments using previous research demonstrate that the proposed method has higher fusion accuracy than other existing data fusion methods.

Multi-sensor InSAR time series fusion for long-term land subsidence monitoring

ABSTRACT Satellite Interferometric Synthetic Aperture Radar (InSAR) is widely used for topographic, geological and natural resource investigations. However, most of the existing InSAR studies of ground deformation are based on relatively short periods and single sensors. This paper introduces a new multi-sensor InSAR time series data fusion method for time-overlapping and time-interval datasets, to address cases when partial overlaps and/or temporal gaps exist. A new Power Exponential Knothe Model (PEKM) fits and fuses overlaps in the deformation curves, while a Long Short-Term Memory (LSTM) neural network predicts and fuses any temporal gaps in the series. Taking the city of Wuhan (China) as experiment area, COSMO-SkyMed (2011–2015), TerraSAR-X (2015–2019) and Sentinel-1 (2019–2021) SAR datasets were fused to map long-term surface deformation over the last decade. An independent 2011–2020 InSAR time series analysis based on 230 COSMO-SkyMed scenes was also used as reference for comparison. The correlation coefficient between the results of the fusion algorithm and the reference data is 0.87 in the time overlapping region and 0.97 in the time-interval dataset. The correlation coefficient of the overall results is 0.78, which fully demonstrates that the algorithm proposed in our paper achieves a similar trend as the reference deformation curve. The experimental results are consistent with existing studies of surface deformation at Wuhan, demonstrating the accuracy of the proposed new fusion method to provide robust time series for the analysis of long-term land subsidence mechanisms.

The Application Of Multi-Sensor Data Fusion Method with Fuzzy Time Series Model to Improve Indoor Water Prediction Accuracy Quality

There is a lot of indoor air pollution, especially from cigarette smoke, wall paint, air fresheners and gas. With this situation, the room uses Air Box WP6003 air quality detection device by transmitting information about air quality through visualization index. This study aims to improve prediction accuracy with fuzzy time series methods processed through 2 naïve and moving average models using forecast transformers and without transformers. The level of prediction accuracy is calculated through several metrics, namely Mean Absolute Percentage Error (MAPE), Sum of Squares Error (SSE), Mean Square Error (MSE), Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). These results can be calculated between the actual value and the predicted value. The data used is 204584 data from 4 parameters including Temperature, TVOC, HCHO and CO2. The test results with the difference from the forecast transformer and without transformer are comparable. Temperature value obtained using naïve with transformer from RMSE of 0.158866 and naïve without transformer of 0.782397, data using moving average with transformer obtained by 0.147546 and moving average without transformer of 0.772570. This can be explained by the error analysis that was tried, where the error rate continued to increase so that the experimental results continued to be far from the actual number. From the test results it can be concluded that the accuracy of air quality prediction using naïve forecast transformer is pretty accurate.

An Innovative multi-sensor data fusion method based on thrice deeply-fusion architecture for multi-object tracking

For the purpose of track maintenance in a cluttered environment with some false alarms, false dismissals and measurement disturbances, this paper presents a new multi-object tracking procedure with uncertainty of the source of the measurement data returned from multi-sensor. The tracking method is a thrice deeply-fusion approach constructed by a primary fusion based on an improved probabilistic data association filter (IPDAF), a secondary fusion with historical motion trajectories, and a thrice fusion with road-markings. It incorporates the existence probabilities of the individual tracks and the variable number of objects based on the Bayesian estimation theory, which can improve the tracking performances effectively in an environment with high clutter density. A binary 2-D assignment is adopted for the optimal data association, which is established as a nonlinear optimization problem. In the motion modeling, it introduces multiple measurement models for different sensors into the method. Then, the estimation could be performed with greater reliability. The computational efficiency is satisfying and it can be used for real-time application, which is verified by two real test scenarios.

A Multisensor Data Fusion Based Anomaly Detection (Ammonia Nitrogen) Approach for Ensuring Green Coastal Environment

Great changes have been brought about by the coastal environment when the economy develops rapidly. Coastal environmental monitoring is the basis and technical guarantee for coastal environmental protection supervision and management. It is one of the important tasks to detect and timely discover coastal seawater anomalies. Usually, a single sensor cannot determine whether the coastal environment or ship operation is an anomaly. Recently, an unmanned surface vehicle for coastal environment monitoring was developed, and stacked autoencoders are used for seawater anomaly detection using multisensor data fusion methods. The multisensor data of pH, conductivity, and ammonia nitrogen are employed to judge the anomaly of seawater. The mean, standard deviation, mean square root, and normalized power spectrum features of multisensor data are extracted, and a stacked autoencoder is employed to fuse these features for anomaly detection. The proposed method is feasible and effective for anomaly detection of coastal water quality and ship operation. Compared with other commonly used methods, the proposed method has a higher recall, precision, and F1 score performance.

Considering Multisensor Data Fusion and Data-Driven Special Effects Animation Simulation

The data-driven simulation algorithm uses the method of surface matching to establish the corresponding relationship between the surfaces of the two dynamic models at the same time. It also calculates the appropriate deformation field to align the animated character surface and then uses the interpolation operation to further expand the new animated character motion. With the promotion of industrial application requirements such as games, film and television, and education and entertainment, computer animation special effects simulation technology has a wide range of application prospects. Based on a multisensor and data-driven approach, this paper conducts a targeted study on the realistic simulation of typical special effects animations. Aiming at the high time complexity and time-consuming problems of simulation algorithms, data-driven simulation methods are studied and scenes or objects that simulate the real environment can be created. They are given different properties and parameters, and the desired results are obtained through computer calculation and rendering. Finally, it can be output into a smooth animation. Based on the animation simulation data and multisensor data fusion method proposed in this paper, the on-site calculation process can be accelerated and efficient animation simulation driven by data can be realized. In 3ds Max 2013, there is also a small function that can sort the bone list in ascending or descending order in alphabetical and numerical order. This helps manage bones attached to the skin modifier more efficiently. This saves a lot of time when picking bones. About 44.3% of the people during the study were proficient in the use of software. After 1200 frames, the number of foam particles increases sharply, and the two-way coupling algorithm takes time. As the number of foam particles decreases, the time required for bidirectional coupling decreases. The article gives a brief description of the data-driven fusion model. The fluid simulation algorithm generates two different sequences by utilizing two different sets of setup parameters. It is then transformed into a spatiotemporal surface as input data for the fusion model. This research contributes to the development of special effects animation simulations.

Karst Collapse Monitoring and Early Warning Evaluation Method Based on Multisensor Internet of Things.

The international community has paid extensive attention to the numerous engineering problems faced by karst areas caused by the increasingly frequent human activities. China has a wide variety of karst forms. Among them, carbonate karst is the most widely distributed, and the development of carbonate karst is relatively strong in many areas. Countless property losses are caused by karst disasters every year. This article aims to study the real-time monitoring and timely early warning of karst collapse through the use of multisensor Internet of Things technology. To this end, this article proposes an improved method for multisensor data fusion. It optimizes and improves the transmission and delivery efficiency of its data. This makes the improved multisensor more in line with the research content of this article in terms of monitoring efficiency. At the same time, related experiments and analyses are designed to compare and analyze the karst collapse and the monitoring efficiency of the sensor. The experimental results of this article show that after the improvement, the anti-interference ability of the monitoring system is increased by 34%. The frequency of early warning has also been improved by 24%, which has high practical application value.