Multi-source Information Fusion Research Articles

Rapid quantification of royal jelly quality by mid-infrared spectroscopy coupled with backpropagation neural network

Royal jelly is rich in nutrients but its quality is greatly affected by storage conditions. To determine the quality of royal jelly accurately and quickly, a qualitative discrimination model was established based on the fusion of conventional parameters and mid-infrared spectrum, using support vector machine. The prediction models for three representative quality parameters were developed by the backpropagation neural network with various algorithms. The results demonstrated that the recognition rate of the multi-source information fusion model was increased to 100% when compared with that of the spectral data preprocessed by Savitzky-golay smoothing (95.83%). The mean square errors of the constructed model for moisture, water-soluble protein, and total sugar were 0.0032, 0.0058, and 0.0069, respectively. The constructed model had an ensured accuracy for the calibration set, with the correlation coefficient of prediction maintained at 0.9353, 0.9533, and 0.9563, which could meet the requirement of non-destructive rapid detection of royal jelly quality.

Traffic load simulation on bridges based on improved non-contact machine vision technique and multi-source information fusion

This study presented a novel traffic load simulation method. Specifically, an improved You Only Look Once (YOLO) detector based on the k-means++ clustering algorithm is implemented to identify vehicle types, and the multi-objective tracking algorithm is used to obtain the spatial location and velocity information of vehicles. Based on the results of detection and tracking, multi-source information fusion techniques are applied to build a vehicle database. Meanwhile, a Monte Carlo-Intelligent Driving Model can be developed for traffic load simulation based on the vehicle database. The applicability and effectiveness of the method are verified by the suspension bridge field test. The results show that the identification accuracy of vehicle type and vehicle number is about 88.28% and 74.09%, respectively. Meanwhile, the vehicle speed error is less than 6.5%. The method can be used to provide a reasonable traffic load model for large-span bridges without the aid of a deck-embedded weigh-in-motion system.

Metallogenic prediction based on geological-model driven and data-driven multisource information fusion: A case study of gold deposits in Xiong’ershan area, Henan Province, China

The traditional metallogenic prediction based on the geological model has been extensively used in prospecting, and the emerging machine learning method has also been successfully applied to mineral prospectivity prediction. In this study, we proposed a method based on the combination of model-driven and data-driven approaches, which was applied to the prospective area of the gold deposits in Xiong’ershan area. In this method, the geological prospecting model was taken as the prior knowledge of the data-driven model, and the metallogenic prediction elements of the prospecting model were constructed as the characteristic variables of the data-driven model, which improved the prediction accuracy of prospecting based on the geological model and reduced the uncertainty of outcome prediction based on data-driven. The XGBoost and LightGBM decision tree algorithms were selected for the data-driven method to create MPM. Then the SHAP framework was used to analyze the characteristic variables of the model, thus showing that geochemical prospecting could be carried out at the intersection of faults in the study area. Finally, the performance indicators based on machine learning and the legacy geological data were used to verify the robustness of the model. The AUC values of the final model were >0.95, and the new exploration targets were delineated in the high anomaly area, indicating the direction for subsequent exploration work.

Mobile augmented reality-based visualization framework for lifecycle O&M support of urban underground pipe networks

The structural deterioration of urban underground pipe networks is the accumulated combined effects of physical, environmental and operational factors during the whole lifecycle. Accurate on-site locating and mapping of pipes and their conditions has been a great challenge for construction, maintenance and emergency response in a dynamic manner. Existing approaches, mostly based on design-centric and static visualization, are unsuitable for implementation within the operation and maintenance (O&M) phase because they are performed in a completely virtual environment and are unable to interact with buried assets on site. An integrated hyperrealistic visualization framework based on mobile augmented reality (MAR) is developed to support various real-life project tasks for O&M support of urban underground pipe networks. The proposed approach consists of three parts: a cloud-based data management platform, a 3D visualization development platform, and a location-based registration method for MAR. A cloud–edge–mobile layer is proposed to manage, analyze, process and visualize multisource data, especially for dynamic data. Field experiments on a real-life urban gas pipe network were carried out to investigate the feasibility of the proposed framework to achieve informed O&M support. The results demonstrate that (1) a smartphone linked to a Lite RTK device allows for both accurate locating and easy operation; (2) both the 3D virtual digital model and the multisource data visually superimpose onto the realistic environment, enabling on-site human–machine interaction in a hyperrealistic manner; and (3) MAR-based immersive visualization contributes to avoiding excavation damage, improving O&M efficiency and assisting emergency responders to make timely and better–informed decisions based on multisource information fusion.

Multi-source monitoring information fusion method for dam health diagnosis based on Wasserstein distance

The fusion of multi-source monitoring information has become the main trend in the field of dam health diagnosis because of the increasing amount of monitoring data that can be obtained from different sensors. However, the Dempster–Shafer (D–S) evidence theory, an important method in multi-source information fusion, may produce counter-intuitive results when fusing conflicting pieces of evidence. To some extent, existing distance measures can deal with highly conflicting evidence, however, the fusion of completely conflicting evidence (indicated by the conflict coefficient K, K = 1) is typically ignored. This study mainly focuses on the fusion of pieces of completely conflicting evidence considering K = 1. The Wasserstein distance in the field of deep learning is introduced to the D–S evidence theory. Based on the foregoing, the belief Wasserstein–1 distance (BWD) method combined with basic probability assignment is proposed to measure evidence distance. The application of the BWD method in dam health diagnosis is presented to demonstrate the validity and effectiveness of this method in multi-source fusion with completely conflicting evidence.

MSIF: Multi-source information fusion based on information sets

Multi-source information fusion is a sophisticated estimating technique that enables users to analyze more precisely complex situations by successfully merging key evidence in the vast, varied, and occasionally contradictory data obtained from various sources. Restricted by the data collection technology and incomplete data of information sources, it may lead to large uncertainty in the fusion process and affect the quality of fusion. Reducing uncertainty in the fusion process is one of the most important challenges for information fusion. In view of this, a multi-source information fusion method based on information sets (MSIF) is proposed in this paper. The information set is a new method for the representation of granularized information source values using the entropy framework in the possibilistic domain. First, four types of common membership functions are used to construct the possibilistic domain as the information gain function (or agent). Then, Shannon agent entropy and Shannon inverse agent entropy are defined, and their summation is used to evaluate the total uncertainty of the attribute values and agents. Finally, an MSIF algorithm is designed by infimum-measure approach. The experimental results show that the performance of Gaussian kernel function is good, which provides an effective method for fusing multi-source numerical data.

Refined On-Manifold IMU Preintegration Theory for Factor Graph Optimization Based on Equivalent Rotation Vector

Multisource information fusion is a major area of interest within the field of unmanned technology. In comparison with utilizing the Kalman filter (KF)-based method, the factor graph optimization (FGO)-based method has recently been proposed as an innovative approach. However, the traditional FGO-based inertial measurement unit (IMU) preintegration process is designed for low-accuracy inertial sensors, which leads to insufficient utilization of information and a decrease in positioning accuracy, in the case of tactical-grade or higher accuracy IMUs. Therefore, this article refines the process on the manifold model and utilizes the equivalent rotation vector to accumulate IMU measurements. The main contribution of our work is the refined on-manifold IMU preintegration theory and the application of the multisample algorithm based on the equivalent rotation vector for obtaining more precise preintegrated IMU measurements. The refined on-manifold IMU preintegration theory is applied based on the FGO algorithm in the global navigation satellite system (GNSS)/inertial navigation system (INS) loosely integrated navigation system. The experiments based on simulated and real-world data were conducted and demonstrate that, compared to a high-precision extended KF (EKF) algorithm and optimization methods, the proposed method achieves more accuracy in a dynamic environment.

A two-stage eco-cooling control strategy for electric vehicle thermal management system considering multi-source information fusion

A two-stage eco-cooling control strategy for electric vehicle thermal management system considering multi-source information fusion

A Multi-Source Weighted Deep Transfer Network for Open-Set Fault Diagnosis of Rotary Machinery

In real industries, there often exist application scenarios where the target domain holds fault categories never observed in the source domain, which is an open-set domain adaptation (DA) diagnosis issue. Existing DA diagnosis methods under the assumption of sharing identical label space across domains fail to work. What is more, labeled samples can be collected from different sources, where multisource information fusion is rarely considered. To handle this issue, a multisource open-set DA diagnosis approach is developed. Specifically, multisource domain data of different operation conditions sharing partial classes are adopted to take advantage of fault information. Then, an open-set DA network is constructed to mitigate the domain gap across domains. Finally, a weighting learning strategy is introduced to adaptively weigh the importance on feature distribution alignment between known class and unknown class samples. Extensive experiments suggest that the proposed approach can substantially boost the performance of open-set diagnosis issues and outperform existing diagnosis approaches.

Intelligent Innovation Platform Design of Chinese Film and Television Music Based on Multi-Dimensional Information Fusion Algorithm

Experienced changes in recording techniques and musical styles. With the development of the times, digital technology is constantly developing and progressing. In the aspect of film and television music creation, the application of digital technology has been innovated accordingly. Using multi-source information fusion to build a music feature system to solve the problem of music recommendation cold start Users provide personalized music recommendations. The optimized high-concurrency Cuckoo Hashing algorithm is implemented on the DPDK platform, and the algorithm is optimized for large page memory, batch processing and preprocessing combined with DPDK features. Music information retrieval is expanded from the traditional retrieval based on keywords such as file name and artist name. At the same time, the evaluation of the example model of the system also verifies that the multi-information source fusion method proposed in this paper can optimize the precision and recall rate of the system.

Characteristic evaluation via multi-sensor information fusion strategy for spherical underwater robots

Currently, most of the existing fusion methods ignore the rich multi-source information of different types of sensor nodes in the underwater unknown environment, which makes it challenging for Autonomous Underwater Vehicles (AUVs) to accurately perceive the external environment and make actionable decisions. Considering the key issues such as attitude estimation, positioning and obstacle avoidance involved in performing AUV tasks, this paper proposed a Multi-Source Information Fusion (MSIF) model for Spherical Underwater Robots (SURs) we developed based on various low-cost sensors. Multi-source information from an Inertial Measurement Unit (IMU), Pressure Sensor Array (PSA), Obstacle Avoidance Sensor Array (OASA), Depth Sensor (DS), Looking-Down Camera (LDC) and Acoustic Communication System (ACS) were fused to enable SUR to obtain high-precision estimated data for attitude estimation, positioning and obstacle avoidance, etc. More precisely, according to the correlation between the sensors, the optimized model was constructed to compensate for angle errors, velocity errors, orientation errors, etc. Subsequently, a machine learning method using Back Propagation Neural Network (BPNN) was proposed to improve the accuracy and effectiveness of the MSIF model through feature selection, data training, and feature estimation, etc. Finally, a series of experiments were performed under different scenarios, such as motion and obstacle avoidance experiments. The theoretical derivation and comprehensive evaluations demonstrated the effectiveness and feasibility of the proposed model, which provided a new reference value for solving issues such as attitude estimation, positioning and obstacle avoidance of AUVs.

Multisource information fusion-based environment perception and dynamic model of underwater vehicle in irregular ocean environment

The irregular ocean environment brings difficulties to observation and affects the movement state of underwater vehicles. Multisource information fusion can provide more reliable results on information-rich datasets. This paper proposes a multisource information fusion-based environment perception and dynamics model for underwater vehicles in irregular ocean environments. First, the multisource observation data is decomposed by the latent feature disentanglement learning model into the latent stable and dynamical features. The dynamical feature is predicted by the retrospect regression method and then integrated with the latent stable feature to predict the ocean field. After that, the environment-vehicle coupling effect is explored by thermodynamics and statics simulation. Then, a real-time dynamic model of the underwater vehicle is constructed by combining multisource information from physical principles, environmental perception, and sensor observations. Finally, extensive experiments are performed on a novel underwater glider and a publicly available South China Sea dataset to verify the proposed method.

PCNet: A structure similarity enhancement method for multispectral and multimodal image registration

Multispectral and multimodal images are of important usage in the field of multi-source visual information fusion. Due to the alternation or movement of image devices, the acquired multispectral and multimodal images are usually misaligned, and hence image registration is pre-requisite. Different from the registration of common images, the registration of multispectral or multimodal images is a challenging problem due to the nonlinear variation of intensity and gradient. To cope with this challenge, we propose the phase congruency network (PCNet) to enhance the structure similarity of multispectral or multimodal images. The images can then be aligned using the similarity-enhanced feature maps produced by the network. PCNet is constructed under the inspiration of the well-known phase congruency. The network embeds the phase congruency prior into two simple trainable layers and series of modified learnable Gabor kernels. Thanks to the prior knowledge, once trained, PCNet is applicable on a variety of multispectral and multimodal data such as flash/no-flash and RGB/NIR images without additional further tuning. The prior also makes the network lightweight. The trainable parameters of PCNet is 2400× and 1500×less than the deep-learning registration method deep homography network (DHN) and unsupervised deep homography network (UDHN), while its registration performance surpasses them. Experimental results validate that PCNet outperforms current state-of-the-art conventional multimodal registration algorithms. Besides, PCNet can act as a complementary part of the deep-learning registration methods, which significantly boosts their registration accuracy. On the Columbia imaging and vision laboratory (CAVE) multispectral dataset, the percentage of the number of images under 1 pixel average corner error (ACE) of UDHN is raised from 0.1% to 82.5% after the processing of PCNet.

Combining non-negative matrix factorization with graph Laplacian regularization for predicting drug-miRNA associations based on multi-source information fusion.

Increasing evidences suggest that miRNAs play a key role in the occurrence and progression of many complex human diseases. Therefore, targeting dysregulated miRNAs with small molecule drugs in the clinical has become a new treatment. Nevertheless, it is high cost and time-consuming for identifying miRNAs-targeted with drugs by biological experiments. Thus, more reliable computational method for identification associations of drugs with miRNAs urgently need to be developed. In this study, we proposed an efficient method, called GNMFDMA, to predict potential associations of drug with miRNA by combining graph Laplacian regularization with non-negative matrix factorization. We first calculated the overall similarity matrices of drugs and miRNAs according to the collected different biological information. Subsequently, the new drug-miRNA association adjacency matrix was reformulated based on the nearest neighbor profiles so as to put right the false negative associations. Finally, graph Laplacian regularization collaborative non-negative matrix factorization was used to calculate the association scores of drugs with miRNAs. In the cross validation, GNMFDMA obtains AUC of 0.9193, which outperformed the existing methods. In addition, case studies on three common drugs (i.e., 5-Aza-CdR, 5-FU and Gemcitabine), 30, 31 and 34 of the top-50 associations inferred by GNMFDMA were verified. These results reveal that GNMFDMA is a reliable and efficient computational approach for identifying the potential drug-miRNA associations.

Negation of the Quantum Mass Function for Multisource Quantum Information Fusion With its Application to Pattern Classification.

In artificial intelligence systems, a question on how to express the uncertainty in knowledge remains an open issue. The negation scheme provides a new perspective to solve this issue. In this paper, we study quantum decisions from the negation perspective. Specifically, complex evidence theory (CET) is considered to be effective to express and handle uncertain information in a complex plane. Therefore, we first express CET in the quantum framework of Hilbert space. On this basis, a generalized negation method is proposed for quantum basic belief assignment (QBBA), called QBBA negation. In addition, a QBBA entropy is revisited to study the QBBA negation process to reveal the variation tendency of negation iteration. Meanwhile, the properties of the QBBA negation function are analyzed and discussed along with special cases. Then, several multisource quantum information fusion (MSQIF) algorithms are designed to support decision making. Finally, these MSQIF algorithms are applied in pattern classification to demonstrate their effectiveness. This is the first work to design MSQIF algorithms to support quantum decision making from a new perspective of "negation", which provides promising solutions to knowledge representation, uncertainty measure, and fusion of quantum information.

Research on Information Fusion Preview Control for Freight Train Speed Tracking

Background: With the increase in the speed and capacity of freight trains, the safety, energy efficiency and intelligent operation of train operation have become increasingly important. Automatic freight trains operation can replace manual operation with automated control systems, which can guarantee the safety of train operation, and improve operational efficiency and reduce operational energy consumption. Objective: Solve the problem of tracking accuracy and stability deterioration caused by multi-information of freight train. Methods: Global navigation satellite system, inertial navigation system and speed measuring motor are selected to construct a speed fusion measurement model by using loosely coupled integrated navigation and improved entropy weight method. Calculate the information quantity of performance index and control quantity in preview control, get the controlled quantity of information fusion optimal preview control. Results: The average tracking error of the multi-source information fusion preview controller is 0.038m/s, which is 49% lower than that of the control experiment. Conclusion: Multi-source information fusion preview controller can effectively reduce the tracking error of freight train speed tracking system and improve the accuracy of automatic freight trains operation.

Time Synchronization and Space Registration of Roadside LiDAR and Camera

The sensing system consisting of Light Detection and Ranging (LiDAR) and a camera provides complementary information about the surrounding environment. To take full advantage of multi-source data provided by different sensors, an accurate fusion of multi-source sensor information is needed. Time synchronization and space registration are the key technologies that affect the fusion accuracy of multi-source sensors. Due to the difference in data acquisition frequency and deviation in startup time between LiDAR and the camera, asynchronous data acquisition between LiDAR and camera is easy to occur, which has a significant influence on subsequent data fusion. Therefore, a time synchronization method of multi-source sensors based on frequency self-matching is developed in this paper. Without changing the sensor frequency, the sensor data are processed to obtain the same number of data frames and set the same ID number, so that the LiDAR and camera data correspond one by one. Finally, data frames are merged into new data packets to realize time synchronization between LiDAR and camera. Based on time synchronization, to achieve spatial synchronization, a nonlinear optimization algorithm of joint calibration parameters is used, which can effectively reduce the reprojection error in the process of sensor spatial registration. The accuracy of the proposed time synchronization method is 99.86% and the space registration accuracy is 99.79%, which is better than the calibration method of the Matlab calibration toolbox.

An improved belief Hellinger divergence for Dempster-Shafer theory and its application in multi-source information fusion

An improved belief Hellinger divergence for Dempster-Shafer theory and its application in multi-source information fusion

Multi-source electricity information fusion methods: A survey

With the vigorous development of the global economy, the demand for electricity quality from all walks of life is also increasing, so it is essential to ensure the electric power grid’s safe, stable, and efficient operation. Multi-source electric power information fusion, as the core technology of electric power grid data processing, has become the foundation to promote the intelligent and automatic development of the electric power grid. This paper presents the first work on the survey of the methods of electricity information fusion. It first gives an overview of the process of electricity information fusion and shows the types of electricity data. Then, we provide different classifications of existing methods in view of communication annotation and electric power data, and conduct a thorough comparison and analysis of them. Moreover, we introduce the relevant data sets and evaluation criteria of electric power information and summarize the corresponding evaluation scenarios. Finally, we conclude the maturity of existing works and provide an outlook on future multi-source electric power information fusion methods.

Reliability acceptance test plan based on multi-level information

The reliability acceptance test (RAT) is often used to decide whether the product should be accepted or rejected based on the reliability requirement. Conventional methods usually require long test duration and large test sample size or bring high risks for both costumer and producer. Therefore, in this paper, multi-source and multi-level information is used to circumvent the above limitations. Firstly, prior as well as the posterior distributions are given and posterior moments have been derived. Then, multi-level information can be converted into system based on system structure, and the RAT plan can be given based on multi-source and multi-level information fusion. Finally, this proposed method is applied to a numerical study and the results prove the effectiveness.