Problem Of Information Fusion Research Articles

Multi-sensor Data Analysis for Aerial Image Semantic Segmentation and Vectorization

Abstract. One of the urgent and constantly in demand problems is updating maps. Maps, representing geo-information in vector form, have undoubted advantages in compactness and ”readability” compared to aerial photographs. The issue of maps actuality is critically important for rational urban planning, precision farming, the relevance of the cadastre and other geospatial applications. Various sources of data are used for maps updating, with aerial imagery being the main and rich source of information. Automatic processing of aerial photographs makes it possible to efficiently extract vector information, providing operational monitoring and accounting for changes that have appeared. The presented study addresses the problem of multi sensor information fusion in order to obtain accurate vector information. We use aerial images as a main data source and additionally the data of laser scanning and ground survey to increase performance of automatic image semantic segmentation and vectorization. The proposed framework is demonstrated on the task of forest monitoring.

Intelligent fault diagnosis methods for hydraulic components based on information fusion: review and prospects

Abstract Hydraulic component faults have the characteristics of nonlinear time-varying signal, strong concealment, and difficult feature extraction, etc. Timely and accurately fault diagnosis of hydraulic components is helpful to curb economic losses and accidents, so researches have carried out a lot of research on hydraulic components. Information fusion technology can combine multi-source data from multiple dimensions to mine fault data characteristics, which effectively improves the accuracy and reliability of fault diagnosis results. However, there is currently a lack of a comprehensive and systematic review in this domain. Therefore, in this paper, the hydraulic components information fusion fault diagnosis technologies are summarized and analyzed, including the main process information fusion fault diagnosis and the research status of information fusion fault diagnosis of hydraulic system. The methods and techniques involved in the fusion process, data source and fusion method of fault diagnosis of hydraulic components information fusion are described and summarized. The problems of information fusion in fault diagnosis of hydraulic components are analyzed, the solutions are discussed, and the research ideas of improving information fusion fault diagnosis are put forward. Finally, digital twin technology is introduced, and the advantages and research status of intelligent fault diagnosis based on digital twin are summarized. On this basis, the intelligent fault diagnosis of hydraulic components based on information fusion is summarized, and the challenges and future research ideas of applying information fusion and digital twin to intelligent fault diagnosis of hydraulic components are put forward and analyzed comprehensively.

CMNet: deep learning model for colon polyp segmentation based on dual-branch structure.

Colon cancer is one of the top three diseases in gastrointestinal cancers, and colon polyps are an important trigger of colon cancer. Early diagnosis and removal of colon polyps can avoid the incidence of colon cancer. Currently, colon polyp removal surgery is mainly based on artificial-intelligence (AI) colonoscopy, supplemented by deep-learning technology to help doctors remove colon polyps. With the development of deep learning, the use of advanced AI technology to assist in medical diagnosis has become mainstream and can maximize the doctor's diagnostic time and help doctors to better formulate medical plans. We propose a deep-learning model for segmenting colon polyps. The model adopts a dual-branch structure, combines a convolutional neural network (CNN) with a transformer, and replaces ordinary convolution with deeply separable convolution based on ResNet; a stripe pooling module is introduced to obtain more effective information. The aggregated attention module (AAM) is proposed for high-dimensional semantic information, which effectively combines two different structures for the high-dimensional information fusion problem. Deep supervision and multi-scale training are added in the model training process to enhance the learning effect and generalization performance of the model. The experimental results show that the proposed dual-branch structure is significantly better than the single-branch structure, and the model using the AAM has a significant performance improvement over the model not using the AAM. Our model leads 1.1% and 1.5% in mIoU and mDice, respectively, when compared with state-of-the-art models in a fivefold cross-validation on the Kvasir-SEG dataset. We propose and validate a deep learning model for segmenting colon polyps, using a dual-branch network structure. Our results demonstrate the feasibility of complementing traditional CNNs and transformer with each other. And we verified the feasibility of fusing different structures on high-dimensional semantics and successfully retained the high-dimensional information of different structures effectively.

A novel information fusion method using improved entropy measure in multi-source incomplete interval-valued datasets

Multi-source data is a comprehensive data type that combines multiple sources of information or datasets. Compared to point-valued data, interval-valued data provides a more accurate representation of the uncertainty and variability associated with objects. In practical situations, data obtained from multiple sources may contain missing values for various reasons. Therefore, it is essential to develop multi-source information fusion technology in order to achieve information fusion or information extraction from multi-source incomplete data. This paper aims to explore the information fusion problem of multi-source incomplete interval-valued datasets. The primary contributions of this study involve utilizing the principle of statistical distribution and KL divergence to establish a metric for measuring the similarity between intervals. Firstly, this approach helps to reduce the problem of disregarding internal information within interval values, which can result in the loss of valuable information. Secondly, we establish an interval fuzzy similarity relation based on the mentioned concept of similarity among interval values. Moreover, we investigate the uncertainty measurement of incomplete interval-valued decision datasets and design an emerging information entropy fusion method. Finally, we comprehensively evaluate the effectiveness of the proposed method. Experimental results indicate that the proposed approach has advantage over the maximum, minimum, mean, and information entropy fusion method based on tolerance relationship. In addition, the distance metric used in this article can improve the fusion classification effect compared to several common interval-valued distance measures.

Drone Based RGBT Tracking with Dual-Feature Aggregation Network

In the field of drone-based object tracking, utilization of the infrared modality can improve the robustness of the tracker in scenes with severe illumination change and occlusions and expand the applicable scene of the drone object tracking task. Inspired by the great achievements of Transformer structure in the field of RGB object tracking, we design a dual-modality object tracking network based on Transformer. To better address the problem of visible-infrared information fusion, we propose a Dual-Feature Aggregation Network that utilizes attention mechanisms in both spatial and channel dimensions to aggregate heterogeneous modality feature information. The proposed algorithm has achieved better performance by comparing with the mainstream algorithms in the drone-based dual-modality object tracking dataset VTUAV. Additionally, the algorithm is lightweight and can be easily deployed and executed on a drone edge computing platform. In summary, the proposed algorithm is mainly applicable to the field of drone dual-modality object tracking and the algorithm is optimized so that it can be deployed on the drone edge computing platform. The effectiveness of the algorithm is proved by experiments and the scope of drone object tracking is extended effectively.

Application of copula-based Bayesian network method to water leakage risk analysis in cross river tunnel of Wuhan Rail Transit Line 3

Water leakage typically infiltrates through bolt holes and segment joints, posing a threat to the stability of tunnel structures. In this paper, we develop a copula-based Bayesian network (CBN) method for dependency modeling of complex systems to conduct risk analysis and management of water leakage in operational tunnels. Thirteen potential factors are used to construct the water leakage risk system, and the permeate water volume is selected as the observation index of water leakage risk. An application to a realistic cross-river tunnel of Wuhan Rail Transit Line 3 is implemented to verify the effectiveness of the developed model. Through correlation analysis, the bolt failure rate and seal breakage and aging rate are diagnosed as critical factors with the greatest impact on the water leakage risk. Forward and backward reasoning enables the model to be dynamically updated to provide decision support for water leakage risk management. This research contributes to the following: (a) expressing the dependence of the factors affecting water leakage; (b) solving the problem of modeling uncertainty and information fusion; and (c) updating the model to realize the safety control of tunnel water leakage.

MJ-GAN: Generative Adversarial Network with Multi-Grained Feature Extraction and Joint Attention Fusion for Infrared and Visible Image Fusion.

The challenging issues in infrared and visible image fusion (IVIF) are extracting and fusing as much useful information as possible contained in the source images, namely, the rich textures in visible images and the significant contrast in infrared images. Existing fusion methods cannot address this problem well due to the handcrafted fusion operations and the extraction of features only from a single scale. In this work, we solve the problems of insufficient information extraction and fusion from another perspective to overcome the difficulties in lacking textures and unhighlighted targets in fused images. We propose a multi-scale feature extraction (MFE) and joint attention fusion (JAF) based end-to-end method using a generative adversarial network (MJ-GAN) framework for the aim of IVIF. The MFE modules are embedded in the two-stream structure-based generator in a densely connected manner to comprehensively extract multi-grained deep features from the source image pairs and reuse them during reconstruction. Moreover, an improved self-attention structure is introduced into the MFEs to enhance the pertinence among multi-grained features. The merging procedure for salient and important features is conducted via the JAF network in a feature recalibration manner, which also produces the fused image in a reasonable manner. Eventually, we can reconstruct a primary fused image with the major infrared radiometric information and a small amount of visible texture information via a single decoder network. The dual discriminator with strong discriminative power can add more texture and contrast information to the final fused image. Extensive experiments on four publicly available datasets show that the proposed method ultimately achieves phenomenal performance in both visual quality and quantitative assessment compared with nine leading algorithms.

Information Fusion over Network Dynamics with Unknown Correlations: An Overview

Survey/review study Information Fusion over Network Dynamics with Unknown Correlations: An Overview Wangyan Li 1, and Fuwen Yang 2,* 1 College of Science, University of Shanghai for Science and Technology, Shanghai 200093, China 2 Griffth School of Engineering, Griffth University, Gold Coast Campus, QLD 4222, Australia * Correspondence: fuwen.yang@griffth.edu.au Received: 24 October 2022 Accepted: 22 November 2022 Published: 23 June 2023 Abstract: Unknown correlations (UCs) generally exist in a wide spectrum of practical multi-source information fusion problems, and thereby, their corresponding fusion problems have become one of the most important topics in information fusion domain. During the past three decades, the research on this topic has been growing rapidly and extensively, and, as a result, various important advances have been reported. In this overview, we intend to summarize the culmination of years of development in the field of information fusion under UCs as a roadmap. First, the potential reasons leading to UCs are investigated. According to the unknown nature of correlations, we further divide UCs into two categories, i.e., fully UCs, and partially UCs. For each category, the corresponding fusion methods are reviewed. Next, this roadmap witnesses the recent development of information fusion under UCs in a distributed way thanks to the popularity of distributed sensing technology. In particular, the distributed fusion techniques based on consensus, diffusion, and multi-object tracking strategies for UCs are examined. Finally, some future perspectives on information fusion under UCs are pointed out.

On Computing Evidential Centroid Through Conjunctive Combination: An Impossibility Theorem

The theory of belief functions (TBF) is now a widespread framework to deal and reason with uncertain and imprecise information, in particular to solve information fusion and clustering problems. Combination functions (rules) and distances are essential tools common to both the clustering and information fusion problems in the context of TBF, which have generated considerable literature. Distances and combination between evidence corpus of TBF are indeed often used within various clustering and classification algorithms, however their interplay and connections have seldom been investigated, which is the topic of this paper. More precisely, we focus on the problem of aggregating evidence corpus to obtain a representative one, and we show through an impossibility theorem that in this case, there is a fundamental contradiction between the use of conjunctive combination rules on the one hand, and the use of distances on the other hand. Rather than adding new methodologies, such results are instrumental in guiding the user among the many methodologies that already exist. To illustrate the interest of our results, we discuss different cases where they are at play.

Early identification of diffusion source in complex networks with evidence theory

The inferring of source in the aftermath of a pandemic outbreak has received immense attention due to its substantial practical potential. In light of the inaccessibility to the status of every individual in the network, observer-based approaches have become an essential research direction for solving this problem. However, the way that utilizes the combined observers to infer the source node, as most existing methods follow, may compromise the algorithm's flexibility and generalizability. To address this issue, we ask the question: can observational information be viewed as pairs of expert-driven information so that it allows us to frame the source localization as an information-fusion problem? To this end, we propose an evidential source localization (ESL) model that utilizes evidence theory to represent the uncertainty caused by limited information and to determine the source node by the information-fusion technique. Moreover, ESL is characterized by its ability to detect sources of disease at an early stage of the pandemic. Rather than focusing on a specific structure, we consider an arbitrary graph as our subject, rendering our method general enough for more applications. Experimental results on real-world networks demonstrate the superiority of ESL in efficiency and effectiveness with the comparison to other state-of-the-art methods.

Privacy protection in intelligent vehicle networking: A novel federated learning algorithm based on information fusion

Federated learning is an effective technique to solve the problem of information fusion and information sharing in intelligent vehicle networking. However, most of the existing federated learning algorithms generally have the risk of privacy leakage. To address this security risk, this paper proposes a novel personalized federated learning with privacy preservation (PDP-PFL) algorithm based on information fusion. In the first stage of its execution, the new algorithm achieves personalized privacy protection by grading users’ privacy based on their privacy preferences and adding noise that satisfies their privacy preferences. In the second stage of its execution, PDP-PFL performs collaborative training of deep models among different in-vehicle terminals for personalized learning, using a lightweight dynamic convolutional network architecture without sharing the local data of each terminal. Instead of sharing all the parameters of the model as in standard federated learning, PDP-PFL keeps the last layer local, thus adding another layer of data confidentiality and making it difficult for the adversary to infer the image of the target vehicle terminal. It trains a personalized model for each vehicle terminal by “local fine-tuning”. Based on experiments, it is shown that the accuracy of the proposed new algorithm for PDP-PFL calculation can be comparable to or better than that of the FedAvg algorithm and the FedBN algorithm, while further enhancing the protection of data privacy.

Research on classification method of multimodal magnetic resonance images of Alzheimer's disease based on generalized convolutional neural networks

Alzheimer's disease (AD) is a progressive and irreversible neurodegenerative disease. Neuroimaging based on magnetic resonance imaging (MRI) is one of the most intuitive and reliable methods to perform AD screening and diagnosis. Clinical head MRI detection generates multimodal image data, and to solve the problem of multimodal MRI processing and information fusion, this paper proposes a structural and functional MRI feature extraction and fusion method based on generalized convolutional neural networks (gCNN). The method includes a three-dimensional residual U-shaped network based on hybrid attention mechanism (3D HA-ResUNet) for feature representation and classification for structural MRI, and a U-shaped graph convolutional neural network (U-GCN) for node feature representation and classification of brain functional networks for functional MRI. Based on the fusion of the two types of image features, the optimal feature subset is selected based on discrete binary particle swarm optimization, and the prediction results are output by a machine learning classifier. The validation results of multimodal dataset from the AD Neuroimaging Initiative (ADNI) open-source database show that the proposed models have superior performance in their respective data domains. The gCNN framework combines the advantages of these two models and further improves the performance of the methods using single-modal MRI, improving the classification accuracy and sensitivity by 5.56% and 11.11%, respectively. In conclusion, the gCNN-based multimodal MRI classification method proposed in this paper can provide a technical basis for the auxiliary diagnosis of Alzheimer's disease.

Multiradar Joint Tracking of Cluster Targets Based on Graph-LSTMs

The cluster target brings a serious challenge to the traditional multisensor multitarget tracking algorithm because of its large number of members and the cooperative interaction between members. Using multiradar joint tracking cluster target is an alternative method to solve the problem of cluster target tracking, but it inevitably brings the problem of radar-target assignment and tracking information fusion. Aiming at the problem of radar-target assignment and tracking information fusion, a joint tracking method based on graph-long short-term memory neural nets (Graph-LSTMs) is proposed. Firstly, we use multivariable stochastic differential equations (SDE) to model the cooperative interaction of cluster members and transform the derived state space model of cluster members into the same form as the constant velocity (CV) motion model, and the target state equation of cluster which can be used for Bayesian filtering iteration is established. Secondly, based on the detection relationship between radars and cluster members, we introduce the detection confirmation matrix and propose a radar-target assignment method to achieve multiple measurements of single member and detection coverage of all cluster members. Then, each radar uses δ-GLMB filter to estimate the motion state of the assigned targets. Finally, on the basis of spatial discretization, the labels of multiple estimates of cluster member states are obtained. We use the designed Graph-LSTMs to learn the cooperative relationship between target states to fuse the labels and obtain better tracking effect. The experimental results show that the proposed method effectively simulates the cluster motion and realizes the joint estimation of cluster target motion state by multiradar. Our method makes up for the defect that a single radar cannot stably track adjacent multiple targets and achieves better estimation fusion effect than the expectation-maximization (EM) algorithm and mean method.

Pseudo Overlap Functions, Fuzzy Implications and Pseudo Grouping Functions with Applications

Overlap and grouping functions are important aggregation operators, especially in information fusion, classification and decision-making problems. However, when we do more in-depth application research (for example, non-commutative fuzzy reasoning, complex multi-attribute decision making and image processing), we find overlap functions as well as grouping functions are required to be commutative (or symmetric), which limit their wide applications. For the above reasons, this paper expands the original notions of overlap functions and grouping functions, and the new concepts of pseudo overlap functions and pseudo grouping functions are proposed on the basis of removing the commutativity of the original functions. Some examples and construction methods of pseudo overlap functions and pseudo grouping functions are presented, and the residuated implication (co-implication) operators derived from them are investigated. Not only that, some applications of pseudo overlap (grouping) functions in multi-attribute (group) decision-making, fuzzy mathematical morphology and image processing are discussed. Experimental results show that, in many application fields, pseudo overlap functions and pseudo grouping functions have greater flexibility and practicability.

Distributed Fusion Estimation for the Measurements with Bounded Disturbances

Abstract The information fusion problem is studied for multi-sensor systems in the presence of bounded disturbances. In this paper, a distributed fusion estimation algorithm is proposed based on the set-membership theory, which obtains the overall estimates based on multi-ellipsoids intersection. A parameter adaptive adjustment scheme is derived to guarantee the performance of the algorithm. The feedback mechanism is also introduced to enhance the estimation procedure. Through theoretical analysis and simulation, the performance of the proposed algorithm is analyzed, and some interesting properties of the proposed algorithm are proved. Results show that the proposed algorithm improves the point estimation accuracy. Compared with the algorithm without feedback, the one with feedback has better local estimation. Meanwhile, the effectiveness of the proposed algorithm in improving state estimation accuracy has been proved by the simulation results.

A post-processing algorithm based on factor graph optimization for autonomous underwater vehicle integrated navigation

Aiming at the requirements of offline high-precision navigation and positioning in the mapping of ocean floor, survey, and other fields for autonomous underwater vehicle (AUV)integrated navigation system, a post-processing algorithm based on factor graph optimization is proposed in this paper. First, the factor graph model is established by the factor graph formulation to address the asynchronous and heterogeneous problem of multi-source information fusion. Then the equivalent inertial measurement unit (IMU) factor is introduced to replace several consecutive IMU factors between two adjacent measurements to further aid high-rate navigation solutions. Second the forward and backward message passing processes are performed independently using the sum-product algorithm in the factor graph model. Finally, the optimal navigation solution is obtained by weighted smoothing the results of the forward and backward message passing processes. Simulation results show this algorithm proposed in this paper provides more accurate navigation accuracy and smoothness, especially in the case of signals loss or sensors fault. Compared with the federal Kalman filtering method, the horizontal positioning errors of the un-simplified factor graph, the simplified factor graph, and the factor graph optimization are improved by about 30%, 34%, and 80%, respectively. The semi-physical environment results also clearly verify the reliability and effectiveness of the proposed method, and its horizontal positioning accuracies are kept within −2∼2 m.

Qualitative capacities: Basic notions and potential applications

Qualitative capacities are set functions valued on a finite chain, increasing in the wide sense with respect to set inclusion. This paper exploits formal analogies between qualitative capacities and numerical capacities: we first recall qualitative counterparts to Möbius transforms, game-theoretic core, and conjugate set functions. However, in the qualitative setting, possibility measures play the same role as probability measures in the quantitative setting. Then possibility and necessity measures sometimes do not convey the same type of information. This situation creates difficulties to interpret qualitative capacities and related notions inspired from the quantitative setting. In particular, we propose three different ways of using qualitative capacities: either as bounds on ill-known possibility or necessity measures, or as tools to express the decision maker attitude in qualitative criteria under uncertainty, or yet qualitative counterparts of belief functions that handle both incompleteness and inconsistency of pieces of information stemming from several sources. In the latter framework possibility and necessity measures do not represent the same type of information. We define order relations between capacities with a view to compare them in terms of informational contents. We also study a counterpart of Dempster rule of combination in the qualitative setting. We compare several capacity combination rules in the framework of an information fusion problem. Finally, we address the problem of eliciting qualitative capacities based on human-originated information.

Thermal Fault Detection and Severity Analysis of Mechanical and Electrical Automation Equipment

Effectively diagnosing thermal faults in key parts of mechanical and electrical automation equipment before they become too serious is of crucial importance for the safe and continuous operation of these equipment. However, existing algorithms are not able to establish stable connections among sensors, so the overall control of thermal faults is not ideal enough. To cope with this issue, this paper aims to study the thermal fault detection of mechanical and electrical automation equipment and analyze the severity of the faults. At first, this paper studied the heterogeneous Multi-Sensor Information Fusion (MSIF) problem of sensors installed in key parts inside the mechanical and electrical automation equipment, and proposed a MSIF algorithm based on the D-S evidential theory. Then, the paper evaluated the influence of damages caused by thermal faults on the different parts of the equipment, providing evidences for the installation of sensors in key parts of the equipment. At last, experimental results proved the effectiveness of the proposed algorithm, and the thermal fault detection results were attained.

An Information Fusion Model between GeoSOT Grid and Global Hexagonal Equal Area Grid

In order to cope with the rapid growth of spatiotemporal big data, data organization models based on discrete global grid systems have developed rapidly in recent years. Due to the differences in model construction methods, grid level subdivision and coding rules, it is difficult for discrete global grid systems to integrate, share and exchange data between different models. Aiming at the problem of information fusion between a GeoSOT grid and global hexagonal equal area grid system, this paper proposes the GeoSOT equivalent aggregation model (the GEA model). We establish a spatial correlation index method between GeoSOT grids and global hexagonal equal area grids, and based on the spatial correlation index, we propose an interoperable transformation method for grid attributes information. We select the POI (points of interest) data of Beijing bus and subway stations and carry out the transformation experiment of hexagonal grid to GeoSOT grid information so as to verify the effectiveness of the GEA model. The experimental results show that when the 17th-level GeoSOT grid is selected as the particle grid to fit the hexagonal grid, the accuracy and efficiency can be well balanced. The fitting accuracy is 95.51%, and the time consumption is 30.9 ms. We establish the associated index of the GeoSOT grid and the hexagonal grid and finally realized the exchange of information.

Singing and Nervous System Regulation Based on Wireless Sensor Network Perception

In order to build an intelligent platform that can be applied to singing and nervous system adjustment, this paper optimizes the positioning and information processing algorithms for wireless sensor network perception. Moreover, this article combines binocular vision to realize the singer’s real-time positioning, combines the singer’s emotion recognition with the intelligent sensor system, and combines the emotion recognition with the adjustment of the nervous system, so that the singer can better control the intelligent platform. In addition, in order to solve the problem of multisensor information fusion, this paper improves the sensor fusion algorithm to make it suitable for the information fusion of vision sensors and information sensors. Finally, this paper designs the functional structure of the system, transmits data through wireless sensor networks, simulates human emotion models, and studies the process of singing and the adjustment of the nervous system. It can be seen from the experimental research results that the method proposed in this paper has a certain effect.