Information Fusion Strategy Research Articles

Early warning of reciprocating compressor valve fault based on deep learning network and multi-source information fusion

An early warning method of compressor valve fault based on multi-parameter signals (vibration, pressure, and temperature) is presented in this work. Due to the complexity working condition, the run data of the compressor are of problems like noise and feature aliasing, which makes it difficult to extract useful features and find out the running law from the original signals. In this work, an improved deep learning network Multi-Level Fusion long short-term memory based on Component Evaluating Empirical Mode Decomposition and Fuzzy C-Means (CEEMD-FCM & MLF-LSTM) for parameter prediction of reciprocating compressor and an information fusion strategy is proposed for compressor valve fault warning. The CEEMD-FCM & MLF-LSTM network consists of data processing block, information learning block, and prediction output block, which is mainly responsible for parameter prediction. In the data processing block, the CEEMD-FCM algorithm is used for parameter decomposition, noise removal, and fuzzy mode (FM) reconstruction, which generates the input for the information learning block to ensure the predicting accuracy and reduce model complexity. MLF-LSTM is constructed to predict the parameter in the future by learning the temporal and spatial characteristics of FMs of the run data. Then, an early warning strategy for compressor valve fault based on multi-source information fusion is developed. Experimental results have verified that the proposed CEEMD-FCM & MLF-LSTM model and early warning strategy could realize early warning of compressor valve fault effectively.

MLCA2F: Multi-Level Context Attentional Feature Fusion for COVID-19 lesion segmentation from CT scans.

In the field of diagnosis and treatment planning of Coronavirus disease 2019 (COVID-19), accurate infected area segmentation is challenging due to the significant variations in the COVID-19 lesion size, shape, and position, boundary ambiguity, as well as complex structure. To bridge these gaps, this study presents a robust deep learning model based on a novel multi-scale contextual information fusion strategy, called Multi-Level Context Attentional Feature Fusion (MLCA2F), which consists of the Multi-Scale Context-Attention Network (MSCA-Net) blocks for segmenting COVID-19 lesions from Computed Tomography (CT) images. Unlike the previous classical deep learning models, the MSCA-Net integrates Multi-Scale Contextual Feature Fusion (MC2F) and Multi-Context Attentional Feature (MCAF) to learn more lesion details and guide the model to estimate the position of the boundary of infected regions, respectively. Practically, extensive experiments are performed on the Kaggle CT dataset to explore the optimal structure of MLCA2F. In comparison with the current state-of-the-art methods, the experiments show that the proposed methodology provides efficient results. Therefore, we can conclude that the MLCA2F framework has the potential to dramatically improve the conventional segmentation methods for assisting clinical decision-making.

Information Fusion Strategies for Collaborative Inertial Radio SLAM

Information fusion strategies for vehicles navigating while aiding their inertial navigation systems (INSs) with terrestrial signals of opportunity (SOPs) are developed and studied. The following problem is considered. Multiple navigating vehicles with access to global navigation satellite system (GNSS) signals are aiding their on-board INSs with GNSS pseudoranges. While navigating, vehicle-mounted receivers draw pseudorange measurements from terrestrial SOPs (e.g., AM/FM radio, digital television, cellular) with unknown emitter positions and unknown and unsynchronized clocks. The vehicles share INS data and SOP pseudoranges to collaboratively estimate the SOPs’ states through an extended Kalman filter using tight coupling. After some time, GNSS signals become unavailable, at which point the navigating vehicles use shared INS and SOP information to continue navigating in a collaborative inertial radio simultaneous localization and mapping (CIRSLAM) framework. This paper develops such CIRSLAM framework and synthesizes what SOP and INS information should be shared between collaborators. Two information fusion strategies are compared: 1) sharing time-of-arrival (TOA) measurements from SOPs; 2) sharing time-difference-of-arrival (TDOA) measurements taken with reference to an SOP. Next, a strategy to efficiently share INS information along with SOP information is discussed. Monte Carlo simulation results are presented that support the analytical findings that vehicles navigating in a CIRSLAM framework, while sharing and fusing SOP TOA measurements, produce a smaller or equal estimation error covariance compared to fusing SOP TDOA measurements. Experimental results are presented demonstrating two unmanned aerial vehicles (UAVs) navigating in a CIRSLAM framework with SOP TOA measurements from terrestrial cellular towers. The final UAVs’ localization error after 30 seconds of GPS unavailability were reduced compared to using an INS alone from around 55 m to around 6 m.

A multiphase information fusion strategy for data-driven quality prediction of industrial batch processes

As one of the most important modes of industrial production, the batch process often involves complex and continuous physicochemical reactions, making it challenging to produce identical products between different batch runs even under the same working condition. Optimization and parameter adjustments depend mainly on a good quality prediction model. However, this industrial process has “3M” characteristics of multiple process variables, multiple production phases, and multiple quality indicators, which bring considerable challenges to the accuracy and robustness of the model. This study proposes a multiphase information fusion strategy for data-driven quality prediction of industrial batch processes. Firstly, aiming for real-world industrial datasets with different sampling frequencies, two types of state variables are summarized, and the multiphase-based cumulative quality model is developed. Secondly, information theory with copula entropy is employed to characterize the association relationships between each state variable and the set of multiple quality indicators; thus, phase-specific critical variables are selected by ranking copula entropy. Lastly, a stacking multiway random forest algorithm is proposed to develop the prediction relationship between phase-specific critical variables and multiple quality indicators. Experiments on a real-world industrial dataset have demonstrated that the proposed method has better accuracy and stronger robustness than previous baseline methods.

Supervised sparse preserving projections model based on distributed principal component analysis for chemical process monitoring

AbstractFor the monitoring of large‐scale chemical processes, the distributed method is often used to extract local feature information and model the extracted local feature information to obtain a process monitoring model. But the distributed process monitoring model often contains more process variables, which makes the local information of the process data flooded. To make up for the insufficient extraction of local information in traditional distributed process monitoring, supervised sparse preserving projections model based on distributed principal component analysis (DPCA‐SSPP) is proposed in this paper. First, the process data are decomposed by the PCA algorithm, and the principal component space and residual space are obtained. Second, the variables of each sub‐block are selected according to the maximum correlation criterion, and the SSPP process monitoring model is established for each sub‐block. Finally, the monitoring results of each sub‐block are combined together to form a global monitoring result through the Bayesian information fusion strategy. The proposed scheme can be proved to be effective through the simulation on a nonlinear numerical example and the Tennessee Eastman benchmark (TE) process.

Multi-robot collaborative radioactive source search based on particle fusion and adaptive step size

Multi-robot with nuclear radiation detectors plays a key role in nuclear safety and emergency disposal of radioactive contamination. However, the existing multi-robot radioactive source search process requires a large amount of computation, resulting in low search efficiency. This paper proposes a collaborative search method for unknown radioactive source through multi-robot information fusion and free energy strategy with adaptive step size. We use the cognitive difference principle to selectively fuse the measurement information of other robots to obtain a preliminary estimate of the probability distribution of radioactive source. Then, the multiple robots conduct particle fusion according to preset conditions to obtain accurate parameters of the radioactive source. Lastly, all of the robots cooperate to complete the search task of unknown radioactive source step by step through a free energy strategy with adaptive step size. Experimental results show that the search success rate of the proposed algorithm is improved to more than 95% with an efficient search.

A Comparative Study on Classification Features between High-Resolution and Polarimetric SAR Images through Unsupervised Classification Methods

Feature extraction and comparison of synthetic aperture radar (SAR) data of different modes such as high resolution and full polarization have important guiding significance for SAR image applications. In terms of image and physical domain for higher spatial resolution single-polarized and coarser spatial resolution quad-pol SAR data, this paper analyzes and compares the feature extraction with unsupervised classification methods. We discover the correlation and complementarity between high-resolution image feature and quad-pol physical scattering information. Therefore, we propose an information fusion strategy, that can conduct unsupervised learning of the landcover classes of SAR images obtained from multiple imaging modes. The medium-resolution polarimetric SAR (PolSAR) data and the high-resolution single-polarized data of the Gaofen-3 satellite are adopted for the selected experiments. First, we conduct the Freeman–Durden decomposition and H/alpha-Wishart classification method on PolSAR data for feature extraction and classification, and use the Deep Convolutional Embedding Clustering (DCEC) algorithm on single-polarized data for unsupervised classification. Then, combined with the quantitative evaluation by confusion matrix and mutual information, we analyze the correlation between characteristics of image domain and physics domain and discuss their respective advantages. Finally, based on the analysis, we propose a refined unsupervised classification method combining image information of high-resolution data and physics information of PolSAR data, that optimizes the classification results of both the urban buildings and the vegetation areas. The main contribution of this comparative study is that it promotes the understanding of the landcover classification ability of different SAR imaging modes and also provides some guidance for future work to combine their respective advantages for better image interpretation.

A fast multi-source information fusion strategy based on deep learning for species identification of boletes

Wild mushroom market is an important economic source of Yunnan province in China, and its wild mushroom resources are also valuable wealth in the world. This work will put forward a method of species identification and optimize the method in order to maintain the market order and protect the economic benefits of wild mushrooms. Here we establish deep learning (DL) models based on the two-dimensional correlation spectroscopy (2DCOS) images of near-infrared spectroscopy from boletes, and optimize the identification effect of the model. The results show that synchronous 2DCOS is the best method to establish DL model, and when the learning rate was 0.01, the epochs were 40, using stipes and caps data, the identification effect would be further improved. This method retains the complete information of the samples and can provide a fast and noninvasive method for identifying boletes species for market regulators.

Weakly Supervised Video Object Segmentation via Dual-attention Cross-branch Fusion

Recently, concerning the challenge of collecting large-scale explicitly annotated videos, weakly supervised video object segmentation (WSVOS) using video tags has attracted much attention. Existing WSVOS approaches follow a general pipeline including two phases, i.e., a pseudo masks generation phase and a refinement phase. To explore the intrinsic property and correlation buried in the video frames, most of them focus on the later phase by introducing optical flow as temporal information to provide more supervision. However, these optical flow-based studies are greatly affected by illumination and distortion and lack consideration of the discriminative capacity of multi-level deep features. In this article, with the goal of capturing more effective temporal information and investigating a temporal information fusion strategy accordingly, we propose a unified WSVOS model by adopting a two-branch architecture with a multi-level cross-branch fusion strategy, named as dual-attention cross-branch fusion network (DACF-Net). Concretely, the two branches of DACF-Net, i.e., a temporal prediction subnetwork (TPN) and a spatial segmentation subnetwork (SSN), are used for extracting temporal information and generating predicted segmentation masks, respectively. To perform the cross-branch fusion between TPN and SSN, we propose a dual-attention fusion module that can be plugged into the SSN flexibly. We also pose a cross-frame coherence loss (CFCL) to achieve smooth segmentation results by exploiting the coherence of masks produced by TPN and SSN. Extensive experiments demonstrate the effectiveness of proposed approach compared with the state-of-the-arts on two challenging datasets, i.e., Davis-2016 and YouTube-Objects.

Phase-Based Displacement Sensor With Improved Spatial Frequency Estimation and Data Fusion Strategy

Phase-based vibration measurement has attra- cted a lot of attention due to its advantages of wireless, non-contact, and full-field measurement capabilities. The phase shifts across the different video frames correspond to the motion of the structure, which makes it possible to measure the vibration with the phase. However, the decoding of the phase shifts and the presence of the phase singularities make the phase-based method noise-sensitive and non-robust when concerning vibration measurement. Within this paper, an improved framework is proposed to enhance the performance of the phase-based technique for vibration measurement. A double filtering approach combined with the O’Shea refinement is introduced to decrease the phase noise to obtain accurate spatial frequency estimates. Then a confidence measure index is constructed to evaluate the reliability of phase responses to avoid the outliers induced by the phase singularities. Finally, an information fusion strategy over multiple scales is developed to enhance the accuracy and robustness of the motion estimation. Both simulated and experimental results verify the effectiveness and accuracy of the proposed framework.

Multi-Fidelity Information Fusion to Model the Position-Dependent Modal Properties of Milling Robots

Robotic machining is a promising technology for post-processing large additively manufactured parts. However, the applicability and efficiency of robot-based machining processes are restricted by dynamic instabilities (e.g., due to external excitation or regenerative chatter). To prevent such instabilities, the pose-dependent structural dynamics of the robot must be accurately modeled. To do so, a novel data-driven information fusion approach is proposed: the spatial behavior of the robot’s modal parameters is modeled in a horizontal plane using probabilistic machine learning techniques. A probabilistic formulation allows an estimation of the model uncertainties as well, which increases the model reliability and robustness. To increase the predictive performance, an information fusion scheme is leveraged: information from a rigid body model of the fundamental behavior of the robot’s structural dynamics is fused with a limited number of estimated modal properties from experimental modal analysis. The results indicate that such an approach enables a user-friendly and efficient modeling method and provides reliable predictions of the directional robot dynamics within a large modeling domain.

Edge Aware Network for Image Dehazing

The learning-based methods have recently shown their advantages in the image dehazing task. However, most existing learning-based methods do not pay much attention to the restoration in the edges of the hazy image, resulting in the edge blur of the dehazing results. To mitigate this issue, in this letter, we propose a novel Edge Aware Network (EA-Net) for image dehazing, which can simultaneously model edge features and contextual features into a single network for restoring haze-free image with sharp edges. Firstly, we extract the multi-scales contextual features of hazy image by a progressive fusion way. Furthmore, the abundant edge features are inferred by a edge subnetwork with Edge Feature Extraction Module(EFEM). Finally, we present an Edge Attention (EA) mechanism to couple the edge features with contextual features at various resolutions for sufficiently leveraging these complementary features. Due to the rich edge information and feature fusion strategy, the fused features can make the haze-free image to be clearer, especially at the edges, which is very important for the high-level vision tasks. Extensive experiments demonstrate that the proposed method achieves significant improvements over the state-of-the-art methods.

A Novel Algorithm Based on a Common Subspace Fusion for Visual Object Tracking

Recent methods for visual tracking exploit a multitude of information obtained from combinations of handcrafted and/or deep features. However, the response maps derived from these feature combinations are often fused using simple strategies such as winner-takes-all or weighted sum approaches. Although some efficient fusion methods have also been proposed, these methods still do not leverage the individual strengths of the different features being fused. In the current work, we propose a novel information fusion strategy comprising a common low-rank subspace for the fusion of different types of features and tracker responses. Firstly, we interpret the response maps as smoothly varying functions which can be efficiently represented using individual low-rank matrices, thus removing high frequency noise and sparse artifacts. Secondly, we estimate a common low-rank subspace which is constrained to remain close to each individual low-rank subspace resulting in an efficient fusion strategy. The proposed algorithm achieves good performance by integrating the information contained in heterogeneous feature types. We demonstrate the efficiency of our algorithm using several combinations of features as well as correlation filter and end-to-end deep trackers. The proposed common subspace fusion algorithm is generic and can be used to efficiently fuse the response maps of varying types of feature representations as well as trackers. Extensive experiments on several tracking benchmarks including OTB100, TC128, VOT-ST 2018, VOT-LT 2018, UAV123, GOT-10K and LaSoT have demonstrated significant performance improvements compared to many SOTA tracking methods.

Research on Information Fusion of Computer Vision and Radar Signals in UAV Target Identification

As one of the crucial sensing methods, multisensor fusion recognition aids the Internet of Things (IoT) in connecting things through ubiquitous perceptual terminals. The small size, sluggish flying speed, low flight altitude, and low electromagnetic intensity of unmanned aerial vehicles (UAVs) have put enormous strain on air traffic management and airspace security. It is urgent to achieve effective UAV target detection. The radio monitoring method, acoustic detection scheme, computer vision, and radar signal detection are commonly used technologies in this field. The radio monitoring approach has low accuracy, the acoustic detection strategy has a limited detection range, computer vision is limited by weather conditions, and the radar signals at low altitudes are influenced by ground clutter. To address these issues, this paper proposes an information fusion strategy based on two levels of fusion: data‐level fusion and decision‐level fusion. In this strategy, Computer vision and radar signals complement each other to improve the detection accuracy. For each level, the method of information fusion is introduced in detail. Furthermore, the effectiveness of the method has been demonstrated by a series of comprehensive experiments. The results show that the accuracy of the fusion method is improved, and the proposed method can still work even when the single method loses function.

Multisource information fusion strategies of mass spectrometry and Fourier transform infrared spectroscopy data for authenticating the age and parts of Vietnamese ginseng

AbstractAiming at two different classification tasks in the field of quality evaluation of valuable Chinese herbal medicine, the applicability of data fusion strategy based on different complementary analysis techniques was studied. In this study, attenuated total reflection Fourier transform infrared spectroscopy (ATR‐FTIR) and ultra‐performance liquid chromatography quadrupole time‐of‐flight mass spectrometry (UPLC‐QTOF/MS) were used to analyze the different parts (including roots, stems, leaves, and fibrils) of 2‐to 5‐year‐old Vietnamese ginseng. The multivariate classification models (orthogonal partial least squares discrimination analysis, OPLS‐DA; support vector machine, SVM) were established using low‐level and mid‐level data fusion methods to identify different parts and age of Vietnamese ginseng. The OPLS‐DA model have shown that the fusion data matrix with low‐level fusion processing could separate Vietnamese ginseng samples with different growth years to the greatest extent, whereas the single data matrix had partial overlap. But 100% of prediction set classification accuracy could be achieved. The SVM model combined with two parameter optimization algorithms (grid search, GS; genetic algorithm, GA) was used to identify Vietnamese ginseng samples with different ages. The mid‐level fusion strategy based on the recursive feature elimination (RFE) features variable extraction method was more suitable for SVM model. In the model established by combining the two parameter optimization methods, the identification effect can reach 83.33%. The results showed that data fusion, as an effective strategy, could distinguish different ages and parts of Vietnamese ginseng.

Multi-Graph Heterogeneous Interaction Fusion for Social Recommendation

With the rapid development of online social recommendation system, substantial methods have been proposed. Unlike traditional recommendation system, social recommendation performs by integrating social relationship features, where there are two major challenges, i.e., early summarization and data sparsity. Thus far, they have not been solved effectively. In this article, we propose a novel social recommendation approach, namely Multi-Graph Heterogeneous Interaction Fusion (MG-HIF), to solve these two problems. Our basic idea is to fuse heterogeneous interaction features from multi-graphs, i.e., user–item bipartite graph and social relation network, to improve the vertex representation learning. A meta-path cross-fusion model is proposed to fuse multi-hop heterogeneous interaction features via discrete cross-correlations. Based on that, a social relation GAN is developed to explore latent friendships of each user. We further fuse representations from two graphs by a novel multi-graph information fusion strategy with attention mechanism. To the best of our knowledge, this is the first work to combine meta-path with social relation representation. To evaluate the performance of MG-HIF, we compare MG-HIF with seven states of the art over four benchmark datasets. The experimental results show that MG-HIF achieves better performance.

A Lightweight Fusion Distillation Network for Image Deblurring and Deraining.

Recently, deep learning-based image deblurring and deraining have been well developed. However, most of these methods fail to distill the useful features. What is more, exploiting the detailed image features in a deep learning framework always requires a mass of parameters, which inevitably makes the network suffer from a high computational burden. We propose a lightweight fusion distillation network (LFDN) for image deblurring and deraining to solve the above problems. The proposed LFDN is designed as an encoder–decoder architecture. In the encoding stage, the image feature is reduced to various small-scale spaces for multi-scale information extraction and fusion without much information loss. Then, a feature distillation normalization block is designed at the beginning of the decoding stage, which enables the network to distill and screen valuable channel information of feature maps continuously. Besides, an information fusion strategy between distillation modules and feature channels is also carried out by the attention mechanism. By fusing different information in the proposed approach, our network can achieve state-of-the-art image deblurring and deraining results with a smaller number of parameters and outperform the existing methods in model complexity.

PGNet: Progressive Feature Guide Learning Network for Three-dimensional Shape Recognition

Three-dimensional (3D) shape recognition is a popular topic and has potential application value in the field of computer vision. With the recent proliferation of deep learning, various deep learning models have achieved state-of-the-art performance. Among them, multiview-based 3D shape representation has received increased attention in recent years, and related approaches have shown significant improvement in 3D shape recognition. However, these methods focus on feature learning based on the design of the network and ignore the correlation among views. In this article, we propose a novel progressive feature guide learning network (PGNet) that focuses on the correlation among multiple views and integrates multiple modalities for 3D shape recognition. In particular, we propose two information fusion schemes from visual and feature aspects. The visual fusion scheme focuses on the view level and employs the soft-attention model to define the weights of views for visual information fusion. The feature fusion scheme focuses on the feature dimension information and employs the quantified feature as the mask to further optimize the feature. These two schemes jointly construct a PGNet for 3D shape representation. The classic ModelNet40 and ShapeNetCore55 datasets are applied to demonstrate the performance of our approach. The corresponding experiment also demonstrates the superiority of our approach.

A survey on dorsal hand vein biometrics

Biometrics technology is one of the most important and effective solutions for personal authentication. In recent years, as one of the emerging biometrics technologies, dorsal hand vein (DHV) biometrics has received a lot of attention. In fact, DHV biometrics has been studied for more than 30 years, during which different problems related to DHV recognition have been addressed. In this paper, we conduct a comprehensive survey on the state-of-the-art in DHV biometrics. Nearly all important aspects of DHV biometrics have been summarized, including the developmental history, data acquisition, databases, preprocessing algorithms, feature extraction and matching algorithms, information fusion schemes and commercial products. We also discuss the challenges and future directions in DHV biometrics research.

An intention multiple-representation model with expanded information

Short text is the main carrier for people to express their ideas and opinions. And it is very important as well a big challenge to understand the meaning of short text or recognize the semantic patterns of different short texts. Most of existing methods use word embedding and short text interaction to learn short text pairs semantic patterns. However, some of these methods are complicated and cannot fully capture the relations of words and the interaction of short text pairs. To solve this problem, a self-attention based model, that is, Knowledge Learning for Matching Question (KLMQ) is proposed. Using part-of-speech to mine the relations of words, and the model obtains the relations of short texts from the grammar, syntax and morphology. Meanwhile, it adopts information fusion strategy to enhance the interaction between short text pairs, which ensures the model works well with the expanded information, such as the order of words, the correlation of words and the relations of short text pairs. To verify the correctness of the proposed model and the efficiency of the expanded information, extensive experiments were carried out on public datasets. Experimental results show that the model performs better than that of traditional neural network models and the expanded information can much improve the performances of intention multiple-representation recognition.