Decision-level Information Fusion Research Articles

A Robust Target Detection Algorithm Based on the Fusion of Frequency-Modulated Continuous Wave Radar and a Monocular Camera

Decision-level information fusion methods using radar and vision usually suffer from low target matching success rates and imprecise multi-target detection accuracy. Therefore, a robust target detection algorithm based on the fusion of frequency-modulated continuous wave (FMCW) radar and a monocular camera is proposed to address these issues in this paper. Firstly, a lane detection algorithm is used to process the image to obtain lane information. Then, two-dimensional fast Fourier transform (2D-FFT), constant false alarm rate (CFAR), and density-based spatial clustering of applications with noise (DBSCAN) are used to process the radar data. Furthermore, the YOLOv5 algorithm is used to process the image. In addition, the lane lines are utilized to filter out the interference targets from outside lanes. Finally, multi-sensor information fusion is performed for targets in the same lane. Experiments show that the balanced score of the proposed algorithm can reach 0.98, which indicates that it has low false and missed detections. Additionally, the balanced score is almost unchanged in different environments, proving that the algorithm is robust.

Late better than early: A decision-level information fusion approach for RGB-Thermal crowd counting with illumination awareness

In this paper, we make the first research effort to address the RGB-Thermal (RGB-T) crowd counting problem with the decision-level late fusion manner. Being different from the existing pixel-level or feature-level fusion methods, our proposition chooses to fuse the density maps yielded by RGB and thermal counterparts via spatially adaptive weighting with RGB illumination-aware attention. Our key intuition to conduct RGB-T density map fusion lies in 2 main folders. First, compared with the raw RGB-T images or convolutional feature maps, RGB-T density maps contain stronger counting-wise semantic meanings. Secondly, they are also of high spatial resolution for revealing fine local details. To fuse them adaptively, a spatial weighting map for each modality, together with an illumination-related RGB weight is generated. In this way, the issues of RGB illumination awareness and local counting pattern characterization ability are concerned jointly. To the best of our knowledge, we are the first to leverage RGB-T crowd counting concerning these 2 issues in a unified way. Meanwhile, cross-modality feature interaction is conducted between RGB and thermal modalities to facilitate spatial weighting map generation. The experiments on 2 well-established RGB-T crowd counting datasets (i.e., RGBT-CC and DroneRGBT) verify the superiority of our proposition. The source code and pretrained models will be released upon acceptance at https://github.com/hustaia/DLF-IA.

Optimization of Innovative Paths of Physical Education Teaching in Primary and Secondary Schools under Information Integration Technology

Abstract This article utilizes information fusion technology to provide a more diversified and hierarchical teaching process for physical education, making the content of physical education classroom richer and more vivid. The article firstly discusses the multi-source heterogeneous information fusion technology. It proposes a mass function that combines the output results of the cumulative layer of probabilistic neural network to construct the D-S evidence theory. The D-S evidence theory realizes the decision-level information fusion of the primary results of probabilistic neural networks. In the study, the TPACK knowledge framework was combined with the informationization platform to establish an intelligent classroom for physical education teaching applicable to primary and secondary schools, and a teaching experiment was designed for the selected research subjects. The experiment was conducted to analyze the data mainly from three dimensions: teachers’ and students’ behaviors in the smart classroom, teaching effectiveness and teaching evaluation. The results of data analysis showed that the average time of each learning node of students at the school was more than 50 seconds. At the same time, the coding coverage of teachers’ teaching behaviors was 41.85% lower than the mean value of coding coverage of students’ learning behaviors. In the experimental group, students’ jump rope scores improved by about 9.05% to 20.44% after one month of learning. These findings suggest that an intelligent classroom for physical education teaching relying on information fusion technology can effectively integrate multi-source data, thus providing strong support for improving the effectiveness of physical education teaching in primary and secondary schools.

De-combination of belief function based on optimization

In the theory of belief functions, the evidence combination is a kind of decision-level information fusion. Given two or more Basic Belief Assignments (BBAs) originated from different information sources, the combination rule is used to combine them to expect a better decision result. When only a combined BBA is given and original BBAs are discarded, if one wants to analyze the difference between the information sources, evidence de-combination is needed to determine the original BBAs. Evidence de-combination can be considered as the inverse process of the information fusion. This paper focuses on such a defusion of information in the theory of belief functions. It is an under-determined problem if only the combined BBA is available. In this paper, two optimization-based approaches are proposed to de-combine a given BBA according to the criteria of divergence maximization and information maximization, respectively. The new proposed approaches can be used for two or more information sources. Some numerical examples and an example of application are provided to illustrate and validate our approaches.

Decision-Level Fusion with a Pluginable Importance Factor Generator for Remote Sensing Image Scene Classification

Remote sensing image scene classification acts as an important task in remote sensing image applications, which benefits from the pleasing performance brought by deep convolution neural networks (CNNs). When applying deep models in this task, the challenges are, on one hand, that the targets with highly different scales may exist in the image simultaneously and the small targets could be lost in the deep feature maps of CNNs; and on the other hand, the remote sensing image data exhibits the properties of high inter-class similarity and high intra-class variance. Both factors could limit the performance of the deep models, which motivates us to develop an adaptive decision-level information fusion framework that can incorporate with any CNN backbones. Specifically, given a CNN backbone that predicts multiple classification scores based on the feature maps of different layers, we develop a pluginable importance factor generator that aims at predicting a factor for each score. The factors measure how confident the scores in different layers are with respect to the final output. Formally, the final score is obtained by a class-wise and weighted summation based on the scores and the corresponding factors. To reduce the co-adaptation effect among the scores of different layers, we propose a stochastic decision-level fusion training strategy that enables each classification score to randomly participate in the decision-level fusion. Experiments on four popular datasets including the UC Merced Land-Use dataset, the RSSCN 7 dataset, the AID dataset, and the NWPU-RESISC 45 dataset demonstrate the superiority of the proposed method over other state-of-the-art methods.

Maximum weight multi-modal information fusion algorithm of electroencephalographs and face images for emotion recognition

In view of the low accuracy of the traditional emotion recognition methods based on facial expressions, an emotion recognition method based on maximum weight multi-modal information fusion of electroencephalographs (EEGs) and facial expression information is proposed in this paper. First, the induced emotional EEG data is converted into the corresponding EEG topographic map data and sent to the convolutional network for training and outputting decision information. Second, the illumination compensation method is utilized to filter the noise of the face image data. Then, the face image data is trained in the multi-scale feature extraction network, and the decision information is output. Finally, aiming at the decision-level information fusion, a weighted fusion method is proposed in this paper for emotion recognition. Experimental tests show that the recognition accuracy of the multi-scale feature extraction network on the CK+ data set and Fer2013 data reached 94.4% and 72%, respectively. Simultaneously, the multi-modal information fusion method achieves 92.6% accuracy in emotion recognition.

A Motor Current Signal-Based Bearing Fault Diagnosis Using Deep Learning and Information Fusion

Bearing fault diagnosis has extensively exploited vibration signals (VSs) because of their rich information about bearing health conditions. However, this approach is expensive because the measurement of VSs requires external accelerometers. Moreover, in machine systems that are inaccessible or unable to be installed in external sensors, the VS-based approach is impracticable. Otherwise, motor current signals (CSs) are easily measured by the inverters that are the available components of those systems. Therefore, the motor CS-based bearing fault diagnosis approach has attracted considerable attention from researchers. However, the performance of this approach is still not good as the VS-based approach, especially in the case of fault diagnosis for external bearings (the bearings that are installed outside of the electric motors). Accordingly, this article proposes a motor CS-based fault diagnosis method utilizing deep learning and information fusion (IF), which can be applied to external bearings in rotary machine systems. The proposed method uses raw signals from multiple phases of the motor current as direct input, and the features are extracted from the CSs of each phase. Then, each feature set is classified separately by a convolutional neural network (CNN). To enhance the classification accuracy, a novel decision-level IF technique is introduced to fuse information from all of the utilized CNNs. The problem of decision-level IF is transformed into a simple pattern classification task, which can be solved effectively by familiar supervised learning algorithms. The effectiveness of the proposed fault diagnosis method is verified through experiments carried out with actual bearing fault signals.

Current-Based Fault Detection and Identification for Wind Turbine Drivetrain Gearboxes

This paper proposes a new fault detection and identification framework for drivetrain gearboxes of wind turbines equipped with doubly-fed induction generators (DFIGs) based on the fusion of DFIG stator and rotor current signals. First, the characteristic frequencies of gearbox faults in DFIG stator and rotor currents are analyzed. Different time- and frequency-domain features of gearbox faults in DFIG stator and rotor current signals are then defined, and the methods to extract these features are introduced. These features are used as the inputs of multiclass support vector machines with probabilistic outputs for fault mode identification. Different schemes that use a single stator or rotor current signal or both stator and rotor current signals for the feature- or decision-level information fusion are designed. Experimental results obtained from a DFIG wind turbine drivetrain test rig are provided to validate the proposed current-based fault detection and identification framework.

Acoustic detection of cracks in the anvil of a large-volume cubic high-pressure apparatus.

A large-volume cubic high-pressure apparatus with three pairs of tungsten carbide anvils is the most popular device for synthetic diamond production. Currently, the consumption of anvils is one of the important costs for the diamond production industry. If one of the anvils is fractured during the production process, the other five anvils in the apparatus may be endangered as a result of a sudden loss of pressure. It is of critical importance to detect and replace cracked anvils before they fracture for reduction of the cost of diamond production and safety. An acoustic detection method is studied in this paper. Two new features, nested power spectrum centroid and modified power spectrum variance, are proposed and combined with linear prediction coefficients to construct a feature vector. A support vector machine model is trained for classification. A sliding time window is proposed for decision-level information fusion. The experiments and analysis show that the recognition rate of anvil cracks is 95%, while the false-alarm rate is as low as 5.8 × 10(-4) during a time window; this false-alarm rate indicates that at most one false alarm occurs every 2 months at a confidence level of 90%. An instrument to monitor anvil cracking was designed based on a digital signal processor and has been running for more than eight months in a diamond production field. In this time, two anvil-crack incidents occurred and were detected by the instrument correctly. In addition, no false alarms occurred.

A gender classification scheme based on multi-region feature extraction and information fusion for unconstrained images

Since gender classification has been interesting in many applications, we proposed a gender classification scheme based on multi-region feature extraction and information fusion in the paper. The proposed gender classification scheme is composed of three parts: pre-processing, multi-region feature extraction, and gender classifier. Before extracting useful information from multiple regions in a facial image, face detection and face orientation correction are performed in the pre-processing. Multi-region feature extraction measures three kinds of features from eyes, internal face, and hair. Since the three kinds of features have their particular properties, a classifier based on decision-level information fusion is utilized to combine these features for gender classification. To evaluate the proposed scheme, a large number of unconstrained images containing different-size faces are captured by using a low-cost webcam and digital cameras. Experimental results show that our proposed scheme can detect facial regions and the location of eyes well. Furthermore, the accuracy of the proposed gender classification scheme is higher than 96 %. These experimental results demonstrate that the proposed scheme can deal with unconstrained images for gender classification.

An improved poly(A) motifs recognition method based on decision level fusion

Polyadenylation is the process of addition of poly(A) tail to mRNA 3′ ends. Identification of motifs controlling polyadenylation plays an essential role in improving genome annotation accuracy and better understanding of the mechanisms governing gene regulation. The bioinformatics methods used for poly(A) motifs recognition have demonstrated that information extracted from sequences surrounding the candidate motifs can differentiate true motifs from the false ones greatly. However, these methods depend on either domain features or string kernels. To date, methods combining information from different sources have not been found yet. Here, we proposed an improved poly(A) motifs recognition method by combing different sources based on decision level fusion. First of all, two novel prediction methods was proposed based on support vector machine (SVM): one method is achieved by using the domain-specific features and principle component analysis (PCA) method to eliminate the redundancy (PCA–SVM); the other method is based on Oligo string kernel (Oligo-SVM). Then we proposed a novel machine-learning method for poly(A) motif prediction by marrying four poly(A) motifs recognition methods, including two state-of-the-art methods (Random Forest (RF) and HMM-SVM), and two novel proposed methods (PCA–SVM and Oligo-SVM). A decision level information fusion method was employed to combine the decision values of different classifiers by applying the DS evidence theory. We evaluated our method on a comprehensive poly(A) dataset that consists of 14,740 samples on 12 variants of poly(A) motifs and 2750 samples containing none of these motifs. Our method has achieved accuracy up to 86.13%. Compared with the four classifiers, our evidence theory based method reduces the average error rate by about 30%, 27%, 26% and 16%, respectively. The experimental results suggest that the proposed method is more effective for poly(A) motif recognition.

Multiple-feature Tracking Based on the Improved Dempster-Shafer Theory

Dempster-Shafer evidence theory is widely used in the fields of decision level information fusion. In order to overcome the problem of the counter-intuitive results encountered when using Dempster’s combination rule to combine the evidences which exist high conflict, a modified sequential weighted evidence combination is proposed. Firstly, the credibility of each evidence can obtained based on K-L distance, besides, the uncertainty of each evidence can obtained based on information entropy. Simultaneously, using the uncertainty of each evidence to improve the credibility of each evidence, then the weights of the bodies of evidence are obtained based on the improved credibility of each evidence, the weights generated are used to modify the bodies of evidence including the previous combination result, the previous evidence and the new arriving body of evidence at current step. Finally, according to the Dempster’s combination rule, the weighted average combination results can be obtained. In the experimental part, the improved method is used to fuse video multiple features in target tracking system and compared the results with the standard D-S theory. The simulation results show that the proposed method has better performance. DOI : http://dx.doi.org/10.11591/telkomnika.v12i1.3364

A multilevel information fusion approach for road congestion detection in VANETs

As city road congestion problems become more serious, many researchers have started to use the technique of vehicle ad hoc networks (VANETs) for road congestion detection. However, various on-board sensors equipped in vehicles may generate lots of atomic messages, which usually cause serious channel competition problems. In this paper, we propose a multilevel information fusion approach by combining the fuzzy clustering-based feature level information fusion (FCMA) and the modified Dempster–Shafer evidence reasoning-based decision level information fusion (D-SEMA). The FCMA can extract the key features from atomic messages, thereby greatly reducing the network traffic load. Furthermore, the D-SEMA mechanism is used to judge whether the road congestion event occurs. Performance analysis and simulation results under ONE simulator show that the proposed multilevel information fusion approach can detect road congestion efficiently with low bandwidth consumption.

Soft-sensing Method of Gas Flow Distribution of Blast Furnace Burden Surface Based on Multi-level Hierarchical Fusion Algorithm

Since the operation of blast furnace (BF) is closely related to the gas flow distribution in terms of stability, high production and low energy consumption, a soft-sensing method of gas flow distribution of BF burden surface based on multi-level hierarchical fusion is accordingly presented in order to solve the diffculty of direct detection for gas flow distribution. First, the multi-source information is registered to the same spatial dimension and temporal dimension. And then, the model of burden distribution and the model of burden surface temperature profile are respectively designed by means of data level fusion, which could reflect the gas distribution in the radial and axial directions. Finally, the gas flow distribution state is obtained by using the fuzzy decision level information fusion algorithm. According to the field simulation to verify the detection method, the method can accurately reflect the gas flow distribution of BF burden surface and offer a necessary guidance for the BF operation.