Decision Level Fusion Algorithm Research Articles

Combined Retrieval of Oil Film Thickness Using Hyperspectral and Thermal Infrared Remote Sensing

An outdoor experiment was conducted to measure the thickness of oil films (0~3000 μm) using an airborne hyperspectral imager and thermal infrared imager, and the spectral response and thermal response of oil films of different thicknesses were analyzed. The classic support vector regression (SVR) model was used to retrieve the oil film thickness. On this basis, the suitable range for retrieving oil film thickness using hyperspectral and thermal infrared remote sensing was explored, and the decision-level fusion algorithm was developed to fuse the retrieval capabilities of hyperspectral and thermal infrared remote sensing for oil film thickness. The following conclusions can be drawn: (1) Based on airborne hyperspectral data and thermal infrared data, the retrieval accuracy of oil films of different thicknesses reached 154.31 μm and 116.59 μm, respectively. (2) The S185 hyperspectral data were beneficial for retrieving thicknesses greater than or equal to 400 μm, and the H20T thermal infrared data were beneficial for retrieving thicknesses greater than 500 μm. (3) The result of the decision-level fusion model based on a fuzzy membership degree was superior to those obtained using a single sensor (hyperspectral or thermal infrared), indicating that it can better integrate the retrieval results of hyperspectral and thermal infrared remote sensing for oil film thickness. Furthermore, the feasibility of using hyperspectral and thermal infrared remote sensing to detect water-in-oil emulsions of different thicknesses was investigated through spectral response and thermal response analysis.

Ship Target Detection Algorithm Based on Decision-Level Fusion of Visible and SAR Images

Aiming at the problem of target detection for multiple source information fusion, in this article a decision-level fusion algorithm for visible and SAR images is proposed. Firstly, using the Faster-RCNN network to detect visible and SAR images to retain the detection results respectively. Secondly, the semantic segmentation of visible images based on U-Net is realized. Finally, based on the detection results of single source and semantic segmentation results of land and sea, a fusion strategy based on decision level is proposed to achieve accurate target detection under multi-source information. Through experimental verification, the detection performance of the proposed algorithm is advantage over that of single-source image detection. The detection accuracy is 2.87% and 4.73% higher, and the recall rate is 3.02% and 0.19% higher than that of visible and SAR images separately. Compared with other target detection algorithms based on traditional image fusion, the proposed method has fewer false detections and missed detections.

Developing a Physiological Signal-Based, Mean Threshold and Decision-Level Fusion Algorithm (PMD) for Emotion Recognition

With the development of computers, artificial intelligence, and cognitive science, engagement in deep communication between humans and computers has become increasingly important. Therefore, affective computing is a current hot research topic. Thus, this study constructs a Physiological signal-based, Mean-threshold, and Decision-level fusion algorithm (PMD) to identify human emotional states. First, we select key features from electroencephalogram and peripheral physiological signals, and use the mean-value method to obtain the classification threshold of each participant and distinguish individual differences. Then, we employ Gaussian Naive Bayes (GNB), Linear Regression (LR), Support Vector Machine (SVM), and other classification methods to perform emotion recognition. Finally, we improve the classification accuracy by developing an ensemble model. The experimental results reveal that physiological signals are more suitable for emotion recognition than classical facial and speech signals. Our proposed mean-threshold method can solve the problem of individual differences to a certain extent, and the ensemble learning model we developed significantly outperforms other classification models, such as GNB and LR.

Multi-Sensor Data Fusion and CNN-LSTM Model for Human Activity Recognition System

Human activity recognition (HAR) is becoming increasingly important, especially with the growing number of elderly people living at home. However, most sensors, such as cameras, do not perform well in low-light environments. To address this issue, we designed a HAR system that combines a camera and a millimeter wave radar, taking advantage of each sensor and a fusion algorithm to distinguish between confusing human activities and to improve accuracy in low-light settings. To extract the spatial and temporal features contained in the multisensor fusion data, we designed an improved CNN-LSTM model. In addition, three data fusion algorithms were studied and investigated. Compared to camera data in low-light environments, the fusion data significantly improved the HAR accuracy by at least 26.68%, 19.87%, and 21.92% under the data level fusion algorithm, feature level fusion algorithm, and decision level fusion algorithm, respectively. Moreover, the data level fusion algorithm also resulted in a reduction of the best misclassification rate to 2%~6%. These findings suggest that the proposed system has the potential to enhance the accuracy of HAR in low-light environments and to decrease human activity misclassification rates.

A cotton leaf nitrogen monitoring model based on spectral-fluorescence data fusion

In the present study, hyperspectral imaging and remote sensing of fluorescence were integrated to monitor the nitrogen content in leaves of drip-irrigated cotton at different growth periods in northern Xinjiang, China. Based on the spectrum and chlorophyll fluorescence parameters of nitrogen content in cotton leaves of different growth periods obtained through the shuffled frog-leaping algorithm (SFLA), the successive projection algorithm (SPA), grey relational analysis (GRA), and competitive adaptive reweighted sampling (CARS), a monitoring model of nitrogen content in cotton leaves was established via on hyperspectral imaging, chlorophyll fluorescence parameters, and spectral-fluorescence data fusion. The results showed that: (1) there were significant positive correlations between the chlorophyll fluorescence parameters Fv'/Fm', Fv/Fm, Yield, Fm, NPQ, and the nitrogen content at each growth period. (2) The effectiveness of chlorophyll fluorescence parameters in inversion of nitrogen content was the highest at the budding period and the blooming period, and the coefficients of determination (R2) of the validation sets were 0.745 and 0.709, respectively. (3) In the monitoring model for cotton leaf nitrogen in the blooming period that was established based on the decision-level algorithm and spectral-fluorescence data fusion, the R2 value of the training set reached 0.961, and that of the validation set was 0.828. In conclusion, the findings of this study suggest that the feature-level fusion and decision-level fusion algorithms of spectral-fluorescence data can effectively improve the accuracy and reliability of cotton leaf nitrogen monitoring.

Decision-level fusion detection method of visible and infrared images under low light conditions

Aiming at the problem of poor effect of object detection with visible images under low light conditions, the decision-level fusion detection method of visible and infrared images is studied. Taking YOLOX as the object detection network based on deep learning, a decision-level fusion detection algorithm of visible and infrared images based on light sensing is proposed. Experiments are carried out on LLVIP dataset, which is a visible-infrared paired dataset for low light vision. Through comparative analysis, it is found that the decision-level fusion algorithm based on Soft-NMS and light sensing obtained the optimal AP value of 69.0%, which is 11.4% higher than the object detection with visible images and 1.1% higher than the object detection with infrared images. The experimental results show that the decision-level fusion algorithm based on Soft-NMS and light sensing can effectively fuse the complementary information of visible and infrared images, and improve the object detection effect under low light conditions.

3D Vehicle Detection Algorithm Based on Multimodal Decision-Level Fusion

3D vehicle detection based on LiDAR-camera fusion is becoming an emerging research topic in autonomous driving. The algorithm based on the Camera-LiDAR object candidate fusion method (CLOCs) is currently considered to be a more effective decision-level fusion algorithm, but it does not fully utilize the extracted features of 3D and 2D. Therefore, we proposed a 3D vehicle detection algorithm based on multimodal decision-level fusion. First, project the anchor point of the 3D detection bounding box into the 2D image, calculate the distance between 2D and 3D anchor points, and use this distance as a new fusion feature to enhance the feature redundancy of the network. Subsequently, add an attention module: squeeze-and-excitation networks, weight each feature channel to enhance the important features of the network, and suppress useless features. The experimental results show that the mean average precision of the algorithm in the KITTI dataset is 82.96%, which outperforms previous state-of-the-art multimodal fusion-based methods, and the average accuracy in the Easy, Moderate and Hard evaluation indicators reaches 88.96%, 82.60%, and 77.31%, respectively, which are higher compared to the original CLOCs model by 1.02%, 2.29%, and 0.41%, respectively. Compared with the original CLOCs algorithm, our algorithm has higher accuracy and better performance in 3D vehicle detection.

Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera

In order to remedy the defects of single sensor in robustness, accuracy, and redundancy of target detection, this paper proposed a method for detecting obstacles in farmland based on the information fusion of a millimeter wave (mmWave) radar and a camera. Combining the advantages of the mmWave radar in range and speed measurement and the camera in type identification and lateral localization, a decision-level fusion algorithm was designed for the mmWave radar and camera information, and the global nearest neighbor method was used for data association. Then, the effective target sequences of the mmWave radar and the camera with successful data association were weighted to output, and the output included more accurate target orientation, longitudinal speed, and category. For the unassociated sequences, they were tracked as new targets by using the extended Kalman filter algorithm and were processed and output during the effective life cycle. Lastly, an experimental platform based on a tractor was built to verify the effectiveness of the proposed association detection method. The obstacle detection test was conducted under the ROS environment after solving the external parameters of the mmWave radar and the internal and external parameters of the camera. The test results show that the correct detection rate of obstacles reaches 86.18%, which is higher than that of a single camera with 62.47%. Furthermore, through the contrast experiment of the sensor fusion algorithms, the detection accuracy of the decision level fusion algorithm was 95.19%, which was higher than 4.38% and 6.63% compared with feature level and data level fusion, respectively.

An Improved Entropy-Weighted Topsis Method for Decision-Level Fusion Evaluation System of Multi-Source Data.

Due to the rapid development of industrial internet technology, the traditional manufacturing industry is in urgent need of digital transformation, and one of the key technologies to achieve this is multi-source data fusion. For this problem, this paper proposes an improved entropy-weighted topsis method for a multi-source data fusion evaluation system. It adds a fusion evaluation system based on the decision-level fusion algorithm and proposes a dynamic fusion strategy. The fusion evaluation system effectively solves the problem of data scale inconsistency among multi-source data, which leads to difficulties in fusing models and low fusion accuracy, and obtains optimal fusion results. The paper then verifies the effectiveness of the fusion evaluation system through experiments on the multilayer feature fusion of single-source data and the decision-level fusion of multi-source data, respectively. The results of this paper can be used in intelligent production and assembly plants in the discrete industry and provide the corresponding management and decision support with a certain practical value.

Gearbox Fault Diagnosis Based on Multi-Sensor and Multi-Channel Decision-Level Fusion Based on SDP

In order to deal with the shortcomings (such as poor robustness) of the traditional single-channel vibration signal in the comprehensive monitoring of the gearbox fault state, a multi-channel decision-level fusion algorithm was proposed based on symmetrized dot pattern (SDP) analysis, with the visual geometry group 16 network (VGG16) fault diagnosis model. Firstly, the SDP method was used to convert the vibration signal of a single multi-channel sensor into an imaging arm. Secondly, the obtained image arm was input into the VGG16 convolutional neural network in order to train the fault diagnosis model that can be obtained. Then, the SDP images of the signals that were to be measured from multiple multi-channel sensors were input into the fault diagnosis model, and the diagnosis results of multiple multi-channel sensors could then be obtained. Experimentally, it was demonstrated that the diagnostic results of multi-channel sensors one, two, and three were more accurate than those of single-channel sensors one, two, and three, by 3.01%, 16.7%, and 5.17%, respectively. However, the fault generation was not generated in a single direction, but rather multiple directions. In order to improve the comprehensiveness of the raw vibration data, a fusion method using DS (Dempster–Shafer) evidence theory was proposed in order to fuse multiple multi-channel sensors, in which the accuracy achieved 99.93% when sensor one and sensor two were fused, which was an improvement of 8.88% and 1.02% over single sensors one and two, respectively. When sensor one and sensor three were fused, the accuracy reached 99.31%, which was an improvement of 8.31% and 6.17% over single sensors one and three, respectively. When sensor two and sensor three were fused, the accuracy reached 99.91%, which was an improvement of 1.00% and 6.74% over single sensors two and three, respectively. When three sensors were fused simultaneously, the accuracy reached 99.99%, which was 8.93%, 1.08%, and 6.81% better than single sensors one, two, and three, respectively. Therefore, it can be proved that the number of sensor channels has a great influence on the diagnosis results.

LiDAR target fusion and algorithm detection based on improved YOLO

. In order to achieve safe driving behavior, the most important point of automatic driving is to detect the target. At present, the judgment of obstacles is based on single sensor, so it is difficult to detect some complex road environment, and it is easy to be missed. Therefore, this paper proposes another system device combined with color camera technology in lidar. This is another detection method proposed on the basis of Yolo, which improves the detection ability of small targets such as non motor vehicles and people. This is based on the Yolo algorithm, using images and other samples to obtain relevant useful data, and finally build the detection system model. Finally, the sensor is introduced to combine the color image and the deep image in order to improve the detection accuracy. Finally, the fusion of decision-making level is verified by test samples. The results show that the improved YOLO algorithm and decision-level fusion algorithm have higher target detection accuracy, can meet the real-time requirements, and can reduce the miss detection rate of small and weak targets such as non-motorized vehicles and pedestrians. Therefore, the method proposed in this paper has good performance and broad application prospects, while taking into account both accuracy and real-time.

Multimodal Decision-level Group Sentiment Prediction of Students in Classrooms

Sentiment analysis can be used to study an individual or a group’s emotions and attitudes towards other people and entities like products, services, or social events. With the advancements in the field of deep learning, the enormity of available information on internet, chiefly on social media, combined with powerful computing machines, it’s just a matter of time before artificial intelligence (AI) systems make their presence in every aspect of human life, making our lives more introspective. In this paper, we propose to implement a multimodal sentiment prediction system that can analyze the emotions predicted from different modal sources such as video, audio and text and integrate them to recognize the group emotions of the students in a classroom. Our experimental setup involves a digital video camera with microphones to capture the live video and audio feeds of the students during a lecture. The students are advised to provide their digital feedback on the lecture as ‘tweets’ on their twitter account addressed to the lecturer’s official twitter account. The audio and video frames are separated from the live streaming video using tools such as lame and ffmpeg. A twitter API was used to access and extract messages from twitter platform. The audio and video features are extracted using Mel-Frequency Cepstral Co-efficients (MFCC) and Haar Cascades classifier respectively. The extracted features are then passed to the Convolutional Neural Network (CNN) model trained on the FER2013 facial images database to generate the feature vector for classification of video-based emotions. A Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM), trained on speech emotion corpus database was used to train on the audio features. A lexicon-based approach with senti-word dictionary and learning based approach with custom dataset trained by Support Vector Machines (SVM) was used in the twitter-texts based approach. A decision-level fusion algorithm was applied on these three different modal schemes to integrate the classification results and deduce the overall group emotions of the students. The use-case of this proposed system will be in student emotion recognition, employee performance feedback, monitoring or surveillance-based systems. The implemented system framework was tested in a classroom environment during a live lecture and the predicted emotions demonstrated the classification accuracy of our approach.

ROAD RECOGNITION BASED ON DECISION LEVEL FUSION OF SAR AND OPTIC DATA

Abstract. Road recognition and extraction based on remotely sensed data is efficient and applicable in much urban management studies. In this research, the capabilities of SPOT and SAR images are investigated for road recognition. Spectral and textural similarities between roads and other urban objects such as building’s roofs many cause some difficulties in road recognition based on SPOT image. On the other hand, SAR images are good for small road recognition but, may have some difficulties for detecting roads among vegetation. The proposed method in this paper is a decision level fusion of SPOT and SAR classification results in order to modify extracted road regions. This method has three main steps; 1) texture feature extraction from each of the SPOT and SAR images, 2) classifying each of the SPOT and SAR images based on SVM classifier, 3) decision level fusion of classification results in order to reduce road recognition difficulties and having optimum road regions. Performing the capabilities of the proposed decision level fusion algorithm for road recognition can improve the quality of the classification for about 21%.

A testing framework for predictive driving features with an electronic Horizon

This paper proposes a novel approach for automated functional testing of map-based fusion algorithms in complex vehicle networks. The hybrid data representation of detailed digital maps and physical automotive sensors provides an extended view of the ego-vehicle environment and thereby facilitates improved inferences and more competent decision-making. It has therefore been instrumental in the ongoing development of predictive driving features, e.g. fuel-efficient driving, traffic sign recognition and highway-pilot. The presented approach utilises a closed-loop Hardware-in-the-Loop (HiL) co-simulation framework to evaluate the performance of the decision level fusion algorithms. The method contains both the structural design and resource-efficient integration into the HiL test bench in the example of traffic sign recognition. In reality, discrepancy between visual and map data is omnipresent due to map errors, old map data or optical detection failure. Through fault injection, defined inconsistencies can be produced within the HiL simulation environment. Amongst others, the fault injection covers the placement and value of traffic signs. These failures can be used for robustness testing of the fusion algorithms. Subsequently, a method for correlation analysis between field observations and synthetic simulation is realised to extend the requirements-based test coverage adaptively and systematically by modular and parameterised scenario specifications. In summary, the results show that the extended HiL environment is capable of generating electronic Horizon data which can easily be adapted or extended.

BMVT-M을 이용한 IR 및 SAR 융합기반 지상표적 탐지

Infrared (IR) target detection is one of the key technologies in Automatic Target Detection/Recognition (ATD/R) for military applications. However, IR sensors have limitations due to the weather sensitivity and atmospheric effects. In recent years, sensor information fusion study is an active research topic to overcome these limitations. SAR sensor is adopted to sensor fusion, because SAR is robust to various weather conditions. In this paper, a Boolean Map Visual Theory-Morphology (BMVT-M) method is proposed to detect targets in SAR and IR images. Moreover, we suggest the IR and SAR image registration and decision level fusion algorithm. The experimental results using OKTAL-SE synthetic images validate the feasibility of sensor fusion-based target detection.

Decision level fusion algorithm with fuzzy densities determined by object

The fuzzy densities of fuzzy integral are determined by the priori static information of labeled samples and cannot be dynamically adjusted by the recognition result of specific objects to be realistic.To overcome this disadvantage,a decision level fusion algorithm with fuzzy integral whose fuzzy densities were determined dynamically by objects was presented.In this algorithm,the fuzzy densities were determined dynamically by combining the discriminabiltity computed from the objective information given by the classifiers after recognizing the specific object and the priori static information.This algorithm was applied to facial expression recognition.The experimental results show that the proposed algorithm can obtain a better fusion result and raise the accuracy of expression recognition.

Feature-level and decision-level fusion of noncoincidently sampled sensors for land mine detection

We present and compare methods for feature-level (predetection) and decision-level (postdetection) fusion of multisensor data. This study emphasizes fusion techniques that are suitable for noncommensurate data sampled at noncoincident points. Decision-level fusion is most convenient for such data, but it is suboptimal in principle, since targets not detected by all sensors will not obtain the full benefits of fusion. A novel algorithm for feature-level fusion of noncommensurate, noncoincidently sampled data is described, in which a model is fitted to the sensor data and the model parameters are used as features. Formulations for both feature-level and decision-level fusion are described, along with some practical simplifications. A closed-form expression is available for feature-level fusion of normally distributed data and this expression is used with simulated data to study requirements for sample position accuracy in multisensor data. The performance of feature-level and decision-level fusion algorithms are compared for experimental data acquired by a metal detector, a ground-penetrating radar, and an infrared camera at a challenging test site containing surrogate mines. It is found that fusion of binary decisions does not perform significantly better than the best available sensor. The performance of feature-level fusion is significantly better than the individual sensors, as is decision-level fusion when detection confidence information is also available ("soft-decision" fusion).