Weighted Fusion Rule Research Articles

A novel multi-sensor hybrid fusion framework

Multi-sensor data fusion has emerged as a powerful approach to enhance the accuracy and robustness of diagnostic systems. However, effectively integrating multiple sensor data remains a challenge. To address this issue, this paper proposes a novel multi-sensor fusion framework. Firstly, a vibration signal weighted fusion rule based on Kullback–Leibler divergence-permutation entropy is introduced, which adaptively determines the weighting coefficients by considering the positional differences of different sensors. Secondly, a lightweight multi-scale convolutional neural network is designed for feature extraction and fusion of multi-sensor data. An ensemble classifier is employed for fault classification, and an improved hard voting strategy is proposed to achieve more reliable decision fusion. Finally, the superiority of the proposed method is validated using modular state detection data from the Kaggle database.

Infrared-visible image fusion method based on multi-scale shearing Co-occurrence filter

Infrared-visible image fusion aims to achieve salient targets and abundant texture details. However, most of fusion methods based on multi-scale decomposition fail to effectively preserve detailed and texture information. To avoid these issues, a novel fusion method is proposed by means of multi-scale shearing Co-occurrence filter (MSCOF). This multi-scale decomposition tool possesses the flexible multi-direction and shift-invariant properties of the Shearlet, as well as the smoothing advantages of the Co-occurrence filter (COF). In this study, we obtain approximate and detail subbands by performing MSCOF decomposition on the original images. A weighted fusion rule based on the saliency intensity value (SIV) is performed to combine approximate subbands. Notably, COF is employed on the saliency intensity maps to better preserve edge information of the original image on the fused approximate subbands. Meanwhile, the improved local gradient energy (ILGE) method is utilized to fuse the detail subbands. The results of experimental analysis fully prove that our method is reliable and feasible, and that the obtained objective evaluation metrics are superior to those of some typical methods.

Multi-branch convolutional attention network for multi-sensor feature fusion in intelligent fault diagnosis of rotating machinery

Multi-sensor data fusion approaches based on deep learning are widely used for fault diagnosis of rotating machinery. Massive sensor data bring not only abundant information but also technical challenges for the fault diagnosis. The main challenge is how to discriminate the discrepancies between multi-sensor data and efficiently fuze these sensor data to improve diagnostic performance. To overcome the challenge, a novel multi-branch convolutional attention network (MBCAN) is proposed for fault diagnosis of the rotating machinery. In this method, a feature extractor with attention mechanism is established to extract fault-related features from the sensor data and reduce irrelevant noise interference hidden in the sensor data. Meanwhile, a multi-branch convolutional pathway is designed to enrich the features. Furthermore, a feature fusion module is constructed by defining an adaptive weighted fusion rule to fuze the extracted features. The performance of the MBCAN is verified on gearbox and centrifugal pump under different noise environments. The experimental results show that the proposed MBCAN has more excellent anti-noise ability and reliable diagnostic performance than other existing approaches.

Medical Image Fusion using ECNN- and OMBO-based Adaptive Weighted Fusion Rule

Medical imaging and information processing technologies are constantly evolving, resulting in a wide range of multimodality therapeutic pictures for clinical illness investigation. Physicians often require medical images produced using various modalities such as computed tomography (CT), magnetic resonance (MR), and positron emission computed tomography (PET) for clinical diagnosis. Many deep learning-based fusion methods have recently been proposed. In Convolutional Neural Network (CNN)-based fusion methods, only the last layer results are used as the image features, which result in the loss of useful information at middle layers. The fusion rule, based on the weighted averaging, causes noises in the source images and suppresses salient features of the image. In order to solve these issues, this paper proposes medical image fusion using Enhanced CNN (ECNN)- and Opposition-based Monarch Butterfly Optimization (OMBO)-based adaptive weighted fusion rule (AWFR). The ECNN contains feature extraction and reconstruction components. Both these components are trained in order to minimize the pixel loss and structural similarity loss. A pair of multimodal medical image is passed as input to the ECNN model to extract the low level and high level features. For the extracted features from ECNN, weighted fusion rule is applied in which OMBO algorithm is applied to adaptively optimize the weights of the fusion rule.

Rolling bearing fault diagnosis method based on multi-sensor two-stage fusion

Despite the great achievements of deep learning methods based on a single sensor in fault diagnosis, learning useful information from multi-sensor data is still a challenge. In order to make full use of multi-sensor information and improve the performance of rolling bearing fault diagnosis, a novel multi-sensor information fusion framework is proposed in this paper. First, a multi-sensor-based multi-frequency information fusion method is proposed. The multi-frequency information of each sensor is segmented first to enhance the datasets, and then a weighted fusion rule based on fuzzy entropy is constructed to fuse the information of different frequency components for multi-sensors. Second, a multi-kernel attention convolutional neural network is designed to realize multi-frequency feature capture, fusion, and fault classification of multi-sensors. Finally, two different rolling bearing datasets are used to implement fault diagnosis experiments. Experimental results show that the proposed method outperforms the comparative methods in terms of diagnostic performance and robustness.

Multi-Band and Polarization SAR Images Colorization Fusion

The image fusion of multi-band and multi-polarization synthetic aperture radar (SAR) images can improve the efficiency of band and polarization information processing. In this paper, we introduce a fusion method that simultaneously fuses multi-band and polarization SAR images. In the method, we first use non-subsampled shearlet transform (NSST) to fuse multi-band and polarization SAR images. The sub-band images decomposed from the NSST are fused by the coefficient of variation (CV) and phase consistency (PC) weighted fusion rules. Subsequently, we extract the band and polarization difference information from the multi-band and polarization SAR images. The fusion image is finally colorized according to the band and polarization differences. In the experiments, we used Ka and S-band multi-polarization SAR images to test the fusion performance. The experiment results prove that the proposed fused images not only preserve much valuable information but also can be interpreted easily.

Innovation of Visual Communication Design of Interactive Packaging for Internet-Famous Food Based on Artificial Intelligence

The combination of visual communication design of food packaging and digital media art is the new trend. In this study, the visual communication design of interactive packaging of Internet-famous food is analyzed, and an innovative identification method of the interactive packaging visual communication design based on artificial intelligence is put forward. The weighted fusion rule of conjunction disjunction double operators is used to fuse the innovative evaluation information of multiple expert groups on interactive packaging, and finally, the innovative multigroup discrimination results of interactive packaging are obtained. Considering both the consistency of evaluation information and the contradiction of evaluation information, the weight factor is introduced to achieve the balance between conjunction operator and disjunction operator. The designed conjunctive disjunctive double operator weighted fusion rule can support the contradiction handling in the innovative discrimination of interactive packaging and has the advantages of convenient application and easy programming.

A novel visible-infrared image fusion method based on visual enhancement and multiscale decomposition

Aiming at the problems of low contrast, low signal-to-noise ratio and scattered energy in the field of optical detection and imaging, a visual enhancement and multi-scale decomposition method for fusion of visible and infrared images is proposed based on the visual characteristics of different bands’ image. Firstly, the infrared image with less texture information and low contrast is preprocessed to weaken the background noise and improve the visual contrast. On this basis, the multi-scale image is decomposed by detail preserving filter, and the saliency map of each scale image is obtained by saliency extraction method. Then, the multi-source images of each scale are fused with saliency map, and binary filtering fusion rules are adopted for the regions with salient details while weighted fusion rules are adopted for other regions. Finally, the images of all scale are reconstructed to get better fusion results. The experimental results show that this method can significantly improve the visual contrast of the fused object, and the objective evaluation indexes are superior to other comparison methods.

2MNet: Multi-sensor and multi-scale model toward accurate fault diagnosis of rolling bearing

Rolling bearing is an indispensable element of rotating machinery, timely and accurate fault diagnosis of rolling bearing plays an important role in the safe and reliable operation of modern industrial systems. Considering the bottleneck that the information collected by a single sensor and single scale features extracted by conventional networks are not comprehensive, a multi-sensor and multi-scale model (2MNet) is proposed to bring a new perspective to accurate fault diagnosis. Most notably, multi-sensor vibration signals in three directions can be fused by defining a novel correlation kurtosis weighted fusion rule. Furthermore, the implication of multi-scale is twofold: one is the multi-scale feature extraction by optimizing the conventional deep residual network and adding dilated convolution, and the other is to achieve multi-scale feature fusion by combining the pyramid principle which can connect deep and shallow features. The superiority and applicability of the model are confirmed by numerical simulation and rolling bearing data.

Multi-Modality Image Fusion in Adaptive-Parameters SPCNN Based on Inherent Characteristics of Image

Due to the characteristics of the diversity and complement of multi-modality images, the fused method in SPCNN of adaptive parameters based on inherent characteristics of image is proposed in this paper, in order to extract the optimal feature information from the different images. Based on the analysis of traditional pulse coupled neural network, the simplified PCNN model based on SCM is designed. In order to highlight the different feature of different modality images, the adaptive parameters of SPCNN are set on the basis of the inherent characteristics of images, and the fused method in NSST is proposed. Firstly, the source image is decomposed by NSST with multiple scales and multiple directions. In the high-frequency coefficients, the SPCNN with adaptive parameters is adopted to calculate the firing time of each coefficient, the high-frequency coefficients are fused by comparing the firing times. In order to protect the details and edge information of the low-frequency coefficients, the combining of regional energy and energy of gradient is regarded as the active level, the low-frequency coefficients is fused by the weighted fusion rule. Finally, the fused image is reconstructed by inverse NSST transform. The experimental results show that the proposed method is superior to the other five classical methods, the fused results of the proposed method are consistence in the human visual perception system. The contrast and detail of fused image are perfect.

An Infrared and Visible Image Fusion Algorithm Based on LSWT-NSST

Regarding the problems of image distortion, edge blurring, Gibbs phenomena in the traditional wavelet transform algorithm and the loss of subtle features in the Non-Subsampled Shearlet Transform (NSST), and considering the physical characteristics of infrared and visible images, an infrared and visible image fusion algorithm based on the Lifting Stationary Wavelet Transform (LSWT) and Non-Subsampled Shearlet Transform is proposed in this paper. First, since LSWT can quickly calculate and has all advantages of traditional WT, it is utilized to decompose infrared and visible images to obtain low-frequency coefficients and multi-scale and multi-directional high-frequency coefficients, respectively. Second, NSST multi-scale decomposition is used to extract the target features and detailed features of the image from the high and low-frequency sub-bands to obtain new high and low-frequency sub-bands. Third, according to the physical characteristics that low and high-frequency coefficients represent, different fusion rules are designed. Discrete Cosine Transform (DCT) and Local Spatial Frequency (LSF) are introduced in the low-frequency sub-band, and LSF adaptive weighted fusion rules are used in the DCT domain. The fusion strategy improves the regional contrast in the high-frequency sub-band with the spectral characteristics of human vision. Finally, the Inverse Lifting Stationary Wavelet Transform (ILSWT) is used to reconstruct the fusion coefficients to obtain the final fused images. To verify the advantages of the proposed algorithm in this paper, the classic and advanced 9 IR and VI fusion algorithms are selected for subjective and objective comparison. In the objective evaluation, a comprehensive ranking index is designed based on 9 classical indicators. Simulation experiments with 10 IR and VI fusion algorithms prove that the proposed algorithm has better performance and flexibility. The results show that the proposed algorithm in this paper fuses the images with clear edges, prominent targets, and good visual perception, and it outperforms state-of-the-art image fusion algorithms.

Efficient DWT based Fusion Algorithm for Improving Contrast and Edge Preservation

The main principle of infrared (IR) image is that it captures thermal radiation of light. The objects that are captured in low light, fog, and snow conditions can be detected clearly in IR image. But the major drawback of IR image is that it provides poor resolution and low texture information. Due to that humans are unable to understand overall scene information present in IR image. Nowadays for the detection of objects in poor weather conditions with improved texture information, the result of visible (VI) and IR image fusion is used. It is mostly used in military, surveillance, and remote sensing applications. The efficient DWT based fusion algorithm for improving contrast and edge preservation is presented in this paper. First morphology hat transform is applied on source images for improving contrast. DWT on decomposition produces low frequency and high frequency sub-bands. A novel mean weighted fusion rule is introduced in this paper for fusing low frequency sub-bands. Its aim is to improve the visual quality of final fused image. The max fusion rule has used for fusing high frequency sub-bands to improve edge information. The final fused image is reconstructed by using IDWT. In this paper, the proposed fusion algorithm has produced improved results both subjectively and as well as objectively when compared to existing fusion methods.

Kernel-Correlated Filtering Target Tracking Algorithm Based on Multi-Features Fusion

When the color of the target is similar to the color of the background or illumination of a scene, scale, and shape of the target change significantly, the traditional kernel-correlated filtering (KCF) algorithm often fails to track the target. Therefore, a multi-features fusion target tracking algorithm based on KCF is proposed. The algorithm makes the most of the illumination and rotation invariance of local binary patterns (LBP) feature, the effective capability of extracting color and contour information from the target by color Name (CN) feature and histogram of oriented gradient (HOG) feature, respectively. The multi-features linear weighted fusion rule is established by calculating the HOG, LBP, and CN histograms of the current frame, and calculating the weighted fusion coefficients of the normalized histograms according to the variance of each histogram adaptively. On this basis, multi-scale sampling is used in the target areas and bilinear interpolation is used to adjust the sample size, which can improve the tracking robustness of the KCF algorithm to adapt to the target scale. The comparison experimental results of 50 video data sets in OTB-2015 show that the average success rate and precision of the proposed algorithm are 24.74% and 8% higher than that of the KCF tracking algorithm, respectively, and also significantly better than the other tracking algorithms ranked top ten in OTB-2015, UAV123, and LaSOT data sets, respectively.

Multi-sensor data fusion using NIHS transform and decomposition algorithms

Multi-spectral image fusion is to enhance the details present in multi-spectral bands with the spatial information available in the panchromatic image. Fused images have the effect of spectral distortions and lack of structural similarity. To overcome these limitations, three methods are proposed using intensity, hue, saturation (IHS) and nonlinear IHS (NIHS) transform along with the Dynamic Mode Decomposition (DMD) and 2D-Empirical Mode Decomposition (2D-EMD or IEMD). An intensity plane is calculated from the NIHS transform. The modes are constructed using DMD by considering the variations between the intensity plane computed using NIHS transforms of a low resolution multi-spectral image and a panchromatic image. Similarly, 2D-EMD is also used for image fusion. Modes are subjected to weighted fusion rule to get an intensity plane with spatial and edge information. Finally, the calculated intensity plane is concatenated along with the hue and saturation plane of low-resolution multi-spectral image and transformed into RGB color space. Thus, the fused images have high spatial and edge information on spectral bands. The experiments and its quality assessment assure that proposed methods perform better than the existing methods.

Robust Sparse Representation Combined With Adaptive PCNN for Multifocus Image Fusion

Multifocus image fusion utilizes the clear parts of multiple images from the same scene to generate a new image, which includes more information about the scene than any of the individual source images. In this paper, a multifocus image fusion method based on robust sparse representation (RSR) and an adaptive pulse-coupled neural network (PCNN) is presented. Each source image is first decomposed with RSR to obtain a sparse coefficients matrix and a residuals matrix. Second, the spatial frequency of the residuals matrix is calculated as the motivation for PCNN neurons, and a salience map of the source image is proposed as an adaptive linking strength for the PCNN. The initial decision map is acquired by comparing the ignition frequency maps of the source images obtained through PCNN. Then, the final decision map is achieved through morphological opening and closing operations. Finally, the fused image is obtained by employing a weighted fusion rule. Experimental results indicate that the proposed method is effective and better than other existing popular fusion methods regarding both objective and subjective evaluations.

Pleiades Satellite Remote Sensing Image Fusion Algorithm Based on Shearlet Transform

According to the characteristics of multi-spectral and panchromatic remote sensing images, this paper proposes a new image fusion algorithm based on nonsubsampled shearlet transform. Firstly, low frequency sub-band and high frequency directional sub-band is obtained by IHS color space conversion, and nonsubsampled shearlet transform apply to original multi-spectral images. Then, the adaptive weighted fusion rules is used to construct the low frequency sub-band coefficients, and regional consistency check and region-based local variance fusion rule are used to construct the high frequency sub-band coefficients. Lastly, the inverse of nonsubsampled shearlet transform and the inverse IHS transform are applied to the sub-band coefficients and then obtained the fused image. The comparison of simulation results with traditional fusion methods indicates that the proposed method have better fusion performance both in spectral characteristics and spatial resolution.

Simultaneous image compression, fusion and encryption algorithm based on compressive sensing and chaos

In this paper, a novel approach based on compressive sensing and chaos is proposed for simultaneously compressing, fusing and encrypting multi-modal images. The sparsely represented source images are firstly measured with the key-controlled pseudo-random measurement matrix constructed using logistic map, which reduces the data to be processed and realizes the initial encryption. Then the obtained measurements are fused by the proposed adaptive weighted fusion rule. The fused measurement is further encrypted into the ciphertext through an iterative procedure including improved random pixel exchanging technique and fractional Fourier transform. The fused image can be reconstructed by decrypting the ciphertext and using a recovery algorithm. The proposed algorithm not only reduces data volume but also simplifies keys, which improves the efficiency of transmitting data and distributing keys. Numerical results demonstrate the feasibility and security of the proposed scheme.

Infrared and visible image fusion using multiscale directional nonlocal means filter

Fusion of infrared and visible images is a significant research area in image analysis and computer vision. The purpose of infrared and visible image fusion is to combine the complementary image information of the source images into a fused image. Thus, it is vital to efficiently represent the important image information of the source images and choose rational fusion rules. To achieve this aim, an image fusion method using multiscale directional nonlocal means (MDNLM) filter is proposed in this paper. The MDNLM combines the feature of preserving edge information by the nonlocal means filter with the capacity of capturing directional image information by the directional filter bank, which can effectively represent the intrinsic geometric structure of images. The MDNLM is a multiscale, multidirectional, and shift-invariant image decomposition method, and we use it to fuse infrared and visible images in this paper. First, the MDNLM is discussed and used to decompose the source images into approximation subbands and directional detail subbands. Then, the approximation and directional detail subbands are fused by a local neighborhood gradient weighted fusion rule and a local eighth-order correlation fusion rule, respectively. Finally, the fused image can be obtained through the inverse MDNLM. Comparison experiments have been performed on different image sets, and the results clearly demonstrate that the proposed method is superior to some conventional and recent proposed fusion methods in terms of the visual effects and objective evaluation.

Infrared and Visible Light Image Fusion Algorithm Based on Compressed Sensing

The Nonsubsampled Contourlet Transform (NSCT) is translation invariant. It also has good line singular characteristics capturing feature in the image processing. The compressed sensing theory has good noise suppression capability. On the basis, this study proposed a novel compressed sensing image fusion algorithm in NSCT transform domain for the infrared image and visible light image fusion. This algorithm adopted the pixel feature energy weighted fusion rule and the neighborhood variance feature information fusion rule for the low-frequency coe‐cients and high-frequency coe‐cients fusion. The compressed sensing algorithm used the Toeplitz matices to observe the high-frequency sub-band coe‐cients. The experiment results show that this method efiectively decreases the amount of processing data, improves the convergence rate, and raises the fusion efiect.

A novel fusion scheme for visible and infrared images based on compressive sensing

An appropriate fusion of infrared and visible images can integrate their complementary information and obtain more reliable and better description of the environmental conditions. Compressed sensing theory, as a low signal sampling and compression method based on the sparsity of signal under a certain transformation, is widely used in various fields. Applying to the image fusion applications, only a part of sparse coefficients are needed to be fused. Furthermore, the fused sparse coefficients can be used to accurately reconstruct the fused image. The CS-based fusion approach can greatly reduce the computational complexity and simultaneously enhance the quality of the fused image. In this paper, an improved image fusion scheme based on compressive sensing is presented. This proposed approach can preserve more detail information, such as edges, lines and contours in comparison to the conventional transform-based image fusion approaches. In the proposed approach, the sparse coefficients of the source images are obtained by discrete wavelet transform. The low and high coefficients of infrared and visible images are fused by an improved entropy weighted fusion rule and a max-abs-based fusion rule, respectively. The fused image is reconstructed by a compressive sampling matched pursuit algorithm after local linear projection using a random Gaussian matrix. Several comparative experiments are conducted. The experimental results show that the proposed image fusion scheme can achieve better image fusion quality than the existing state-of-the-art methods.