Conventional Fusion Methods Research Articles

A fractal dimension based framework for night vision fusion

In this paper, a novel fusion framework is proposed for night-vision applications such as pedestrian recognition, vehicle navigation and surveillance. The underlying concept is to combine low-light visible and infrared imagery into a single output to enhance visual perception. The proposed framework is computationally simple since it is only realized in the spatial domain. The core idea is to obtain an initial fused image by averaging all the source images. The initial fused image is then enhanced by selecting the most salient features guided from the root mean square error &#x0028 RMSE &#x0029 and fractal dimension of the visual and infrared images to obtain the final fused image. Extensive experiments on different scene imaginary demonstrate that it is consistently superior to the conventional image fusion methods in terms of visual and quantitative evaluations.

A Pseudo Multi-Exposure Fusion Method Using Single Image

This paper proposes a novel pseudo multi-exposure image fusion method based on a single image. Multi-exposure image fusion is used to produce images without saturation regions, by using photos with different exposures. However, it is difficult to take photos suited for the multi-exposure image fusion when we take a photo of dynamic scenes or record a video. In addition, the multi-exposure image fusion cannot be applied to existing images with a single exposure or videos. The proposed method enables us to produce pseudo multi-exposure images from a single image. To produce multi-exposure images, the proposed method utilizes the relationship between the exposure values and pixel values, which is obtained by assuming that a digital camera has a linear response function. Moreover, it is shown that the use of a local contrast enhancement method allows us to produce pseudo multi-exposure images with higher quality. Most of conventional multi-exposure image fusion methods are also applicable to the proposed multi-exposure images. Experimental results show the effectiveness of the proposed method by comparing the proposed one with conventional ones.

Translate SAR Data into Optical Image Using IHS and Wavelet Transform Integrated Fusion

Although synthetic aperture radar (SAR) sensors function well at all times and under all weather conditions, the images they produce are not intuitively straightforward. A novel idea based on data fusion is introduced to translate SAR data into optical image in this paper. The proposed SAR-optical image translation is implemented using an intensity–hue–saturation (IHS) and wavelet transform integrated fusion (IHSW), so as to preserve as much as spatial details from SAR data, and minimize the spectral distortion of translated output. COSMO-SkyMed and ENVISAT-ASAR images are translated into optical images with the fusion of Landsat TM images, and the fusion results are compared with some conventional fusion methods, as well as the texture synthesis approach. Quality assessment of different fused outputs is carried out by using six quality indices. Visual and statistical comparisons of the final results indicate that the proposed approach achieves an effective translation from SAR to optical image and is superior to texture synthesis-based algorithm in terms of preserving spatial and spectral information. The proposed translation technique presents an alternative to improve the interpretability of SAR images.

Fusion of SAR and Multispectral Images Using Random Forest Regression for Change Detection

In order to overcome the insufficiency of single remote sensing data in change detection, synthetic aperture radar (SAR) and optical image data can be used together for supplementation. However, conventional image fusion methods fail to address the differences in imaging mechanisms and cannot overcome some practical limitations such as usage in change detection or temporal requirement of the optical image. This study proposes a new method to fuse SAR and optical images, which is expected to be visually helpful and minimize the differences between two imaging mechanisms. The algorithm performs the fusion by establishing relationships between SAR and multispectral (MS) images by using a random forest (RF) regression, which creates a fused SAR image containing the surface roughness characteristics of the SAR image and the spectral characteristics of the MS image. The fused SAR image is evaluated by comparing it to those obtained using conventional image fusion methods and the proposed method shows that the spectral qualities and spatial qualities are improved significantly. Furthermore, for verification, other ensemble approaches such as stochastic gradient boosting regression and adaptive boosting regression are compared and overall it is confirmed that the performance of RF regression is superior. Then, change detection between the fused SAR and MS images is performed and compared with the results of change detection between MS images and between SAR images and the result using fused SAR images is similar to the result with MS images and is improved when compared to the result between SAR images. Lastly, the proposed method is confirmed to be applicable to change detection.

Enhancing image visuality by multi-exposure fusion

Image visuality enhancement aims at increasing visual quality of a given image to convey more useful information. The key for visuality enhancement is to comprehensively exploit the details of the image scene. However, one (or several) observed image only provides partial information of the scene. To address this problem, we present a novel multi-exposure fusion based visuality enhancement method in this study. Firstly, we propose a simulated exposure model to synthesize the results of the observed image under different exposure conditions. Then, white balancing and image gradient are separately performed on those simulated exposure results. By doing these, scene details in different exposure conditions as well as image feature spaces (i.e., color and gradient spaces) can be well exploited. To appropriately take advantage of those resultant details, we adopt the Laplacian pyramid based linear fusion framework to integrate them into the enhanced image in a multi-scale way. Different from conventional fusion methods, we develop a more powerful weight map for fusion, which is able to simultaneously highlight pixels with good exposedness, contrast, saturation as well as Gamut. Experimental results demonstrate that the proposed method can effectively enhance the visuality of the observed image. Compared with existing methods, the proposed method well highlights the fine details as well as avoiding halo artifacts.

Microwave-assisted dissolution of highly refractory dysprosium-titanate (Dy2TiO5) followed by chemical characterization for major and trace elements using ICP-MS, UV-visible spectroscopy and conventional methods

Abstract Dysprosium-titanate (Dy2TiO5), being highly refractory in nature, its dissolution using conventional (hot-plate and fusion) methods is very difficult. Hence, for quantitative dissolution, a microwave method has been developed. The instrumental parameters and amount of acids has been optimized. Studies have been carried out for precise and accurate estimation of major elements such as Dy, Ti, and Mo. An anion exchange column has been used to separate Mo, Dy and Ti. Analysis of these elements has been carried out using ICP-MS, UV-visible spectroscopy, and gravimetric methods. In the developed method, precipitation of molybdenum and dysprosium has been done using α-benzoine oxime, and oxalic acid respectively. These precipitates have been converted into their respective oxide form. The purities of these oxides (Dy2O3 and MoO3) have been determined using ICP-MS. The method has been validated using synthetic samples where it is found that accuracy of Dy and Mo is >99% and precision is <1 (%RSD). The titanium has been determined using UV-visible spectroscopy with accuracy >98% and precision <2 (%RSD).

Multi-modality image fusion based on enhanced fuzzy radial basis function neural networks

In clinical applications, single modality images do not provide sufficient diagnostic information. Therefore, it is necessary to combine the advantages or complementarities of different modalities of images. Recently, neural network technique was applied to medical image fusion by many researchers, but there are still many deficiencies. In this study, we propose a novel fusion method to combine multi-modality medical images based on the enhanced fuzzy radial basis function neural network (Fuzzy-RBFNN), which includes five layers: input, fuzzy partition, front combination, inference, and output. Moreover, we propose a hybrid of the gravitational search algorithm (GSA) and error back propagation algorithm (EBPA) to train the network to update the parameters of the network. Two different patterns of images are used as inputs of the neural network, and the output is the fused image. A comparison with the conventional fusion methods and another neural network method through subjective observation and objective evaluation indexes reveals that the proposed method effectively synthesized the information of input images and achieved better results. Meanwhile, we also trained the network by using the EBPA and GSA, individually. The results reveal that the EBPGSA not only outperformed both EBPA and GSA, but also trained the neural network more accurately by analyzing the same evaluation indexes.

Automatic focus and fusion image algorithm using nonlinear correlation: Image quality evaluation

We propose a new autofocus and fusion algorithm to enhance the characteristics of the image in less run-time. The autofocus method selects the best-focused image (BFI) in a stack of images captured at different distances from the object. A vector is defined for each image in the stack, which contains elements selected by spiral scanning of the image. The spectrum of each vector is calculated using the Fourier transform then applying non-linear correlation to the reference vector spectrum and those of each of the corresponding to the images in the stack the BFI is determined. The fusion is carried out with a subset of images that have a focus measure value close to the BFI ones. The parabolic filter is applied to determine the relevant elements of the images that will be included in the fusion. The results evaluation is conducted using a multi-image metric, a quality measure that represents the percentage improvement of the fused image. The comparison was made with other fusion methods such as standard wavelets, resulting in that the autofocus and fusion algorithm (AFA) method obtained the highest quality indices. It is concluded that the AFA improves the quality of the images in less time than the conventional fusion methods.

Multi-focus image fusion method based on discrete Tchebichef transform and focus measure

Transform-based image fusion methods are widely used in multi-focus image fusion owing to their promising fusion effect, and their noise robustness. However, the time complexity of conventional transform-based image fusion methods is generally high. In this paper, a multi-focus image fusion method based on the discrete Tchebichef transform (DTT) and a focus measure is proposed. According to the relationship between DTT and a correlation analysis, the focus measure of image blocks in source images can be evaluated by limited low-order DTT coefficients. Hence, the source images are fused by the principle of maximum focus measure. Our experimental results show that the proposed method can reduce the fusion time while ensuring the fusion effect, and exhibit high noise robustness during image fusion.

An adaptive weighted fusion model with two subspaces for facial expression recognition

Automatic facial expression recognition has received considerable attention in the research areas of computer vision and pattern recognition. To achieve satisfactory accuracy, deriving a robust facial expression representation is especially important. In this paper, we present an adaptive weighted fusion model (AWFM), aiming to automatically determine optimal weighted values. The AWFM integrates two subspaces, i.e., unsupervised and supervised subspaces, to represent and classify query samples. The unsupervised subspace is formed by differentiated expression samples generated via an auxiliary neutral training set. The supervised subspace is obtained through the reconstruction of intra-class singular value decomposition based on low-rank decomposition from raw training data. Our experiments using three public facial expression datasets confirm that the proposed model can obtain better performance compared to conventional fusion methods as well as state-of-the-art methods from the literature.

Synthetic aperture radar and color visible images fusion algorithm based on region division strategy

Multisensor image fusion can be used as an advanced technique for image enhancement. A fusion method based on region division strategy is proposed for synthetic aperture radar (SAR) and color visible images. The SAR target feature detection function is first constructed to divide the images into the SAR target region and non-SAR target region. Meanwhile, SAR image despeckling is considered during the generation of this detection function. Then, two different fusion rules are designed for these two regions. More specifically, the fusion rule adopted in the SAR target region is fusing the high-frequency feature within the SAR image, and the fusion rule employed in the non-SAR target region is maintaining the spectral and detail information of the color visible image. Experimental results on several pairs of SAR and color visible images demonstrate the effectiveness of the proposed method. Compared with the conventional image fusion methods, the proposed method provides better results for all evaluation criteria, including spectral distortions, feature similarity, mutual information, and peak signal-to-noise ratio.

Infrared and visual image fusion method based on discrete cosine transform and local spatial frequency in discrete stationary wavelet transform domain

In order to promote the performance of infrared and visual image fusion and provide better visual effects, this paper proposes a hybrid fusion method for infrared and visual image by the combination of discrete stationary wavelet transform (DSWT), discrete cosine transform (DCT) and local spatial frequency (LSF). The proposed method has three key processing steps. Firstly, DSWT is employed to decompose the important features of the source image into a series of sub-images with different levels and spatial frequencies. Secondly, DCT is used to separate the significant details of the sub-images according to the energy of different frequencies. Thirdly, LSF is applied to enhance the regional features of DCT coefficients, and it can be helpful and useful for image feature extraction. Some frequently-used image fusion methods and evaluation metrics are employed to evaluate the validity of the proposed method. The experiments indicate that the proposed method can achieve good fusion effect, and it is more efficient than other conventional image fusion methods.

Near-infrared fusion via a series of transfers for noise removal

Near-infrared imaging has been considered as a solution for providing high quality photographs in dim lighting conditions. This imaging system captures two types of multimodal images, namely, near-infrared gray image (NGI) and visible color image (VCI). NGI is noise free and grayscale, whereas the VCI has colors and contains noise. Moreover, there are significant edge and brightness discrepancies between NGI and VCI. In order to deal with this problem, a new transfer-based fusion method is proposed for noise removal. Different from conventional fusion approaches, the proposed method conducts a series of transfers, namely, contrast, detail, and color transfers. First, the proposed contrast and detail transfers aim at solving the serious discrepancy problem to create a new noise-free and detail-preserving NGI. Second, the proposed color transfer models the unknown colors from the denoised VCI through a linear transform, and then transfers the natural-looking colors into the newly generated NGI. The experimental results show that the proposed transfer-based fusion method is highly successful in solving the discrepancy problem. The edges and textures are clearly described and the noise is completely removed from the fused images. Most of all, the proposed method is superior to the conventional fusion methods, guided filtering, and state-of-the-art fusion methods based on scale map and layer decomposition.

Fusion of synthetic aperture radar and visible images based on variational multiscale image decomposition

Multisensor image fusion can be used as an advanced technique for image enhancement. We propose a synthetic aperture radar (SAR) and visible image fusion method under the framework of a variational multiscale image decomposition (VMID) model. In the proposed method, two fusion rules are respectively designed to fuse the structure and texture components obtained with a VMID model of source images. A fusion rule based on the curvelet transform is employed for fusing the structure component and the local energy criterion is adopted to construct the texture component. Moreover, considering the influence of speckle noise present in an SAR image, its first two texture components are skipped. The final fused result is composed by the obtained structure and texture components. The experimental results on several pairs of SAR and visible images demonstrate the effectiveness of the proposed method. Compared with the conventional image fusion methods, our fused results have a finer structure and are more robust against speckle noise.

Satellite image fusion using fuzzy logic

Abstract Image fusion is a method of combining the Multispectral (MS) and Panchromatic (PAN) images into one image contains more information than any of the input. Image fusion aim is to decrease unknown and weaken common data in the fused output image at the same time improving necessary information. Fused images are helpful in various applications like, remote sensing, computer vision, biometrics, change detection, image analysis and image classification. Conventional fusion methods are having some side effects like assertive spatial information and uncertain color information is an usually the problem in PCA and wavelet transform based fusion is a computationally in depth process. In order to overcome these side effects and to propose alternative soft computing fusion approach for conventional fusion methods we exploit image fusion using fuzzy logic technique to fuse two source images obtained from different sensors to enhance both spectral and spatial information. The proposed work here further compared with two common fusion methods like, principal component analysis (PCA) and wavelet transform along with quality assessment metrics. Exploratory outputs demonstrated in order that fuzzy based image fusion technique can actively retains more information compared to PCA and wavelet transform approaches while enhancing the spatial and spectral resolution of the satellite images.

A novel infrared and visible image fusion algorithm based on shift-invariant dual-tree complex shearlet transform and sparse representation

In this paper, a novel shift-invariant dual-tree complex shearlet transform (SIDCST) is constructed and applied to infrared and visible image fusion. Firstly, the mathematical morphology is used for the source images. Then, the images are decomposed by SIDCST to obtain the low frequency sub-band coefficients and high frequency sub-band coefficients. For the low frequency sub-band coefficients, a novel sparse representation (SR)-based fusion rule is presented. For the high frequency sub-band coefficients, a scheme based on the theory of adaptive dual-channel pulse coupled neural network (2APCNN) is presented, and the energy of edge is used for the external input of 2APCNN. Finally, the fused image is obtained by performing the inverse SIDCST. The experimental results show that the proposed approach can obtain state-of-the-art performance compared with conventional image fusion methods in terms of both objective evaluation criteria and visual quality.

Spectral–Spatial Classification of Hyperspectral Images Using Probabilistic Weighted Strategy for Multifeature Fusion

A hyperspectral image classification method based on probabilistic weighted fusion of multiple spectral–spatial features is proposed in this letter. First, dimensionality reduction and feature extraction of hyperspectral images are conducted by minimum noise fraction. Then, two spectral–spatial features are composed through a combination of texture features and multiscale morphological features with characteristic images obtained by minimum noise fraction. Next, a support vector machine classifier is employed to classify each spectral–spatial feature. Finally, a probabilistic weighted fusion model is established and applied for probabilistic fusion of the classification output of every single feature. The proposed method is validated by classification experiments using Reflective Optics Spectrographic Imaging System (ROSIS) and Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) images, which achieves the overall accuracy of 96.32% and 96.49%, respectively. The results indicate that the accuracy of hyperspectral image classification is improved effectively by the proposed method and the method has superior classification performance to other conventional multifeature fusion methods.

FF-SKPCCA: Kernel probabilistic canonical correlation analysis

Several information fusion methods are developed for increasing the recognition accuracy in multimodal systems. Canonical correlation analysis (CCA), cross-modal factor analysis (CFA) and their kernel versions are known as successful fusion techniques but they cannot digest the data variability. Probabilistic CCA (PCCA) is suggested as a linear fusion method to capture input variability. A new kernel PCCA (KPCCA) is proposed here to capture both the nonlinear correlations of sources and input variability. The functionality of KPCCA decreases when the number of samples, which determines the size of kernel matrix increases. In the conventional fusion methods the latent variables of different modalities are concatenated; consequently, a large-scale covariance matrix with just limited number of samples must be estimated To overcome this drawback, a sparse KPCCA (SKPCCA) is introduced which scarifies the covariance matrix elements at the cost of decreasing its rank. In the final stage of the gradual evolution of KPCCA, a new feature fusion manner is proposed for SKPCCA (FF-SKPCCA) as a second stage fusion. This proposed method unifies the latent variables of two modalities into a feature vector with an acceptable size. Audio-visual databases like M2VTS (for speech recognition) eNTERFACE and RML (for emotion recognition) are applied to assess FF-SKPCCA compared to state-of-the-art fusion methods. The comparative results indicate the superiority of the proposed method in most cases.

Ultrasound-assisted rapid and efficient one-pot synthesis of furanyl spirooxindolo and spiroquinoxalinopyrrolizidines by 1,3-dipolar cycloaddition: a green protocol

Efficient synthesis of novel spirooxindolo and spiroquinoxalinopyrrolizidine derivatives was expediently accomplished with regioselectivity via one-pot, three-component 1,3-dipolar cycloaddition using ultrasonication. Chalcones derived from both heteroaryl methyl ketones and furfural were used as dipolarophiles in these reactions. The synthesized compounds were analyzed by 1H and 13C nuclear magnetic resonance (NMR), mass spectrometry, and elemental (CHN) analysis. Single-crystal X-ray diffraction studies of one of the compounds (11d) proved the structure and regiochemistry of the cycloaddition. The ultrasound methodology is clearly advantageous, and the desired products were obtained in moderate to good yield in shorter reaction time compared with conventional heating and fusion methods.

Fusion of infrared and visible images based on BEMD and NSDFB

This paper presents a new fusion method based on the adaptive multi-scale decomposition of bidimensional empirical mode decomposition (BEMD) and the flexible directional expansion of nonsubsampled directional filter banks (NSDFB) for visible-infrared images. Compared with conventional multi-scale fusion methods, BEMD is non-parametric and completely data-driven, which is relatively more suitable for non-linear signals decomposition and fusion. NSDFB can provide direction filtering on the decomposition levels to capture more geometrical structure of the source images effectively. In our fusion framework, the entropies of the two patterns of source images are firstly calculated and the residue of the image whose entropy is larger is extracted to make it highly relevant with the other source image. Then, the residue and the other source image are decomposed into low-frequency sub-bands and a sequence of high-frequency directional sub-bands in different scales by using BEMD and NSDFB. In this fusion scheme, two relevant fusion rules are used in low-frequency sub-bands and high-frequency directional sub-bands, respectively. Finally, the fused image is obtained by applying corresponding inverse transform. Experimental results indicate that the proposed fusion algorithm can obtain state-of-the-art performance for visible-infrared images fusion in both aspects of objective assessment and subjective visual quality even for the source images obtained in different conditions. Furthermore, the fused results have high contrast, remarkable target information and rich details information that are more suitable for human visual characteristics or machine perception.