Simple Image Fusion Research Articles

A multi-channel neural network model for multi-focus image fusion

The objective of multi-focus image fusion (MFIF) is to generate a fully focused image through integrating multiple partially focused source images. Most of the existing methods do not fully consider the local gradient variation rate of the source image, which makes it difficult to accurately distinguish the small defocused (focused) region covered by the large focused (defocused) region. In addition, these methods also cause edge blurring because they do not take into account misregistration of the source images. To address these issues, in this paper, we propose a simple and effective multi-focus image fusion framework based on multi-channel Rybak neural network (MCRYNN) model. Specifically, the proposed MCRYNN model is highly sensitive to local gradient changes of images based on input receptive fields, which can process multiple source images in parallel and extract the features of focused regions. Moreover, the decision maps can accurately be generate in proposed method based on the information interaction effect of parallel network structure for multi-focus image fusion task. Finally, we conduct qualitative and quantitative experiments on public datasets, and the results show that the performance of the proposed method outperforms the state-of-the-art methods.

Gaussian of Differences: A Simple and Efficient General Image Fusion Method.

The separate analysis of images obtained from a single source using different camera settings or spectral bands, whether from one or more than one sensor, is quite difficult. To solve this problem, a single image containing all of the distinctive pieces of information in each source image is generally created by combining the images, a process called image fusion. In this paper, a simple and efficient, pixel-based image fusion method is proposed that relies on weighting the edge information associated with each pixel of all of the source images proportional to the distance from their neighbors by employing a Gaussian filter. The proposed method, Gaussian of differences (GD), was evaluated using multi-modal medical images, multi-sensor visible and infrared images, multi-focus images, and multi-exposure images, and was compared to existing state-of-the-art fusion methods by utilizing objective fusion quality metrics. The parameters of the GD method are further enhanced by employing the pattern search (PS) algorithm, resulting in an adaptive optimization strategy. Extensive experiments illustrated that the proposed GD fusion method ranked better on average than others in terms of objective quality metrics and CPU time consumption.

Stepwise Fusion of Hyperspectral, Multispectral and Panchromatic Images with Spectral Grouping Strategy: A Comparative Study Using GF5 and GF1 Images

Since hyperspectral satellite images (HSIs) usually hold low spatial resolution, improving the spatial resolution of hyperspectral imaging (HSI) is an effective solution to explore its potential for remote sensing applications, such as land cover mapping over urban and coastal areas. The fusion of HSIs with high spatial resolution multispectral images (MSIs) and panchromatic (PAN) images could be a solution. To address the challenging work of fusing HSIs, MSIs and PAN images, a novel easy-to-implement stepwise fusion approach was proposed in this study. The fusion of HSIs and MSIs was decomposed into a set of simple image fusion tasks through spectral grouping strategy. HSI, MSI and PAN images were fused step by step using existing image fusion algorithms. According to different fusion order, two strategies ((HSI+MSI)+PAN and HSI+(MSI+PAN)) were proposed. Using simulated and real Gaofen-5 (GF-5) HSI, MSI and PAN images from the Gaofen-1 (GF-1) PMS sensor as experimental data, we compared the proposed stepwise fusion strategies with the traditional fusion strategy (HSI+PAN), and compared the performances of six fusion algorithms under three fusion strategies. We comprehensively evaluated the fused results through three aspects: spectral fidelity, spatial fidelity and computation efficiency evaluation. The results showed that (1) the spectral fidelity of the fused images obtained by stepwise fusion strategies was better than that of the traditional strategy; (2) the proposed stepwise strategies performed better or comparable spatial fidelity than traditional strategy; (3) the stepwise strategy did not significantly increase the time complexity compared to the traditional strategy; and (4) we also provide suggestions for selecting image fusion algorithms using the proposed strategy. The study provided us with a reference for the selection of fusion strategies and algorithms in different application scenarios, and also provided an easy-to-implement solution and useful references for fusing HSI, MSI and PAN images.

A simple and effective multi-focus image fusion method based on local standard deviations enhanced by the guided filter

This paper develops a simple and effective multi-focus image fusion method based on local standard deviations of the corresponding Laplacian images of source images and further enhanced by the guided filter. The underlying idea of this pixel-based approach is that the sharper pixels generally should have a comparatively higher local variance and hence higher local standard deviation. We first apply the Laplacian operation on each partially focused source image of the same scene, estimate the local standard deviation for each pixel, and enhance the local standard deviations using the guided filter. We then employ the filtered local standard deviation of the Laplacian image as an initial focus measure and combine it with the small region removal strategy to construct a decision map for pixel selection. Finally, according to the decision map, the target all-in-focus fused image is formed pixel-by-pixel. A variant of the proposed method with further guided filtering on the decision map is also developed. Numerical results demonstrate the proposed methods’ high performance compared with some state-of-the-art techniques.

Fusion of infrared and visible images using neutrosophic fuzzy sets

Fusion of infrared and visible image is a technology which combines information from two different sensors for the same scene. It also gives extremely effective information complementation, which is widely used for the monitoring systems and military fields. Due to limited field depth in an imaging device, visible images can’t identify some targets that may not be apparent due to poor lighting conditions or because that the background color is similar to the target. To deal with this problem, a simple and efficient image fusion approach of infrared and visible images is proposed to extract target’s details from infrared images and enhance the vision in order to improve the performance of monitoring systems. This method depends on maximum and minimum operations in neutrosophic fuzzy sets. Firstly, the image is transformed from its spatial domain to the neutrosophic domain which is described by three membership sets: truth membership, indeterminacy membership, and falsity membership. The indeterminacy in the input data is handled to provide a comprehensive fusion result. Finally, deneutrosophicised process is made which means that the membership values are retransformed into a normal image space. At the end of the study, experimental results are applied to evaluate the performance of this approach and compare it to the recent image fusion methods using several objective evaluation criteria. These experiments demonstrate that the proposed method achieves outstanding visual performance and excellent objective indicators.

Joint image fusion and super-resolution for enhanced visualization via semi-coupled discriminative dictionary learning and advantage embedding

In recent years, image fusion has attracted more and more attention, and many excellent methods have emerged. However, only a few studies on joint image fusion and super-resolution have been carried out, and the performance of existing methods is far from that of simple image fusion. To tackle such problem, we propose a novel joint fusion and super-resolution framework based on discriminative dictionary learning. Specifically, we first jointly learn two pairs of low-rank and sparse dictionaries (LRSD) and a conversion dictionary. One pair is used to represent the low-rank and sparse components of low-resolution input images, and the other is used to reconstruct high-resolution fused result; the conversion dictionary is used to establish the relationship between coding coefficients of low-resolution image and high-resolution image. To compensate for the loss of details, structure information compensation dictionary (SICD) is also learned, and the lost information is compensated by SICD and thus visualization of final results is enhanced. To integrate advantages of excellent image fusion methods into the fused and reconstructed results, we propose a deconvolution-based advantage embedding scheme. The experimental results verify the effectiveness and advantages of our method over other competitive ones.

Multifocus Image Fusion Based on Fast Guided Filter and Focus Pixels Detection

As an effective technique of information fusion, multifocus image fusion has attracted increasing attention for image processing and computer vision. The goal of multifocus image fusion method is integrating all focus information from source images into fused result. In this paper, we propose a novel simple and effective multifocus image fusion technique based on fast guided filter and focus pixels detection. In order to detect the focused pixels correctly from source images, we develop a new multi-scale sum modified Laplacian technique. For the decision maps learning stage, based on the block consistency verification and faster guided filter techniques, we generate a series of binary focus decision maps to express the focus property of each pixel. In the fusion stage, we employ neighbor distance filter to extract detail pixels of source images, then the informative highpass images and the energetic lowpass images can be generated. The fused results are developed by constructing the corresponding decision maps and the neighbor distance filtered images. Compared with some state-of-the-art fusion methods, experimental results clearly demonstrate the superiority of the proposed method in terms of both comprehensive subjective assessment and some well-known quantitative evaluations.

An Investigative Approach and Analysis on Fusion Techniques of Images for Medical Beneficial Applications

Objectives: Image fusion is the practice of combining appropriate data in sequence from a position of images into a distinct image and the fused or combined image contains supplementary information than any of the contributed image. Methods: The most important methods of the image fusion engages the pyramid based image fusion, simple image fusion, and wavelet based image fusion. To compute the quality and excellence of images is for purpose of evaluation of image fusion performance measures of Entropy, Peak signal to Noise ratio ,Correlation Coefficient(CC), RMS inaccuracy, SD(Standard Deviation), Edge Detection which is considered , High Pass Correlation of Image, Average Gradient of image has been introduced. Entropy is for the determination of data informational quantity, Peak signal to Noise ratio is for the evaluation of image error, Correlation Coefficient is utilized to come across with the similarities connecting the contributed and the fused complex image, RMS inaccuracy is collective noise sandwiched between the fused and the innovative input image Findings: In this research paper an analysis is done on images with the approach of image fusion techniques of wavelet transform and focuses on their assessment and evaluation based on the superiority of the harvested or produced image. Conclusion: The outputs are verified that performance in terms of lesser entropy and greater PSNR gives common sequence of information. Here SWT displays good performance and high-quality performance is always obtained by using wavelet transform. Wavelet transform has enhanced capability to recognize the border path feature and superior Medical Image analysis.

Enhanced disease characterization through multi network functional normalization in fMRI.

Conventionally, structural topology is used for spatial normalization during the pre-processing of fMRI. The co-existence of multiple intrinsic networks which can be detected in the resting brain are well-studied. Also, these networks exhibit temporal and spatial modulation during cognitive task vs. rest which shows the existence of common spatial excitation patterns between these identified networks. Previous work (Khullar et al., 2011) has shown that structural and functional data may not have direct one-to-one correspondence and functional activation patterns in a well-defined structural region can vary across subjects even for a well-defined functional task. The results of this study and the existence of the neural activity patterns in multiple networks motivates us to investigate multiple resting-state networks as a single fusion template for functional normalization for multi groups of subjects. We extend the previous approach (Khullar et al., 2011) by co-registering multi group of subjects (healthy control and schizophrenia patients) and by utilizing multiple resting-state networks (instead of just one) as a single fusion template for functional normalization. In this paper we describe the initial steps toward using multiple resting-state networks as a single fusion template for functional normalization. A simple wavelet-based image fusion approach is presented in order to evaluate the feasibility of combining multiple functional networks. Our results showed improvements in both the significance of group statistics (healthy control and schizophrenia patients) and the spatial extent of activation when a multiple resting-state network applied as a single fusion template for functional normalization after the conventional structural normalization. Also, our results provided evidence that the improvement in significance of group statistics lead to better accuracy results for classification of healthy controls and schizophrenia patients.

Near-infrared and visible light image fusion algorithm for face recognition

In order to improve face recognition accuracy, we present a simple near-infrared (NIR) and visible light (VL) image fusion algorithm based on two-dimensional linear discriminant analysis (2DLDA). We first use two such schemes to extract two classes of face discriminant features of each of NIR and VL images separately. Then the two classes of features of each kind of images are fused using the matching score fusion method. At last, a simple NIR and VL image fusion approach is exploited to combine the scores of NIR and VL images and to obtain the classification result. The experimental results show that the proposed NIR and VL image fusion approach can effectively improve the accuracy of face recognition.

Discrete Cosine Transform based fusion of multi-focus images for visual sensor networks

This paper presents a simple and efficient multi-focus image fusion scheme explicitly designed for wireless visual sensor systems equipped with resource constrained, battery powered image sensors employed in surveillance, hazardous environment like battlefields etc. Here the fusion of multi-focus images is based on higher valued Alternating Current (AC) coefficients calculated in Discrete Cosine Transform (DCT) domain. The proposed method overcomes the computation and energy limitation of low power devices and is investigated in terms of image quality and computation energy. Simulations are performed using Atmel Atmega128 processor of Mica 2 mote to measure the resultant energy savings. The experimental results verify the significant efficiency improvement of the proposed method in output quality and energy consumption, when compared with other fusion techniques in DCT domain.

Infrared Target Detection and Location for Visual Surveillance Using Fusion Scheme of Visible and Infrared Images

The main goal of image fusion is to combine substantial information from different images of the same scene into a single image that is suitable for human and machine perception or for further image-processing tasks. In this study, a simple and efficient image fusion approach based on the application of the histogram of infrared images is proposed. A fusion scheme to select adaptively weighted coefficients for preserving salient infrared targets from the infrared image and for obtaining most spatial detailed information from the visible image is presented. Moving and static infrared targets in the fused image are labeled with different colors. This technique enhances perception of the image for the human visual system. In view of the modalities of infrared images, low resolution, and low signal-to-noise ratio, an anisotropic diffusion equation model is adopted to remove noise and to effectively preserve edge information before the fusion stage. By using the proposed method, relevant spatial information is preserved and infrared targets are clearly identified in the resulting fused images.

Graphic-processing-unit-accelerated real-time exposure fusion method using pixel-level optimal exposure criterion

High dynamic range (HDR) imaging is an important and challenging research topic in computational photography. A simple but effective image fusion method is proposed to accomplish the multi-exposure image composition in both static and dynamic scenes. The foundation of the proposed method is an experiential criterion that optimizes the exposure that occurs at a dramatic alteration point in the low dynamic range image sequence (LDRI). To extract these well-exposed pixel vectors, each pixel curve formed by the pixel vectors at same position along all frames in the LDRIs is first preprocessed by the chord length parameterization. Then a single high-quality pseudo-HDR image can be extracted directly and efficiently from the LDRIs using a pixel-level fusion index matrix derived from the first- and second-order difference quotients of the preprocessed pixel curves. The main advantage of the proposed method is its use of a single independent pixel in computing. It is highly parallel, allowing a graphic processing unit-based, real-time implementation. The experiments on various scenes discussed here indicate that the proposed exposure fusion method can combine a large image sequence with 10 megapixels into a visually compelling pseudo-HDR image at a rate of 30 frames/s on a consumer hardware.

The complex bidimensional empirical mode decomposition

A new method for computing complex bidimensional empirical mode decomposition (BEMD) is presented in this paper. The proposed complex-BEMD uses four quadrant spectra to apply standard BEMD to four real-valued 2D signals. The so-generated intrinsic mode functions (IMFs) are 2D complex-valued, which facilitates the extension of the standard BEMD to the complex domain. The proposed complex-BEMD can be successful for the analysis of real-world 2D complex-valued signals, such as 2D NMR signals. Moreover, the proposed complex-BEMD can be applied for color image processing. A simple color image fusion algorithm based upon the proposed complex-BEMD has also been developed to have the exhibition of the potential. By our proposed complex-BEMD and image fusion algorithm, the well-fused results can be obtained, if the mode mixing in BEMD is alleviated.

Spatial Domain Method for Fusing MultiFocus Images using Measure of Fuzziness

simple and effective image fusion technique has been designed to convey all the meaningful information from the images of the same scene with diverse focuses into a final composite image. This paper presents an alternate approach to the problem of multi-focus image fusion using index of fuzziness. The fusion performance of the proposed method has been evaluated using an objective measure. Experimental results on several pairs of images indicate that the proposed method works well when compared to some of the image fusion methods.

Efficient fusion scheme for multi-focus images by using blurring measure

In this paper, a simple and efficient multi-focus image fusion approach is proposed. As for the multi-focus images, all of them are obtained from the same scene with different focuses. So the images can be segmented into two kinds regions based on out of and on the focus, which directly leads to a region based fusion, i.e., finding out all of the regions on the focus from the source images, and merging them into a combination image. This poses the question of how to locate the regions on the focus from the input images. Considering that the details or scales are different in the regions which are not and on the focuses, blurring measure method in this paper is used to locate the regions based on the blocking degree. This new fusion method can significantly reduce the amount of distortion artifacts and the loss of contrast information. These are usually observed in fused images in the conventional fusion schemes. The fusion performance of proposed method has been evaluated through informal visual inspection and objective fusion performance measurements, and results show the advantages of the approach compared to conventional fusion approaches.

Accuracy of Image Fusion Using a Fixation Device for Whole-Body Cancer Imaging

The purpose of this study was to evaluate the clinical feasibility of a simple image fusion technique with PET and CT images acquired separately using a vacuum cushion as a fixation device. Forty-four patients underwent whole-body PET using 18F-fluoro-2-deoxy-D-glucose (FDG) followed by CT with IV contrast material. The patients were carefully fixed in an individually molded cushion to provide the same positioning for both examinations. The PET and CT images were fused on a workstation by using the lower margin of the urinary bladder as a reference. The degree of misregistration was evaluated for the physiologic uptake of the liver and kidneys and for the pathologic uptake of lesions. The average deviation of the center point of the liver between the two images was 6.6 +/- 8.7 (SD) mm in the craniocaudal direction, 1.9 +/- 5.1 mm in the anteroposterior direction, and 2.3 +/- 7.0 mm in the right-left direction. This value in the craniocaudal direction was 4.7 +/- 8.7 mm in the right kidney and 4.0 +/- 8.8 mm in the left kidney. Above the diaphragm, the deviations of the center point of movable and static lesions were 11.7 +/- 3.4 mm and 10.4 +/- 5.3 mm, respectively. Below the diaphragm, those of movable and static lesions were 9.7 +/- 2.5 mm and 6.9 +/- 2.9 mm, respectively. Our preliminary data indicate that this technique is a simple and practical method for manual image fusion that may be acceptable in clinical settings.

Visualization of intracranial vessel anatomy using high resolution MRI and a simple image fusion technique

A new technique for fusion and 3D viewing of high resolution magnetic resonance (MR) angiography and morphological MR sequences is reported. Scanning and image fusion was possible within 20 min on a standard 1.5 T MR-scanner. The procedure was successfully performed in 10 consecutive cases with excellent visualization of wall and luminal aspects of the intracranial segments of the internal carotid artery, the vertebrobasilar system and the anterior, middle and posterior cerebral artery.

Integration of combined PET/CT images with conformal radiation treatment planning

The synergy between combined PET/CT imaging and conformal radiotherapy in the management of cancer patients is explored. The advantages of PET/CT imaging over simple image fusion and the benefits of incorporating additional information provided by functional imaging into radiotherapy planning usually performed solely on anatomical images are reviewed. Applications are identified in the abdomen, where there is an absence of identifiable structures on PET, and in the head and neck, where many small structures lie in very close proximity. Methods of integrating images obtained from a Siemens biograph LSO combined PET/CT scanner into several vendors’ radiation therapy planning systems were developed, and full connectivity was established with two cancer treatment centres. Phantom experiments were performed to test the transfer of the DICOM images, and to validate localization and volumetric accuracy. Modification in the planning target volumes (PTV) was demonstrated for a number of clinical case studies. In one recurrent lung cancer case the PTV were reduced by 74%. The methodological groundwork has been established for future clinical projects, which will assess the clinical impact of integrating PET/CT images into RTP systems.

Integrating human brain maps

Perception, action, cognition, and emotion can now be mapped in the brain by a growing family of techniques. Positron emission tomography, functional magnetic resonance imaging, event-related electrical potentials, event-related magnetic fields, and other non-invasive imaging techniques are rapidly evolving and providing an increasingly rich literature on the functional organization of the human brain. Although no two techniques map identical physiological processes or physical parameters, replications of functionally specific maps by different techniques indicate sufficient common ground for multimodality integration. The process of integration is multi-tiered. Recent advances in integration range from simple image fusion, to model-based synthetic analyses, to collective databases for neural-system modeling. Spatially, temporally, physiologically, and cognitively accurate computational models of the neural systems of human behavior are the ultimate objective of functional brain mapping. This objective will be reached only through integrating the diversity of modern brain-mapping methods.