Image Fusion Process Research Articles

Medical Image Fusion: A Literature Review Present Solutions and Future Directions

In the current era of technological development, medical imaging plays an important role in many applications of medical diagnosis and therapy. This requires more accurate images with much more details and information for correct medical diagnosis and therapy. Medical image fusion is one of the solutions for obtaining both high spatial and high spectral information in a single image. Multimodal medical image fusion provides a remarkable improvement in the fused image quality. This review paper covers the following items, the basic definition of the image fusion process, applications, advantages, and disadvantages of fusion procedures, A short summery of radiological imaging modalities, usage, applications, and observed advantages and disadvantages of each modality is introduced to provide a comprehensive view of medical imaging modalities. Through this, a hybrid imaging combinations are suggested for better Image visualization for different human organs. The paper ends with some novel trends in the medical image fusion topic.

Multimodal Imaging for Clinical Target Volume Definition in Prone Whole-Breast Irradiation: A Single Institution Experience

Aim: The aim was identification of reference structures for breast clinical target volume (CTV) in prone position, throughout image fusion process. Materials & methods: We analyzed breast glandular tissue distribution in 20 diagnostic MRIs, referring to structures reported in ESTRO guidelines for supine irradiation. The volume containing breast glandular tissue in all cases was defined as MRI prone CTV (MRIpCTV). Then in ten subsequent patients planned for prone irradiation, MRI and computed tomography (CT) simulation was acquired. MRIpCTV was defined followed by our findings and transferred to CT for definitive delineation. Results: MRIpCTV was defined by the caudal edge of clavicular head, 3 mm above inframammary fold, by the medial thoracic artery, by a plane passing through the lateral surface of pectoralis muscles, by the anterior surface of pectoralis muscles and 3 mm from the skin. Deformed CTV was consistent with anatomy on CT; the limits chosen for MRIpCTV fit adequately also for CT. Conclusion: Prone irradiation is an alternative set up for selected cases, so the sample is very small. However, our suggestions could be of aid in defining prone CTV. The good consistency between MRI and CT seems to confirm that MRI may be unnecessary in routine practice.

Compressive sensing image fusion in heterogeneous sensor networks based on shearlet and wavelet transform

Heterogeneous image fusion is a technique of fusing images captured by different sensors into one image, then the fused image will present more information than the original images. This paper studies the compressive sensing image fusion algorithm and applies shearlet and wavelet transforms to represent the image sparsely. By compressing the sampled coefficients of the original images, the computational complexity in the image fusion process is reduced and the fusion efficiency is improved. We focus on the image fusion rules of compressive domain. Image coefficients of different frequencies are compressed by various sampling rates and fused according to different fusion rules. So an ideal fusion results can be obtained under a low sampling rate.

Percutaneous Thermal Ablation with Ultrasound Guidance. Fusion Imaging Guidance to Improve Conspicuity of Liver Metastasis.

To assess whether fusion imaging-guided percutaneous microwave ablation (MWA) can improve visibility and targeting of liver metastasis that were deemed inconspicuous on ultrasound (US). MWA of liver metastasis not judged conspicuous enough on US was performed under CT/US fusion imaging guidance. The conspicuity before and after the fusion imaging was graded on a five-point scale, and significance was assessed by Wilcoxon test. Technical success, procedure time, and procedure-related complications were evaluated. A total of 35 patients with 40 liver metastases (mean size 1.3±0.4cm) were enrolled. Image fusion improved conspicuity sufficiently to allow fusion-targeted MWA in 33 patients. The time required for image fusion processing and tumors' identification averaged 10±2.1min (range 5-14). Initial conspicuity on US by inclusion criteria was 1.2±0.4 (range 0-2), while conspicuity after localization on fusion imaging was 3.5±1 (range 1-5, p<0.001). Technical success rate was 83% (33/40) in intention-to-treat analysis and 100% in analysis of treated tumors. There were no major procedure-related complications. Fusion imaging broadens the scope of US-guided MWA to metastasis lacking adequate conspicuity on conventional US. Fusion imaging is an effective tool to increase the conspicuity of liver metastases that were initially deemed non visualizable on conventional US imaging.

Intensity–hue–saturation-based image fusion using iterative linear regression

The image fusion process basically produces a high-resolution image by combining the superior features of a low-resolution spatial image and a high-resolution panchromatic image. Despite its common usage due to its fast computing capability and high sharpening ability, the intensity–hue–saturation (IHS) fusion method may cause some color distortions, especially when a large number of gray value differences exist among the images to be combined. This paper proposes a spatially adaptive IHS (SA-IHS) technique to avoid these distortions by automatically adjusting the exact spatial information to be injected into the multispectral image during the fusion process. The SA-IHS method essentially suppresses the effects of those pixels that cause the spectral distortions by assigning weaker weights to them and avoiding a large number of redundancies on the fused image. The experimental database consists of IKONOS images, and the experimental results both visually and statistically prove the enhancement of the proposed algorithm when compared with the several other IHS-like methods such as IHS, generalized IHS, fast IHS, and generalized adaptive IHS.

Image Fusion on Coloured and Gray Scale Multi Focus Images by using Hybrid DWT-DCT

Any piece of information is meaningful only when it is able to convey the content about it. The clarity and quality of information is important. Image Fusion is a technique to improve the quality and quantity of information from a set of images. By the process of image fusion the more information from each of the given images is combining together to generate a resultant image whose quality is maximum to any of the input images. We proposed a Hybrid DWT-DCT method to fuse multi focus images. In this technique we convert the image data from spatial domain to transform domain.. Then decompose the transform data into four parts that is LL, LH, HL, HH part. This decomposition Process again applies in LL part at two levels. After this three level decomposition we combine the input image data by applying average method using DCT and get fused data. This data are in transform domain again convert to spatial domain by applying IDCT and IDWT method and get final fused image with better visual Quality. After getting the result of fused image we compare the quality measure parameters of different technique like PCA, DCT, average pixel, maximum pixel, minimum pixel, HDWT method to hybrid DWT-DCT method. And conclude that the PSNR value and Entropy of fused image have better result as compare to other techniques. Due to this the fused image has better visual Quality as well as more informative data would contain in fused image.

A Novel Fusion Rule for Medical Image Fusion in Complex Wavelet Transform Domain

Medical image fusion is being used at large by clinical professionals for improved diagnosis and treatment of diseases. The main aim of image fusion process is to combine complete information from all input images into a single fused image. Therefore, a novel fusion rule is proposed for fusing medical images based on Daubechies complex wavelet transform (DCxWT). Input images are first decomposed using DCxWT. The complex coefficients so obtained are then fused using normalized correlation based fusion rule. Finally, the fused image is obtained by inverse DCxWT with all combined complex coefficients. The performance of the proposed method has been evaluated and compared both visually and objectively with DCxWT based fusion methods using state-of art fusion rules as well as with existing fusion techniques. Experimental results and comparative study demonstrate that the proposed fusion technique generates better results than existing fusion rules as well as with other fusion techniques.

ADVANCED EXTRACTION OF SPATIAL INFORMATION FROM HIGH RESOLUTION SATELLITE DATA

Abstract. In this paper authors processed five satellite image of five different Middle-European cities taken by five different sensors. The aim of the paper was to find methods and approaches leading to evaluation and spatial data extraction from areas of interest. For this reason, data were firstly pre-processed using image fusion, mosaicking and segmentation processes. Results going into the next step were two polygon layers; first one representing single objects and the second one representing city blocks. In the second step, polygon layers were classified and exported into Esri shapefile format. Classification was partly hierarchical expert based and partly based on the tool SEaTH used for separability distinction and thresholding. Final results along with visual previews were attached to the original thesis. Results are evaluated visually and statistically in the last part of the paper. In the discussion author described difficulties of working with data of large size, taken by different sensors and different also thematically.

ADVANCED EXTRACTION OF SPATIAL INFORMATION FROM HIGH RESOLUTION SATELLITE DATA

In this paper authors processed five satellite image of five different Middle-European cities taken by five different sensors. The aim of the paper was to find methods and approaches leading to evaluation and spatial data extraction from areas of interest. For this reason, data were firstly pre-processed using image fusion, mosaicking and segmentation processes. Results going into the next step were two polygon layers; first one representing single objects and the second one representing city blocks. In the second step, polygon layers were classified and exported into Esri shapefile format. Classification was partly hierarchical expert based and partly based on the tool SEaTH used for separability distinction and thresholding. Final results along with visual previews were attached to the original thesis. Results are evaluated visually and statistically in the last part of the paper. In the discussion author described difficulties of working with data of large size, taken by different sensors and different also thematically.

An Improved Image Fusion and Segmentation using FLICM with GA for Medical Diagonosis

Background: Medical image fusion has become a common term used in medical diagnostics and treatment. Now a day's numerous wavelet transform based fusion methods proposed for overcoming the spectral distortion in the conventional fusion techniques such as PCA and IHS. DWT based image fusion process has the spatial distortion and lack of information. So selecting suitable wavelet transform for fusion is an important task. Method: In the proposed system, Dual tree Discrete Wavelet transform based Fusion and GA based FLICM segmentation is proposed. First, two different types of medical images-MRI and CT are decomposed by Dual tree Discrete Wavelet Transform (DDWT). Next, DDWT coefficients from two input images are fused pixel-by pixel by choosing the average of the approximation coefficients. Findings: Genetic algorithm is applied for the fused image in order to generate the optimal cluster centers of its own. From the clusters the affected portion cells are identified using FLICM segmentation process. Improvements: The performance of the proposed method is tested for various medical images and the results are analyzed. The fusion results shows DDWT based fusion technique provides around 5db improved PSNR performance when compared to the DWT based fusion. The GA optimizer based FLICM segmentation technique reduces the computation time. For various medical images the segmentation time is approximately reduced as half when compared to the FLICM segmentation technique without GA.

Image fusion via nonlocal sparse K-SVD dictionary learning.

Image fusion aims to merge two or more images captured via various sensors of the same scene to construct a more informative image by integrating their details. Generally, such integration is achieved through the manipulation of the representations of the images concerned. Sparse representation plays an important role in the effective description of images, offering a great potential in a variety of image processing tasks, including image fusion. Supported by sparse representation, in this paper, an approach for image fusion by the use of a novel dictionary learning scheme is proposed. The nonlocal self-similarity property of the images is exploited, not only at the stage of learning the underlying description dictionary but during the process of image fusion. In particular, the property of nonlocal self-similarity is combined with the traditional sparse dictionary. This results in an improved learned dictionary, hereafter referred to as the nonlocal sparse K-SVD dictionary (where K-SVD stands for the K times singular value decomposition that is commonly used in the literature), and abbreviated to NL_SK_SVD. The performance of the NL_SK_SVD dictionary is applied for image fusion using simultaneous orthogonal matching pursuit. The proposed approach is evaluated with different types of images, and compared with a number of alternative image fusion techniques. The resultant superior fused images using the present approach demonstrates the efficacy of the NL_SK_SVD dictionary in sparse image representation.

Multifocus watermarking approach based on discrete cosine transform.

Image fusion process consolidates data and information from various images of same sight into a solitary image. Each of the source images might speak to a fractional perspective of the scene, and contains both "pertinent" and "immaterial" information. In this study, a new image fusion method is proposed utilizing the Discrete Cosine Transform (DCT) to join the source image into a solitary minimized image containing more exact depiction of the sight than any of the individual source images. In addition, the fused image comes out with most ideal quality image without bending appearance or loss of data. DCT algorithm is considered efficient in image fusion. The proposed scheme is performed in five steps: (1) RGB colour image (input image) is split into three channels R, G, and B for source images. (2) DCT algorithm is applied to each channel (R, G, and B). (3) The variance values are computed for the corresponding 8 × 8 blocks of each channel. (4) Each block of R of source images is compared with each other based on the variance value and then the block with maximum variance value is selected to be the block in the new image. This process is repeated for all channels of source images. (5) Inverse discrete cosine transform is applied on each fused channel to convert coefficient values to pixel values, and then combined all the channels to generate the fused image. The proposed technique can potentially solve the problem of unwanted side effects such as blurring or blocking artifacts by reducing the quality of the subsequent image in image fusion process. The proposed approach is evaluated using three measurement units: the average of Q(abf), standard deviation, and peak Signal Noise Rate. The experimental results of this proposed technique have shown good results as compared with older techniques.

Hybrid dual-tree complex wavelet transform and support vector machine for digital multi-focus image fusion

This study proposed a new method for multi-focus image fusion using hybrid wavelet and classifier. The image fusion process was formulated as a two-class classification problem: in and out-of-focus classes. First, a six-dimensional feature vector was extracted using sub-bands of dual-tree complex wavelet transform (DT-CWT) coefficients from the source images, which were then projected by a trained two-class support vector machine (SVM) to the class labels. A bacterial foraging optimization algorithm (BFOA) was developed to obtain the optimal parameters of the SVM. The output of the classification system was used as a decision matrix for fusing high-frequency wavelet coefficients from multi-focus source images in different directions and decomposition levels of the DT-CWT. After the high and low-frequency coefficients of the source images were fused, the final fused image was obtained using the inverse DT-CWT. Several existing methods were compared with the proposed method. Experimental results showed that our presented method outperformed the existing methods, in visual effect and in objective evaluation.

Image Fusion-Based Change Detection for Flood Extent Extraction Using Bi-Temporal Very High-Resolution Satellite Images

Change detection based on satellite images acquired from an area at different dates is of widespread interest, according to the increasing number of flood-related disasters. The images help to generate products that support emergency response and flood management at a global scale. In this paper, a novel unsupervised change detection approach based on image fusion is introduced. The approach aims to extract the reliable flood extent from very high-resolution (VHR) bi-temporal images. The method takes an advantage of the spectral distortion that occurs during image fusion process to detect the change areas by flood. To this end, a change candidate image is extracted from the fused image generated with bi-temporal images by considering a local spectral distortion. This can be done by employing a universal image quality index (UIQI), which is a measure for local evaluation of spectral distortion. The decision threshold for the determination of changed pixels is set by applying a probability mixture model to the change candidate image based on expectation maximization (EM) algorithm. We used bi-temporal KOMPSAT-2 satellite images to detect the flooded area in the city of N′djamena in Chad. The performance of the proposed method was visually and quantitatively compared with existing change detection methods. The results showed that the proposed method achieved an overall accuracy (OA = 75.04) close to that of the support vector machine (SVM)-based supervised change detection method. Moreover, the proposed method showed a better performance in differentiating the flooded area and the permanent water body compared to the existing change detection methods.

Multifocus noisy image fusion using contourlet transform

Image fusion is a challenging area of research with a variety of applications. The process of image fusion collects information from different sources and combines them in a single composite image. The composite fused image can better describe the scene than any of the source images. In this paper, we have proposed a method for noisy image fusion in contourlet domain. The proposed method works equally well for fusion of noise free images. Contourlet transform is a multiscale, multidirectional transform with various aspect ratios. These properties make it more suitable for image fusion than other conventional transforms. In the proposed work, the fusion algorithm is combined with a denoising algorithm to reverse the effect of noise. In the proposed method, we have used a level dependent threshold that is based on standard deviation of contourlet coefficients, mean and median of the absolute contourlet coefficients. Experimental results demonstrate that the proposed method performs well in the presence of different types of noise. Performance of the proposed method is compared with principal components analysis and sharp fusion based methods as well as other fusion methods based on variants of wavelet transform like dual tree complex wavelet transform, discrete wavelet transform, lifting wavelet transform, multiwavelet transform, stationary wavelet transform and pyramid transform using six standard quantitative quality metrics (entropy, standard deviation, edge strength, fusion factor, sharpness and peak signal to noise ratio). The combined qualitative and quantitative evaluation of the experimental results shows that the proposed method performs better than other methods.

Effectiveness of Using the TIR Band in Landsat 8 Image Classification

This paper discusses the effectiveness of using Landsat 8 TIR (Thermal Infrared) band images to improve the accuracy of landuse/landcover classification of urban areas. According to classification results for the study area using diverse band combinations, the classification accuracy using an image fusion process in which the TIR band is added to the visible and near infrared band was improved by 4.0%, compared to that using a band combination that does not consider the TIR band. For urban area landuse/landcover classification in particular, the producer’s accuracy and user’s accuracy values were improved by 10.2% and 3.8%, respectively. When MLC (Maximum Likelihood Classification), which is commonly applied to remote sensing images, was used, the TIR band images helped obtain a higher discriminant analysis in landuse/landcover classification.

Multi-level image fusion and enhancement for target detection

In this paper, a novel infrared-to-visible image fusion algorithm for enhancing contrast and visibility is proposed. A multi-level method based on the characteristics of images and the properties of the targets is designed to complete the image fusion process, where a contrast enhancement method is added in the low-frequency information of the layered images and the edge information is enhanced in the high-frequency information using the correlation between the low- and high-frequency components. In the experiments, three groups of infrared-to-visible images were used to demonstrate the effectiveness of the multi-level fusion method. All the evaluation indexes, such as standard deviation and information entropy, were significantly higher than other existing methods. Thus, the experimental results verified the effectiveness of the proposed image fusion methods. The quality of the fusion images was improved for better differentiability in terms of contrast and features of the targets.

Study on Multi-objective Optimization Problem of Multi-source Image Fusion

Multi-source image fusion will integrate multiple images derived from the same scene or target collected into a new image to obtain more accurate and more complete description about the scene or target. The multi-objective optimi- zation problem of multi-source image fusion is researched in transform domain. Based on the analysis of multi-objective optimization theory and algorithms, an adaptive differential evolution algorithm is proposed. With adaptive variance fac- tor, dynamical crossover probability function and optimal elite ordering strategy, the algorithm reflects not only good search capability but also good convergence. When applied to multi-objective optimization of multi-source image fusion of transform domain, it will be an effective solution to the comprehensive evaluation in the image fusion process.

LEARNING NONRIGID DEFORMATIONS FOR CONSTRAINED POINT-BASED REGISTRATION FOR IMAGE-GUIDED MR-TRUS PROSTATE INTERVENTION.

This paper presents and validates a low-dimensional nonrigid registration method for fusing magnetic resonance imaging (MRI) and trans-rectal ultrasound (TRUS) in image-guided prostate biopsy. Prostate cancer is one of the most prevalent forms of cancer and the second leading cause of cancer-related death in men in the United States. Conventional clinical practice uses TRUS to guide prostate biopsies when there is a suspicion of cancer. Pre-procedural MRI information can reveal lesions and may be fused with intra-procedure TRUS imaging to provide patient-specific, localization of lesions for targeting. The state-of-the-art MRI-TRUS nonrigid image fusion process relies upon semi-automated segmentation of the prostate in both the MRI and TRUS images. In this paper, we develop a fast, automated nonrigid registration approach to MRI-TRUS fusion based on a statistical deformation model of intra-procedural deformations derived from a clinical sample.

Multi-Focus Raw Bayer Pattern Image Fusion for Single-Chip Camera

In this paper, an efficient patch-based image fusion approach for raw images of single-chip imaging devices incorporated with the Bayer CFA pattern is presented. The multi-source raw Bayer pattern images are firstly parted into half overlapped patches. Then, the patches with maximum clarity measurement defined for raw Bayer pattern image are selected as the fused patches. Next, all the fused local patches are merged with weighted average method in order to reduce the blockness effect of fused raw Bayer pattern image. Finally, the real color fused image is obtained by gradient based demosaicing technology. The multi-source raw Bayer pattern data is fused before demosaicing, so that the multi-sensor system will be more efficient and the artifacts introduced in demosaicing processing do not accumulate in image fusion processing. For comparison, the raw images are also interpolated firstly, and then various image fusion methods are used to get the fused color images. Experimental results show that the proposed algorithm is valid and very effective.