Quality Of Fused Image Research Articles

RETRACTED: Multi-spectral and panchromatic image fusion approach using stationary wavelet transform and swarm flower pollination optimization for remote sensing applications

Abstract This paper proposes a multi-spectral (MS) and panchromatic (Pan) image fusion approach based on the flower pollination algorithm optimization (FPA). The FPA is used to get an optimal fused image. The image fusion quality depends on the choice of the weight of fusion rule. The proposed approach uses FPA to optimize the weights of a fusion rule to make a perfect image fusion process. FPA is a nature-inspired algorithm, based on the characteristics of a flower pollination process. FPA averts trapping in local optimal solution. In this paper, the remote sensing image fusion based on flower pollination algorithm is compared to several states of the art image fusion approaches including Intensity-hue-saturation (IHS) image fusion; stationary wavelets transform image fusion based on the average weight fusion rule (SWT-AW) and the image fusion based on the particle swarm optimization (PSO). The experimental results used MODIS satellite series with spatial resolutions 250 m, 500 m, and 1 km, which are low spatial resolution and multispectral images; and Pan image of SPOT satellite is high spatial resolution 10 m to produce synthetic imagery at SPOT spatial resolutions and MODIS multispectral resolution at the same time. The experimental results prove that the proposed remote sensing image fusion approach can illustrate a better performance than the other approaches. The experimental results show that the approach offers up to 20% enhancement in Peak Signal to Noise Ratio (PSNR), 1% enhancement in Structural Similarity Index (SSIM), 1% and 0.5% enhancement in entropy information (EI) than best existing particle swarm optimization (PSO) approach. The results indicate that the proposed approach outperforms over existing approaches.

Objective Evaluation Method of Fusion Image Quality Based on the Completed Local Binary Pattern

Objective Evaluation Method of Fusion Image Quality Based on the Completed Local Binary Pattern

Fusion of Infrared and Visible Images based on NSCT and Modified PCNN

For purpose of improving the quality of multi-source fusion images, this paper introduces a fusion method of infrared and visible images based on the combination of the non-subsampled contourlet transform (NSCT) and the improved pulse coupled neural network model (PCNN). To begin with, registered images are decomposed by NSCT, the low frequency and high frequency sub-bands are also obtained. Then, the energy of Laplacian (SML) is used for the link intensity of PCNN, computing spatial frequency (SF) with a low frequency sub-band, and the gradient energy (EOG) with a high frequency sub-band. By means of treating them as external input of PCNN. Then fused coefficients are selected by sum of ignition output amplitude maximum, a fused image is finally obtained by inverse NSCT. Experimental results show that the fusion image has a high definition and effectively preserves information of the source image.

An image fusion method based on content cognition

A cognition-based image fusion algorithm for multi-source images is put forward in this paper. The fusion process is guided by the abstract representation acqui red by the prior knowledge which implies human deep understandings for the targets and the scene. Different fusion rules are applied for contents of different importance, which make the image fusion more free and flexible. Image fusion tests for visible and infrared image are conducted. And we compare our fusion results with that of region-based fusion algorithms. Both experimental results and image fusion quality indexes show the superiorities of the fusion algorithm based on content cognition.

STANDARDIZING QUALITY ASSESSMENT OF FUSED REMOTELY SENSED IMAGES

Abstract. The multitude of available operational remote sensing satellites led to the development of many image fusion techniques to provide high spatial, spectral and temporal resolution images. The comparison of different techniques is necessary to obtain an optimized image for the different applications of remote sensing. There are two approaches in assessing image quality: 1. Quantitatively by visual interpretation and 2. Quantitatively using image quality indices. However an objective comparison is difficult due to the fact that a visual assessment is always subject and a quantitative assessment is done by different criteria. Depending on the criteria and indices the result varies. Therefore it is necessary to standardize both processes (qualitative and quantitative assessment) in order to allow an objective image fusion quality evaluation. Various studies have been conducted at the University of Osnabrueck (UOS) to establish a standardized process to objectively compare fused image quality. First established image fusion quality assessment protocols, i.e. Quality with No Reference (QNR) and Khan's protocol, were compared on varies fusion experiments. Second the process of visual quality assessment was structured and standardized with the aim to provide an evaluation protocol. This manuscript reports on the results of the comparison and provides recommendations for future research.

Multi-focus image fusion based on window empirical mode decomposition

In order to improve multi-focus image fusion quality, a novel fusion algorithm based on window empirical mode decomposition (WEMD) is proposed. This WEMD is an improved form of bidimensional empirical mode decomposition (BEMD), due to its decomposition process using the adding window principle, effectively resolving the signal concealment problem. We used WEMD for multi-focus image fusion, and formulated different fusion rules for bidimensional intrinsic mode function (BIMF) components and the residue component. For fusion of the BIMF components, the concept of the Sum-modified-Laplacian was used and a scheme based on the visual feature contrast adopted; when choosing the residue coefficients, a pixel value based on the local visibility was selected. We carried out four groups of multi-focus image fusion experiments and compared objective evaluation criteria with other three fusion methods. The experimental results show that the proposed fusion approach is effective and performs better at fusing multi-focus images than some traditional methods.

Medical Image Fusion: A Literature Review Present Solutions and Future Directions

In the current era of technological development, medical imaging plays an important role in many applications of medical diagnosis and therapy. This requires more accurate images with much more details and information for correct medical diagnosis and therapy. Medical image fusion is one of the solutions for obtaining both high spatial and high spectral information in a single image. Multimodal medical image fusion provides a remarkable improvement in the fused image quality. This review paper covers the following items, the basic definition of the image fusion process, applications, advantages, and disadvantages of fusion procedures, A short summery of radiological imaging modalities, usage, applications, and observed advantages and disadvantages of each modality is introduced to provide a comprehensive view of medical imaging modalities. Through this, a hybrid imaging combinations are suggested for better Image visualization for different human organs. The paper ends with some novel trends in the medical image fusion topic.

Satellite Images Fusion Using Modified PCA Substitution Method

In this paper, a new tunable approach for fusion the satellite images that fall in different electromagnetic wave ranges is presented, which gives us the ability to make one of the images features little superior on the other without reducing the general resultant image fusion quality, this approach is based on the principal component analysis (PCA) fusion method. A comparison made is between the results of the proposed approach and two fusion methods (they are: the PCA fusion method and the projection of eigenvectors on the bands fusion method), and the comparison results show the validity of this new method.

Predicting the Quality of Fused Long Wave Infrared and Visible Light Images.

The capability to automatically evaluate the quality of long wave infrared (LWIR) and visible light images has the potential to play an important role in determining and controlling the quality of a resulting fused LWIR-visible light image. Extensive work has been conducted on studying the statistics of natural LWIR and visible images. Nonetheless, there has been little work done on analyzing the statistics of fused LWIR and visible images and associated distortions. In this paper, we analyze five multi-resolution-based image fusion methods in regards to several common distortions, including blur, white noise, JPEG compression, and non-uniformity. We study the natural scene statistics of fused images and how they are affected by these kinds of distortions. Furthermore, we conducted a human study on the subjective quality of pristine and degraded fused LWIR-visible images. We used this new database to create an automatic opinion-distortion-unaware fused image quality model and analyzer algorithm. In the human study, 27 subjects evaluated 750 images over five sessions each. We also propose an opinion-aware fused image quality analyzer, whose relative predictions with respect to other state-of-the-art models correlate better with human perceptual evaluations than competing methods. An implementation of the proposed fused image quality measures can be found at https://github.com/ujemd/NSS-of-LWIR-and-Vissible-Images. Also, the new database can be found at http://bit.ly/2noZlbQ.

Efficient fusion of osseous and vascular details in wavelet domain

Image fusion is important medical image processing technique which aims at fusing complimentary information between two images. A little literature has been found on pre-processing of the source images prior to fusion which directly affects the image fusion quality. In this paper we have proposed an effective image fusion method based on IHS-wavelet transform along with pre-processing of source image in a selected sequence of spatial and transform domain techniques before fusion to create a highly informative fused image. The proposed methodology has outperformed six other state-of-the-art image fusion techniques both in terms of objective evaluation and visual results on osseous and vascular 2-D data

Saturation-based quality assessment for colorful multi-exposure image fusion

Multi-exposure image fusion is becoming increasingly influential in enhancing the quality of experience of consumer electronics. However, until now few works have been conducted on the performance evaluation of multi-exposure image fusion, especially colorful multi-exposure image fusion. Conventional quality assessment methods for multi-exposure image fusion mainly focus on grayscale information, while ignoring the color components, which also convey vital visual information. We propose an objective method for the quality assessment of colored multi-exposure image fusion based on image saturation, together with texture and structure similarities, which are able to measure the perceived color, texture, and structure information of fused images. The final image quality is predicted using an extreme learning machine with texture, structure, and saturation similarities as image features. Experimental results for a public multi-exposure image fusion database show that the proposed model can accurately predict colored multi-exposure image fusion image quality and correlates well with human perception. Compared with state-of-the-art image quality assessment models for image fusion, the proposed metric has better evaluation performance.

Evaluation of Effective Parameters on Quality of Magnetic Resonance Imaging-computed Tomography Image Fusion in Head and Neck Tumors for Application in Treatment Planning.

Background:In radiation therapy, computed tomography (CT) simulation is used for treatment planning to define the location of tumor. Magnetic resonance imaging (MRI)-CT image fusion leads to more efficient tumor contouring. This work tried to identify the practical issues for the combination of CT and MRI images in real clinical cases. The effect of various factors is evaluated on image fusion quality.Materials and Methods:In this study, the data of thirty patients with brain tumors were used for image fusion. The effect of several parameters on possibility and quality of image fusion was evaluated. These parameters include angles of the patient's head on the bed, slices thickness, slice gap, and height of the patient's head.Results:According to the results, the first dominating factor on quality of image fusion was the difference slice gap between CT and MRI images (cor = 0.86, P < 0.005) and second factor was the angle between CT and MRI slice in the sagittal plane (cor = 0.75, P < 0.005). In 20% of patients, this angle was more than 28° and image fusion was not efficient. In 17% of patients, difference slice gap in CT and MRI was >4 cm and image fusion quality was <25%.Conclusion:The most important problem in image fusion is that MRI images are taken without regard to their use in treatment planning. In general, parameters related to the patient position during MRI imaging should be chosen to be consistent with CT images of the patient in terms of location and angle.

KOMPSAT-2/3/3A호의 영상융합에 대한 품질평가 프로토콜의 비교분석

최근 영상융합 기법의 품질평가를 위하여 다양한 방법이 제안되었다. 품질평가를 위한 일련의 과정들을 통틀어 프로토콜이라 정의되었으며, Wald's 프로토콜, QNR 프로토콜, Khan's 프로토콜 등의 다양한 품질평가 프로토콜이 제시되었다. 본 논문에서는 KOMPSAT-2/3/3A 위성영상을 활용하여 제시된 세 가지 품질평가 프로토콜을 각기 다른 융합 기법에 적용하였을 때 나타나는 결과를 비교하고, KOMPSAT 위성영상에의 활용가능성을 알아보는 한편 각 프로토콜의 장단점을 분석하였다. 이 때 기존의 연구와 달리 융합 영상의 품질이 시각적으로 뚜렷하게 구분되는 융합 기법을 사용하여 육안 분석을 통한 정성적 분석을 진행함과 동시에 각 품질평가 프로토콜을 통한 정량적 분석을 수행하였다. 이를 통해 분광정보 보존의 측면과 공간 정보 보존의 측면에서 정성적 결과와 정량적 결과의 유사성을 파악하였다. 분석 결과, 각 프로토콜의 과정상 특징을 반영하는 결과를 나타내었다. Wald's 프로토콜의 경우 정성적/정량적 분석 결과가 동일하였으나, 프로토콜 수행 과정이 번거로우며 특히 영상의 공간해상도를 강제로 저하시킨 후 융합을 진행하여도 그 결과가 실제 공간해상도의 융합 결과와 동일하다는 전제를 가정하고 있다는 한계점이 존재하였다. QNR 프로토콜의 경우 원 영상의 융합 결과를 평가할 수 있다는 장점이 존재하였다. 그러나 분광정보 보존 분석 시 Wavelet 융합 결과의 aliasing 현상을 반영하지 못하는 등의 불안정성이 존재하여 프로토콜을 활용할 때 주의해야하는 단점이 있었다. Khan's 프로토콜의 경우 분광정보 보존 분석의 경우에 정성적/정량적 분석 결과가 Wavelet 융합 기법에서 일치하지 않았다. 이는 품질평가 시 공간해상도를 강제로 저하시킬 때 융합 영상의 품질이 변하여 발생하는 문제였다. 공간정보 보존 분석의 경우 Wald's 프로토콜의 한계점과 QNR 프로토콜의 단점을 모두 보완하여 품질평가를 수행하는 프로토콜이라 할 수 있음을 확인하였다. Many image fusion quality assessment techniques, which include Wald's, QNR and Khan's protocols, have been proposed. A total procedure for the quality assessment has been defined as the quality assessment protocol. In this paper, we compared the performance of the three protocols using KOMPSAT-2/3/3A satellite images, and tested the applicability to the fusion quality assessment of the KOMPSAT satellite images. In addition, we compared and analyzed the strengths and weaknesses of the three protocols. We carried out the qualitative and quantitative analysis of the protocols by applying five fusion methods to the KOMPSAT test images. Then we compared the quantitative and qualitative results of the protocols from the aspects of the spectral and spatial preservations. In the Wald's protocol, the results from the qualitative and quantitative analysis were almost matched. However, the Wald's protocol had the limitations 1) that it is timeconsuming due to downsampling process and 2) that the fusion quality assessment are performed by using downsampled fusion image. The QNR protocol had an advantage that it utilizes an original image without downsampling. However, it could not find the aliasing effect of the wavelet-fused images in the spectral preservation. It means that the spectral preservation assessment of the QNR protocol might not be perfect. In the Khan's protocol, the qualitative and quantitative analysis of the spectral preservation was not matched in the wavelet fusion. This is because the fusion results were changed in the downsampling process of the fused images. Nevertheless, the Khan's protocol were superior to Wald's and QNR protocols in the spatial preservation.

Ultrasound fusion image error correction using subject-specific liver motion model and automatic image registration

PurposeUltrasound fusion imaging is an emerging tool and benefits a variety of clinical applications, such as image-guided diagnosis and treatment of hepatocellular carcinoma and unresectable liver metastases. However, respiratory liver motion-induced misalignment of multimodal images (i.e., fusion error) compromises the effectiveness and practicability of this method. The purpose of this paper is to develop a subject-specific liver motion model and automatic registration-based method to correct the fusion error. MethodsAn online-built subject-specific motion model and automatic image registration method for 2D ultrasound-3D magnetic resonance (MR) images were combined to compensate for the respiratory liver motion. The key steps included: 1) Build a subject-specific liver motion model for current subject online and perform the initial registration of pre-acquired 3D MR and intra-operative ultrasound images; 2) During fusion imaging, compensate for liver motion first using the motion model, and then using an automatic registration method to further correct the respiratory fusion error. Evaluation experiments were conducted on liver phantom and five subjects. ResultsIn the phantom study, the fusion error (superior-inferior axis) was reduced from 13.90±2.38mm to 4.26±0.78mm by using the motion model only. The fusion error further decreased to 0.63±0.53mm by using the registration method. The registration method also decreased the rotation error from 7.06±0.21° to 1.18±0.66°. In the clinical study, the fusion error was reduced from 12.90±9.58mm to 6.12±2.90mm by using the motion model alone. Moreover, the fusion error decreased to 1.96±0.33mm by using the registration method. ConclusionsThe proposed method can effectively correct the respiration-induced fusion error to improve the fusion image quality. This method can also reduce the error correction dependency on the initial registration of ultrasound and MR images. Overall, the proposed method can improve the clinical practicability of ultrasound fusion imaging.

Patient movement compensation for 3D echocardiography fusion.

Limited field of view (FOV) is a major problem for 3D real-time echocardiography (3DRTE), which results in an incomplete representation of cardiac anatomy. Various image registration techniques have been proposed to improve the field of view in 3DRTE by fusing multiple image volumes. However, these techniques require significant overlap between the individual volumes and rely on high image resolution and high signal-to-noise ratio. Changes in the heart position due to patient movement during image acquisition can also reduce the quality of image fusion. In this paper, we propose a multi-camera based optical tracking system which 1) eliminates the need for image overlap and 2) compensates for patient movement during acquisition. We compensate for patient movement by continuously tracking the patient position using skin markers and incorporating this information into the fusion process. We fuse volumes acquired during R-R wave peaks based on Electrocardiogram (ECG) data to account for retrospective image acquisition. The fusion technique was validated using a heart phantom (Shelley Medical Imaging Technologies) and on one healthy volunteer. The fused ultrasound volumes could be generated in within 2 seconds and were found to have complete myocardial boundaries alignment upon visual assessment. No stitching artefacts or movement related artefacts were observed in the fused image.

A novel dictionary learning approach for multi-modality medical image fusion

Multi-modality medical image fusion technology can integrate the complementary information of different modality medical images, obtain more precise, reliable and better description of lesions. Dictionary learning based image fusion draws a great attention in researchers and scientists, for its high performance. The standard learning scheme uses entire image for dictionary learning. However, in medical images, the informative region takes a small proportion of the whole image. Most of the image patches have limited and redundant information. Taking all the image patches for dictionary learning brings lots of unvalued and redundant information, which can influence the medical image fusion quality. In this paper, a novel dictionary learning approach is proposed for image fusion. The proposed approach consists of three steps. Firstly, a novel image patches sampling scheme is proposed to obtain the informative patches. Secondly, a local density peaks based clustering algorithm is conducted to classify the image patches with similar image structure information into several patch groups. Each patch group is trained to a compact sub-dictionary by K-SVD. Finally the sub-dictionaries are combined to a complete, informative and compact dictionary. In this dictionary,only important and useful information which can effectively describe the medical image are selected. To show the efficiency of the proposed dictionary learning approach, the sparse coefficient vectors are estimated by a simultaneous orthogonal matching pursuit (SOMP) algorithm with the trained dictionary, and fused by max-L1 rules. The comparative experimental results and analyses reveal that the proposed method achieves better image fusion quality than existing state-of-the-art methods.

Quality assessment of remote sensing image fusion using feature-based fourth-order correlation coefficient

In remote sensing fusion, the spatial details of a panchromatic (PAN) image and the spectrum information of multispectral (MS) images will be transferred into fused images according to the characteristics of the human visual system. Thus, a remote sensing image fusion quality assessment called feature-based fourth-order correlation coefficient (FFOCC) is proposed. FFOCC is based on the feature-based coefficient concept. Spatial features related to spatial details of the PAN image and spectral features related to the spectrum information of MS images are first extracted from the fused image. Then, the fourth-order correlation coefficient between the spatial and spectral features is calculated and treated as the assessment result. FFOCC was then compared with existing widely used indices, such as Erreur Relative Globale Adimensionnelle de Synthese, and quality assessed with no reference. Results of the fusion and distortion experiments indicate that the FFOCC is consistent with subjective evaluation. FFOCC significantly outperforms the other indices in evaluating fusion images that are produced by different fusion methods and that are distorted in spatial and spectral features by blurring, adding noise, and changing intensity. All the findings indicate that the proposed method is an objective and effective quality assessment for remote sensing image fusion.

Fast and Efficient Satellite Imagery Fusion Using DT-CWT Proportional and Wavelet Zero-Padding

Among the various image fusion or pan-sharpening methods, those wavelet-based methods provide superior radiometric quality. However, the fusion processing is not only simple but also flexible, since many low- and high-frequency sub-bands are often produced in the wavelet domain. To address this issue, a novel DT-CWT (Dual-Tree Complex Wavelet Transform) proportional to the fusion method by a WZP (Wavelet Zero-Padding) is proposed. The proposed method produces a single high-frequency image in the spatial domain that is injected into the LRM (Low-Resolution Multispectral) image. Thus, a wavelet domain fusion can be simplified to spatial domain fusion. In addition, in the proposed DT-CWTP (DT-CWT Proportional) fusion method, it is unnecessary to decompose the LRM image by adopting WZP. The comparison indicates that the proposed fusion method is nearly five times faster than the DT-CWT with SW (Substitute-Wavelet) fusion method, meanwhile simultaneously maintaining the radiometric quality. The conducted experiments with WorldView-2 satellite images demonstrated promising results with the computation efficiency and fused image quality.

Medical Image Fusion based on Shearlets and Human Feature Visibility

image fusion is a technique that integrates complementary information from multimodality images. The fused image is more suitable for treatment plan strategies. In this paper, an efficient medical image fusion method has been proposed based on shearlet transform and human visibility feature as fusion rule. Image fusion rule is the solution that influences the quality of image fusion. The multimodal medical images were first decomposed using the shearlet transform then fusion rules were applied to shearlet coefficients. The low-frequency coefficients are fused by human visibility feature method. While, the high frequency coefficients are fused by the maximum selection fusion rule. The final fusion image is obtained by directly applying inverse shearlet transform to the fused coefficients. The technique proposed has successfully been used in CT/MRI image fusion for tumor diagnosis. The visual experiments and quantitative assessments demonstrate the effectiveness of this method compared to present image fusion. Keywordshearlet transform; medical image fusion; human visibility feature; multimodality; CT/MRI image.

Gradient entropy metric and p-Laplace diffusion constraint-based algorithm for noisy multispectral image fusion

Noise is easily mistaken as useful features of input images, and therefore, significantly reducing image fusion quality. In this paper, we propose a novel gradient entropy metric and p-Laplace diffusion constraint-based method. Specifically, the method is based on the matrix of structure tensor to fuse the gradient information. To minimize the negative effects of noise on the selections of image features, the gradient entropy metric is proposed to construct the weight for each gradient of input images. Particularly, the local adaptive p-Laplace diffusion constraint is constructed to further suppress noise when rebuilding the fused image from the fused gradient field. Experimental results show that the proposed method effectively preserves edge detail features of multispectral images while suppressing noise, achieving an optimal visual effect and more comprehensive quantitative assessments compared to other existing methods.