Quality Of Fused Image Research Articles

Attention Mechanism Based Semi-Supervised Multi-Gain Image Fusion

High-dynamic range imaging technology is an effective method to improve the limitations of a camera’s dynamic range. However, most current high-dynamic imaging technologies are based on image fusion of multiple frames with different exposure levels. Such methods are prone to various phenomena, for example motion artifacts, detail loss and edge effects. In this paper, we combine a dual-channel camera that can output two different gain images simultaneously, a semi-supervised network structure based on an attention mechanism to fuse multiple gain images is proposed. The proposed network structure comprises encoding, fusion and decoding modules. First, the U-Net structure is employed in the encoding module to extract important detailed information in the source image to the maximum extent. Simultaneously, the SENet attention mechanism is employed in the encoding module to assign different weights to different feature channels and emphasis important features. Then, a feature map extracted from the encoding module is input to the decoding module for reconstruction after fusing by the fusion module to obtain a fused image. Experimental results indicate that the fused images obtained by the proposed method demonstrate clear details and high contrast. Compared with other methods, the proposed method improves fused image quality relative to several indicators.

A Fast Image Fusion With Discrete Cosine Transform

A fast and effective image fusion method based on discrete cosine transform (DCT) is proposed. The fusion quality and computation time of current DCT-based image fusion methods largely depend on the selected block size, and the selection of the block size is very difficult in practice. A novel image fusion method based on DCT coefficient matrix features is proposed, which can effectively overcome the above difficulties. A DCT-based image fusion framework is proposed, which decomposes each source image into a base layer and a detail layer for image fusion. And optimize the calculation method of the base layer to better preserve the structure of the image. The effectiveness of the proposed method is verified by six image databases with more than 90 pairs of source images in total. The experimental results show that the proposed method can obtain the most effective results in terms of visual quality and objective evaluation of medical image fusion, and the fusion time is more efficient.

Medical image fusion based on laws of texture energy measures in stationary wavelet transform domain

AbstractMedical image fusion is widely used in various clinical procedures for the precise diagnosis of a disease. Image fusion procedures are used to assist real‐time image‐guided surgery. These procedures demand more accuracy and less computational complexity in modern diagnostics. Through the present work, we proposed a novel image fusion method based on stationary wavelet transform (SWT) and texture energy measures (TEMs) to address poor contrast and high‐computational complexity issues of fusion outcomes. SWT extracts approximate and detail information of source images. TEMs have the capability to capture various features of the image. These are considered for fusion of approximate information. In addition, the morphological operations are used to refine the fusion process. Datasets consisting of images of seven patients suffering from neurological disorders are used in this study. Quantitative comparison of fusion results with visual information fidelity‐based image fusion quality metric, ratio of spatial frequency error, edge information‐based image fusion quality metric, and structural similarity index‐based image fusion quality metrics proved the superiority. Also, the proposed method is superior in terms of average execution time to state‐of‐the‐art image fusion methods. The proposed work can be extended for fusion of other imaging modalities like fusion of functional image with an anatomical image. Suitability of the fused images by the proposed method for image analysis tasks needs to be studied.

Theoretical analysis of Tsallis entropy-based quality measure for weighted averaging image fusion

Entropy-based metrics are often employed for objective image fusion quality assessment due to single layer implementation of entropy and a small parameter set. In this paper, we present the theoretical analysis of image fusion quality measure based on Tsallis entropy. The purpose of this study is to theoretically assess if the considered quality measure is able to fulfill the desired behaviors that are expected from an ideal information-based image fusion quality metric. To assess the Tsallis quality measure, the paper employs an image formation model to obtain a closed-form expression for quality while weighted averaging is used as a fusion algorithm. The paper demonstrates that the Tsallis-based quality measure violates the desired behaviors regarding the response to variation of signal-to-noise ratio and effect of entropy order on the measured quality sign. Investigations on real images are also performed and the results agree with the theoretical analysis.

Human Visual Perception-Based Multi-Exposure Fusion Image Quality Assessment

Compared with ordinary single exposure images, multi-exposure fusion (MEF) images are prone to color imbalance, detail information loss and abnormal exposure in the process of combining multiple images with different exposure levels. In this paper, we proposed a human visual perception-based multi-exposure fusion image quality assessment method by considering the related perceptual features (i.e., color, dense scale invariant feature transform (DSIFT) and exposure) to measure the quality degradation accurately, which is closely related to the symmetry principle in human eyes. Firstly, the L1 norm of chrominance components between fused images and the designed pseudo images with the most severe color attenuation is calculated to measure the global color degradation, and the color saturation similarity is added to eliminate the influence of color over-saturation. Secondly, a set of distorted images under different exposure levels with strong edge information of fused image is constructed through the structural transfer, thus DSIFT similarity and DSIFT saturation are computed to measure the local detail loss and enhancement, respectively. Thirdly, Gauss exposure function is used to detect the over-exposure or under-exposure areas, and the above perceptual features are aggregated with random forest to predict the final quality of fused image. Experimental results on a public MEF subjective assessment database show the superiority of the proposed method with the state-of-the-art image quality assessment models.

Perceptual Evaluation for Multi-Exposure Image Fusion of Dynamic Scenes.

A common approach to high dynamic range (HDR) imaging is to capture multiple images of different exposures followed by multi-exposure image fusion (MEF) in either radiance or intensity domain. A predominant problem of this approach is the introduction of the ghosting artifacts in dynamic scenes with camera and object motion. While many MEF methods (often referred to as deghosting algorithms) have been proposed for reduced ghosting artifacts and improved visual quality, little work has been dedicated to perceptual evaluation of their deghosting results. Here we first construct a database that contains 20 multiexposure sequences of dynamic scenes and their corresponding fused images by nine MEF algorithms. We then carry out a subjective experiment to evaluate fused image quality, and find that none of existing objective quality models for MEF provides accurate quality predictions. Motivated by this, we develop an objective quality model for MEF of dynamic scenes. Specifically, we divide the test image into static and dynamic regions, measure structural similarity between the image and the corresponding sequence in the two regions separately, and combine quality measurements of the two regions into an overall quality score. Experimental results show that the proposed method significantly outperforms the state-of-the-art. In addition, we demonstrate the promise of the proposed model in parameter tuning of MEF methods.1.

Medical image fusion based on saliency and adaptive similarity judgment

Information regarding single-modality medical images remains limited at present. Therefore, the details of relevant organizations cannot be reflected by such information and misdiagnosis may result in the clinical setting. A novel image fusion method based on nonsubsampled contourlet transform (NSCT) and visual saliency is presented for multimode medical images to solve the aforementioned problem and improve the effect of fused images. In this method, an image is initially decomposed via NSCT to obtain low- and high-frequency coefficients and derive the Achanta (AC) saliency map of the image. Subsequently, the similarity of low- and high-frequency subbands is calculated using the standard deviation and energy of a region. Similarity is determined through an adaptive threshold. Meanwhile, the saliency map is utilized to guide the fusion of high- and low-frequency coefficients, with emphasis on the visual salient target area of the source image. Lastly, inverse NSCT is used to reconstruct the fused image. The proposed algorithm can effectively improve the quality of multimodal medical image fusion and increase complementary information among different modalities as demonstrated by the experimental results. This algorithm is superior to existing medical image fusion algorithms and more in accord with human visual characters.

An optimal wavelet-based multi-modality medical image fusion approach based on modified central force optimization and histogram matching

This paper introduces an optimal solution for wavelet-based medical image fusion using different wavelet families and Principal Component Ana1ysis (PCA) based on the Modified Central Force Optimization (MCFO) technique. The main motivation of this work is to increase the quality of medical fused images in order to provide correct diagnosis of diseases for the objective of optimal therapy. This can be achieved by fusing medical images of different modalities using an optimization technique based on the MCFO. The MCFO technique gives the optimum gain parameters that achieve the best fused image quality. Histogram matching is applied to improve the overall values of the Peak Signal-to-Noise Ratio (PSNR), entropy, local contrast, and quality of the fused image. A comparative study is performed between the proposed algorithm, the traditional Discrete Wavelet Transform (DWT), and the PCA fusion using maximum fusion rule. The proposed algorithm is evaluated subjectively and objectively with different fusion quality metrics. Simulation results demonstrate that the proposed MCFO optimized wavelet-based fusion algorithm using Haar wavelet and histogram matching achieves a superior performance with the highest image quality and clearest image details in a very short processing time.

Perceptual Quality Assessment of Pan-Sharpened Images

Pan-sharpening (PS) is a method of fusing the spatial details of a high-resolution panchromatic (PAN) image with the spectral information of a low-resolution multi-spectral (MS) image. Visual inspection is a crucial step in the evaluation of fused products whose subjectivity renders the assessment of pansharpened data a challenging problem. Most previous research on the development of PS algorithms has only superficially addressed the issue of qualitative evaluation, generally by depicting visual representations of the fused images. Hence, it is highly desirable to be able to predict pan-sharpened image quality automatically and accurately, as it would be perceived and reported by human viewers. Such a method is indispensable for the correct evaluation of PS techniques that produce images for visual applications such as Google Earth and Microsoft Bing. Here, we propose a new image quality assessment (IQA) measure that supports the visual qualitative analysis of pansharpened outcomes by using the statistics of natural images, commonly referred to as natural scene statistics (NSS), to extract statistical regularities from PS images. Importantly, NSS are measurably modified by the presence of distortions. We analyze six PS methods in the presence of two common distortions, blur and white noise, on PAN images. Furthermore, we conducted a human study on the subjective quality of pristine and degraded PS images and created a completely blind (opinion-unaware) fused image quality analyzer. In addition, we propose an opinion-aware fused image quality analyzer, whose predictions with respect to human perceptual evaluations of pansharpened images are highly correlated.

On the use of joint sparse representation for image fusion quality evaluation and analysis

In this paper, a Spare Representation (SR) based fusion quality evaluation and analysis method is proposed. This method employs Joint Sparse Representation (JSR) to extract the source image remnants after fusion. These atom-level remnants indicate the fusion quality intuitively, and permit the analysis of fusion effect in learned feature space. Our analysis results indicate that high salient atoms always present poor expressions in fusion results. An improved fusion rule is designed to emphasis high salient atoms accordingly. In experiments, the effectiveness of our method was verified and the characteristics of atom JSR remnants were investigated in detail first. Then the new fusion rule was tested to demonstrate the value of JSR remnant analysis. The objective and subjective comparison results indicate that the proposed analytical evaluation metric can measure fusion quality and analysis atom fusion effect accurately. The new fusion rule provides a valuable alternative for SR fusion algorithm design.

In-depth analysis of Tsallis entropy-based measures for image fusion quality assessment

Entropy-based measures are popular for objective image fusion quality assessment due to a small parameter set for implementation and independency of ground-truth image as the reference for evaluation. We focus on Tsallis entropy and consider mutual entropy and entropic distance as the two entropic measures for image fusion quality assessment. To perform an in-depth analysis over quality measures and evaluate to what extent they are able to fulfill desired behaviors that are expected from ideal image fusion quality measures, we separately conduct theoretical analysis for each of them. To this goal, we employ an image formation model to obtain a closed-form expression for quality while weighted averaging is used as fusion algorithm. Our study shows that the so-called measures do not always satisfy the expected desired behaviors. We also provide explanations for unexpected behaviors that can improve the accuracy of image fusion quality measure in application. Investigations on real images are also performed, and the results verify the output of theoretical analysis.

MRI-TRUS fusion biopsy of the prostate: Quality of image fusion in a clinical setting.

Prostate cancer (PCa) is one of the most common malignancies in men. The diagnostic standard to confirm prostate cancer is the transrectal ultrasound-guided biopsy. However, this procedure is associated with the underdetection of clinically significant prostate cancer and therefore needs to be improved. In the last years MRI fusion based targeted biopsy gained importance as consequence. In this study, we evaluated the quality of MRI ultrasound image fusion and evaluated factors influencing the image fusion quality. This was done by comparing fusion quality with the histopathological findings in the defined MRI target on the one hand and the PIRADS score on the other hand. Single arm study including patients with elevated prostate specific antigen (PSA) and a multiparametric MRI showing a suspicious lesion underwent a MRI fusion targeted biopsy at our institution. MRI fusion targeted biopsy and an additional 12-core transrectal ultrasound (TRUS) guided biopsy was performed using the Philips Percunav device (Philips Medical Systems, Bothell, WA). The fusion accuracy was rated by two experienced clinicians (1 radiologist, 18 years of experience, 1 urologist, 5 years of experience) using a five-point rank scale (1 = best) and comparing the result with the histological findings in the target and the PIRADS score. The detection rate of clinically significant cancer (Gleason 7a or greater) by MRI-ultrasound fusion targeted biopsy was 58.6% (17/29) compared to 50% (19/38) in the standard transrectal ultrasound-guided approach. PCa was found in 36.4% (4/11 patients) of patients with a PIRADS 3 lesion, in 57.7% (15/26 patients) of patients with a PIRADS 4 lesion. In 76.9% (10/13 patients) of patients with a PIRADS 5 lesion PCa was diagnosed. No statistical significance was found comparing the quality of registration either with the PIRADS (p = 0.7873) nor with the Gleason score (p = 0.4376). The study is limited by the small number of patients. MRI fusion based targeted biopsy improves the identification of clinical significant cancer. The Gleason score of detected PCa is not influenced by the quality of fusion.

Multimodal medical image fusion based on Yager’s intuitionistic fuzzy sets

The objective of image fusion for medical images is to combine multiple images obtained from various sources into a single image suitable for better diagnosis. Most of the state-of-the-art image fusing technique is based on nonfuzzy sets, and the fused image so obtained lags with complementary information. Intuitionistic fuzzy sets (IFS) are determined to be more suitable for civilian, and medical image processing as more uncertainties are considered compared with fuzzy set theory. In this paper, an algorithm for effectively fusing multimodal medical images is presented. In the proposed method, images are initially converted into Yager’s intuitionistic fuzzy complement images (YIFCIs), and a new objective function called intuitionistic fuzzy entropy (IFE) is employed to obtain the optimum value of the parameter in membership and non-membership functions. Next, the YIFCIs are compared using contrast visibility (CV) to construct a decision map (DM). DM is refined with consistency verification to create a fused image. Simulations on several pairs of multimodal medical images are performed and compared with the existing fusion methods, such as simple average, discrete cosine transform (DCT), redundant wavelet transform (RWT), intuitionistic fuzzy set, fuzzy transform and interval-valued intuitionistic fuzzy set (IVIFS). The superiority of the proposed method is presented and is justified. Fused image quality is also verified with various quality metrics, such as spatial frequency (SF), average gradient (AG), fusion symmetry (FS), edge information preservation (QAB/F), entropy (E) and computation time (CoT).

Multi-scale fidelity measure for image fusion quality assessment

Image fusion is considered an effective enhancing methodology widely included in high-quality imaging systems. Nevertheless, like other enhancing techniques, output quality assessment is made within small sample subjective evaluation studies which are very limited in predicting the human-perceived quality of general image fusion outputs. Simple, blind, universal and perceptual-like methods for assessing composite image quality are still a challenge, partially solved only in particular applications. In this paper, we propose a fidelity measure, called MS-QW with two major characteristics related to natural image statistics framework: A multi-scale computation and a structural similarity score. In our experiments, we correlate the scores of our measure with subjective ratings and state of the art measures included in the 2015 Waterloo IVC multi-exposure fusion (MEF) image database. We also use the measure to rank correctly the classical general fusion methods included in the Image Fusion Toolbox for medical, infra-red and multi-focus image examples. Moreover, we study the scores variability and statistical discrimination power with the TNO night vision database using the Friedman test. Finally, we define a new leave one out procedure based on our fidelity measure that selects the best subset of images (within a collection of distorted and unregistered cell phone type images) that provides a defect-free composite output. We exemplify the procedure with the fusion of a collection of images from Latour and Van Dongen paintings suffering from glass highlights and speckle noise, among other artifacts. The proposed multiscale quality measure MS-QW demonstrates improvement over the previous single-scale similarity measures towards a fidelity assessment between quantitative image fusion quality metrics and human perceptual qualitative scores.

Modeling of image, video and text fusion quality data packet system for aerospace complex products based on business intelligence

For the construction of image, video and text fusion quality data packet system during the whole life of complex products, a business intelligence-based logic modeling method is proposed in this paper. As the amount of Polymorphic data from multiple distributed sources continues to grow exponentially, automation tools are becoming critical to decision makers. The balanced scorecard method is used as the basis for modeling, and the traditional dimensions are modified slightly to meet the requirements of quality data management in aerospace enterprises. A data warehouse with predefined fact and dimension tables is created, and a technical solution is provided to meet the requirements and scales of enterprises. In terms of model applications, any enterprise quality manager can extract value from data in the quality data package system for improvement. Online analytical processing (OLAP) cubes support major tasks, such as key quality feature source tracing and analysis, quality issue data mining and integrated quality data delivery.

Investigation of Computed Tomography Exposure Dose for Whole-body 18F-FDG PET/CT Examination in Chugoku-Shikoku Regions

We investigated the way of thinking about the CT dose setting, the exposure dose (CTDIvol, DLP) and the image quality (standard deviation of liver: SDliver) of whole-body PET/CT examinations in the Chugoku and Shikoku regions for the optimized CT dose setting. It was researched in the target 29 facilities which is equipped with 18F-FDG, PET/CT scanner in that regions. We examined how to determine the dose of complete physical PET/CT setting, the CT radiation dose (CTDIvol and DLP) and the image quality of CT fusion image that is the standard deviation of liver CT value in each facility. The optimized CT dose was lower than the diagnostic CT in many facilities. The 75th percentile CTDIvol was 6.01 mGy. The 75th percentile DLP was 560.9 mGy ×cm. The SDliver was 14.6±5.3 HU. The CT condition was low setting in comparison with diagnostic CT in many facilities. The exposure dose was lower than the diagnostic CT dose. The image quality of the normal region of liver was very close to the diagnostic CT. Even though it was low dose, images were less noise components.

Synthetic aperture radar and optical remote sensing image fusion for flood monitoring in the Vietnam lower Mekong basin: a prototype application for the Vietnam Open Data Cube

ABSTRACT Flood monitoring systems are crucial for flood management and consequence mitigation in flood prone regions. Different remote sensing techniques are increasingly used for this purpose. However, the different approaches suffer various limitations, including cloud and weather effects (optical data), and low spatial resolution and poor colour presentation (synthetic aperture radar data). This study fuses two data types (Landsat and Sentinel-1) to overcome these limitations and produce better quality images for a prototype flood application in the Vietnam Open Data Cube (VODC). Visual and quantitative evaluation of fused image quality revealed improvement in the images compared with the original scenes. Ground-truth data was used to develop the study flood extraction algorithm and we found a good agreement between our results and SERVIR Mekong (a joint initiative by the US agency for International Development (USAID), National Aeronautics and Space Administration (NASA), Myanmar, Thailand, Cambodia, Laos and Vietnam) maps. While the algorithm is run on a personal computer (PC), it has a clear potential to be developed for application on a big data system.

Multispectral image fusion using super-resolution conditional generative adversarial networks

In multispectral image fusion scenarios, deep learning has been widely applied. However, the fusion performance and image quality are still restricted by inflexible architecture and supervised learning mode. We proposed multispectral image fusion using super-resolution conditional generative adversarial networks (MS-cGANs) based on conditional cGANs, which produces the fused image through the flexible encode-and-decode procedure. In the proposed network, a least square model is extended to solve the gradients vanishing problem in cGANs. Then, to improve the fusion quality, the multiscale features are used to preserve the details. Furthermore, the image resolution is promoted by adding the perceptual loss in object function and injecting the super-resolution structure into a deconvolution procedure. In experimental results, MS-cGANs demonstrates a significant performance in fusing multispectral images and top-ranking image quality compared with the state-of-the-art methods.

Remote sensing image fusion via RPCA and adaptive PCNN in NSST domain

In order to improve fused image quality of multi-spectral (MS) image and panchromatic (PAN) image, a new remote sensing image fusion algorithm based on robust principal component analysis (RPCA) and non-subsampled shearlet transform (NSST) is proposed. First, the first principle component PC1 of MS image is extracted via principal component analysis (PCA). Then, the component PC1 and PAN image are decomposed by NSST to get the low and high frequency subbands, respectively. For the low frequency subband, the sparse matrix of PAN image by RPCA decomposition is used to guide the fusion rule; for the high frequency subbands, the fusion rule employed is based on adaptive PCNN model. Finally, the fusion image is obtained by inverse NSST transform and inverse PCA transform. The experimental results illustrate that the proposed fusion algorithm outperforms other classical fusion algorithms (PCA, Curvelet, NSCT, NSST and NSCT-PCNN) in terms of visual quality and objective evaluation in whole, and achieve better fusion performance.

Diabetic plantar pressure analysis using image fusion

Plantar pressure images analysis is the key issue of designing comfortable shoe products through last customizing system, which has attracted the researchers’ curiosity toward image fusion as an application of medical and industrial imaging. In the current work, image fusion has been applied using wavelet transform and compared with Laplace Pyramid. Using image fusion rules of Mean-Max, we presented a plantar pressure image fusion method employing haar wavelet transform. It was compared in different composition layers with the Laplace pyramid transform. The experimental studies deployed the haar, db2, sym4, coif2, and bior5.5 wavelet basis functions for image fusion under decomposition layers of 3, 4, and 5. Evaluation metrics were measured in the case of the different layer number of wavelet decomposition to determine the best decomposition level and to evaluate the fused image quality using with different wavelet functions. The best wavelet basis function and decomposition layers were selected through the analysis and the evaluation measurements. This study established that haar wavelet transform with five decomposition levels on plantar pressure image achieved superior performance of 89.2817% mean, 89.4913% standard deviation, 5.4196 average gradient, 14.3364 spatial frequency, 5.9323 information entropy and 0.2206 cross entropy.