Multi-scale fidelity measure for image fusion quality assessment

Abstract

Image fusion is considered an effective enhancing methodology widely included in high-quality imaging systems. Nevertheless, like other enhancing techniques, output quality assessment is made within small sample subjective evaluation studies which are very limited in predicting the human-perceived quality of general image fusion outputs. Simple, blind, universal and perceptual-like methods for assessing composite image quality are still a challenge, partially solved only in particular applications. In this paper, we propose a fidelity measure, called MS-QW with two major characteristics related to natural image statistics framework: A multi-scale computation and a structural similarity score. In our experiments, we correlate the scores of our measure with subjective ratings and state of the art measures included in the 2015 Waterloo IVC multi-exposure fusion (MEF) image database. We also use the measure to rank correctly the classical general fusion methods included in the Image Fusion Toolbox for medical, infra-red and multi-focus image examples. Moreover, we study the scores variability and statistical discrimination power with the TNO night vision database using the Friedman test. Finally, we define a new leave one out procedure based on our fidelity measure that selects the best subset of images (within a collection of distorted and unregistered cell phone type images) that provides a defect-free composite output. We exemplify the procedure with the fusion of a collection of images from Latour and Van Dongen paintings suffering from glass highlights and speckle noise, among other artifacts. The proposed multiscale quality measure MS-QW demonstrates improvement over the previous single-scale similarity measures towards a fidelity assessment between quantitative image fusion quality metrics and human perceptual qualitative scores.