Multi-scale pixel-based image fusion using multivariate empirical mode decomposition.

Naveed Rehman,Muhammad Akhtar,Danilo Mandic,Klaus Mcdonald-Maier,Shoaib Ehsan,Syed Abdullah

doi:10.3390/s150510923

Abstract

A novel scheme to perform the fusion of multiple images using the multivariate empirical mode decomposition (MEMD) algorithm is proposed. Standard multi-scale fusion techniques make a priori assumptions regarding input data, whereas standard univariate empirical mode decomposition (EMD)-based fusion techniques suffer from inherent mode mixing and mode misalignment issues, characterized respectively by either a single intrinsic mode function (IMF) containing multiple scales or the same indexed IMFs corresponding to multiple input images carrying different frequency information. We show that MEMD overcomes these problems by being fully data adaptive and by aligning common frequency scales from multiple channels, thus enabling their comparison at a pixel level and subsequent fusion at multiple data scales. We then demonstrate the potential of the proposed scheme on a large dataset of real-world multi-exposure and multi-focus images and compare the results against those obtained from standard fusion algorithms, including the principal component analysis (PCA), discrete wavelet transform (DWT) and non-subsampled contourlet transform (NCT). A variety of image fusion quality measures are employed for the objective evaluation of the proposed method. We also report the results of a hypothesis testing approach on our large image dataset to identify statistically-significant performance differences.

Highlights

Image fusion refers to combining multiple images to produce a single output image that carries the salient features of all fused images
We have introduced a method to perform the fusion of multiple images using the multivariate empirical mode decomposition (MEMD) algorithm
The method operates by first employing MEMD to decompose input images into their constituent scale images and subsequently applying a pixel-based fusion scheme to the resulting sub-images, obtaining a fused image containing all relevant details from the input images

Summary

Introduction

Image fusion refers to combining multiple images to produce a single output image that carries the salient features of all fused images. Fusion techniques are relevant in cases where limitations of optical sensors and imaging conditions make it difficult to capture complete detail in a single image [1]. For naturally illuminated scenes, the information content in an image depends on the orientation of the light source relative to that of the object of interest; shadows may cause loss of information [3]. To overcome these impediments, multiple images containing partial information about a scene can be fused to reconstruct its complete information. Spatial domain fusion methods work directly with image (pixels) intensity values. The key steps involved are: (1) the generation of a quantitative map of the information content for each image; (2) comparison of information content at the pixel level;

Methods

Results

Conclusion