Multimodel fusion method via sparse representation at pixel-level and feature-level

Abstract

In current fusion methods based on sparse representation (SR) and different frequency, the SR is usually applied to fusion of the low-frequency components. In contrast, the direct fusion is usually adopted for high-frequency components due to their significant diversity. However, the effect of the direct fusion is degraded by the redundant information resulting from the correlation between original signals. A multimodel fusion framework is proposed by applying the SR to low-frequency fusion at pixel-level and high-frequency fusion at feature-level, respectively. First, the multimodal images are decomposed into high-frequency and low-frequency components by nonsubsampled contourlet transform (NSCT). Second, the universal high-frequency dictionary is constructed by using the fast independent component analysis (ICA) of the source high-frequency and its subband images. They represent the general feature part and unique feature part for the high-frequency signals, respectively. The universal low-frequency dictionary is constructed by using the original low-frequency signals. Third, the direct fusion of the high-frequency is converted into sparse coefficients fusion in fast ICA domain. Moreover, the multiple directive contrasts by modifying sum-modified Laplacian are taken as the fusion rule. The low-frequency signals are fused by using an activity measure based on weights. Finally, the fused image is obtained by inverse NSCT on the merged components. The experiments are conducted on three types of image pairs, and the results demonstrate that the proposed method outperforms seven state-of-the-art methods, in terms of four subjective and objective evaluations.