MsRAN: a multi-scale residual attention network for multi-model image fusion.

Abstract

Fusion is a critical step in image processing tasks. Recently, deep learning networks have been considerably applied in information fusion. But the significant limitation of existing image fusion methods is the inability to highlight typical regions of the source image and retain sufficient useful information. To address the problem, the paper proposes a multi-scale residual attention network (MsRAN) to fully exploit the image feature. Its generator network contains two information refinement networks and one information integration network. The information refinement network extracts feature at different scales using convolution kernels of different sizes. The information integration network, with a merging block and an attention block added, prevents the underutilization of information in the intermediate layers and forces the generator to focus on salient regions in multi-modal source images. Furthermore, in the phase of model training, we add an information loss function and adopt a dual adversarial structure, enabling the model to capture more details. Qualitative and quantitative experiments on publicly available datasets validate that the proposed method provides better visual results than other methods and retains more detail information.