A Novel Multi-Focus Image Fusion Network with U-Shape Structure.

Tao Pan,Bin Tan,Jian Yao,Jiaqin Jiang,Bin Wang

doi:10.3390/s20143901

Tao Pan, Bin Tan + Show 3 more

Open Access

https://doi.org/10.3390/s20143901

Copy DOI

Abstract

Multi-focus image fusion has become a very practical image processing task. It uses multiple images focused on various depth planes to create an all-in-focus image. Although extensive studies have been produced, the performance of existing methods is still limited by the inaccurate detection of the focus regions for fusion. Therefore, in this paper, we proposed a novel U-shape network which can generate an accurate decision map for the multi-focus image fusion. The Siamese encoder of our U-shape network can preserve the low-level cues with rich spatial details and high-level semantic information from the source images separately. Moreover, we introduce the ResBlocks to expand the receptive field, which can enhance the ability of our network to distinguish between focus and defocus regions. Moreover, in the bridge stage between the encoder and decoder, the spatial pyramid pooling is adopted as a global perception fusion module to capture sufficient context information for the learning of the decision map. Finally, we use a hybrid loss that combines the binary cross-entropy loss and the structural similarity loss for supervision. Extensive experiments have demonstrated that the proposed method can achieve the state-of-the-art performance.

Highlights

Obtaining an all-in-focus image of a scene is essential for many computer vision and image analysis tasks
To accurately generate a high-quality decision map for multi-focus image fusion, we propose three improvements based on the U-shape network backbone, which are (1) using the Siamese encoder in our U-shape network to retain the multi-level features from two source images, (2) introducing
We visualize a part of the decision maps generated by our network and the final fusion results. Please note that these decision maps do not go through any post-processing steps, such as consistency verification (CV), small region removal, morphological operations, guided filters and conditional random field optimization (CRF)

Summary

Introduction

Obtaining an all-in-focus image of a scene is essential for many computer vision and image analysis tasks. Multi-focus image fusion is a common method used to solve this issue by the way of image processing. The existing multi-focus image fusion methods can be divided into two categories, i.e., the transform domain methods and the spatial domain methods [1,2]. The source images are decomposed into a special domain according to a certain transform method. The transformed coefficients are fused based on artificially designed fusion criteria

Methods

Results

Conclusion