Green Fluorescent Protein and Phase Contrast Image Fusion Via Detail Preserving Cross Network

Abstract

In cell and molecular biology, the fusion of green fluorescent protein (GFP) and phase contrast (PC) images aims to generate a composite image, which can simultaneously display the functional information in the GFP image related to the molecular distribution of biological living cells and the structural information in the PC image such as nucleus and mitochondria. In this paper, we propose a detail preserving cross network (DPCN), which consists of a structural-guided functional feature extraction branch (SFFEB), a functional-guided structural feature extraction branch (FSFEB) and a detail preserving module (DPM), to address the GFP and PC image fusion issue. Unlike traditional parallel multi-branch architectures used for multiple inputs, the SFFEB and the FSFEB are interacted via a cross manner to fuse the functional information from the GFP image and the structural information from the PC image more adequately. Moreover, the DPM is composed of eight multi-scale convolutional blocks (MSCBs) associated with short, medium, and long skip connections to further extract the detail information from the source images. Experimental results on the popular Arabidopsis thaliana cell database demonstrate that the proposed method outperforms the state-of-the-art methods in terms of both qualitative and quantitative evaluations. The proposed method is also extended to deal with the functional and structural image fusion issue in medical imaging, and the promising results obtained exhibit its good generalizability. The code of our method is available at https://github.com/yuliu316316/DPCN-Fusion.