FRNet: an end-to-end feature refinement neural network for medical image segmentation

Abstract

Medical image segmentation is a crucial but challenging task for computer-aided diagnosis. In recent years, fully convolutional network-based methods have been widely applied to medical image segmentation. U-shape-based approaches are one of the most successful structures in this medical field. However, the consecutive down-sampling operations in the encoder lead to the loss of spatial information, which is important for medical image segmentation. In this paper, we present a novel lightweight end-to-end feature refinement network (FRNet) to address this issue. The structure of our model is simple and efficient. Specifically, the network adopts an encoder-decoder network as backbone, where two additional paths, spatial refinement path and semantic refinement path, are applied on the encoder and decoder, respectively, to improve the detailed representation ability and discriminative ability of our model. In addition, we introduce a feature adaptive fusion block (FAF block) that effectively combines features of different depths. The proposed FRNet can be trained in an end-to-end way. We have evaluated our method on three different medical image segmentation tasks. Experimental results show that FRNet has better performance than the state-of-the-art approaches. It achieves a high average accuracy without any post-processing of 0.968 and 0.936 for blood vessel segmentation and skin lesion segmentation, respectively. We further demonstrate that our method can be easily applied to other network structures to improve their performance.