FAFS-UNet: Redesigning skip connections in UNet with feature aggregation and feature selection

Abstract

In recent years, the encoder–decoder U-shaped network architecture has become a mainstream structure for medical image segmentation. Its biggest advantage lies in the incorporation of shallow features into deeper layers of the network through skip connections. However, according to our research, there are still some limitations in the skip connection part of the network: (1) The information from the encoder stage is not completely and effectively supplemented to the decoder stage; (2) The decoder receives the supplemented feature information from the encoder indiscriminately, which sometimes leads to the poor performance of the model. Therefore, to effectively address these limitations, we have redesigned the skip connections in UNet using a feature aggregation and feature selection approach. We firstly design the FA module to aggregate all encoder features and perform local multi-scale information extraction to obtain the complete multi-scale aggregated features. Further, we design the FS module to actively perform specific selection of these aggregated features through the decoder, thus effectively guiding the semantic recovery of the decoder. Finally, we conduct experiments on several medical image datasets, and the results show that our method has higher segmentation accuracy compared with other methods.