SFE-TRANSUNET: A TRANSFORMER-BASED U-NET WITH SKIPPED FEATURES ENHANCER FOR MEDICAL IMAGE SEGMENTATION

Abstract

In the field of medical image segmentation, deep convolutional neural networks have achieved satisfying performance in the past decade or so. However, there are some shortcomings. First, the convolutional neural network model cannot provide good insight into the remote dependencies in the image. Second, medical imaging datasets are typically small, which leads to a much higher risk of overfitting in model training. To address these limitations, we innovatively designed the skipped features enhancer (SFE) to enhance the impact of preserved details. To gain insight into remote dependencies in images, this model (SFE-TransUNet) is based on transformer. Additionally, a different scale convolutional layer (additional information capturer) before and after the Transformer Encoder to fuse the features to retain more information of the original data. In addition, a gate mechanism was introduced in the multi-head self-attention (MHSA), and. Finally, an attention block with residuals. SFE-TransUNet was evaluated on two public medical image segmentation datasets. Experimental results show that it achieves better performance than other related Transformer-based architectures. Code available at https://github.com/xackz/SFE-TransUNet .