CSwinDoubleU-Net: A double U-shaped network combined with convolution and Swin Transformer for colorectal polyp segmentation

Abstract

The early detection of polyps from colonoscopy can reduce the risk of colorectal cancer development and give timely treatment. Accurate polyp segmentation during colonoscopy examinations can help clinicians locate the polyps which is of great significance in the clinical prevention of colorectal cancer. However, due to large variations in terms of size, color, texture, and morphology of polyps, the similarity between the polyp lesions and their background, the variations of illumination, motion blur, low-contrast areas, intestinal contents during image acquisitions, precisely polyp segmentation is still an open issue. To overcome the challenges above, a new double U-shaped image segmentation network combining convolution structure and Shifted Windows (Swin) Transformer, namely CSwinDoubleU-Net, is proposed in this paper. The developed CSwinDoubleU-Net consists of two U-shaped encode-decode structures associated with a pure CNN-based structure and a pure Transformer-based one, and a feature fusion module that interacts with the two U-shaped parts. The first U-shaped codecs network utilizes the multi-convolution layers to extract the local feature information and the coordinate attention module in each skip connection to effectively reduce the loss of spatial information and obtain the accurate position information of the encoded features. Next, the second U-shaped codecs network uses the Swin Transformer layers with the sliding window to extract global feature information further. At last, a convolutional feature and self-attention feature fusion module (CSFFM) is designed to deeply fuse the local convolutional features extracted from the first U-shape structure and the global self-Attention features extracted from the second U-shape structure. The obtained multi-category and multi-dimensional fused feature information can help to recover the boundary features of polyps. Extensive experiments are conducted to validate the proposed CSwinDoubleU-Net on five publicly available datasets, including CVC-ClinicDB, Kvasir, CVC-ColonDB, CVC-T, and ETIS-Larib. The results show that the proposed model can outperform some state-of-the-art methods and achieve high segmentation performance for polyp images.