BMCS-Net: A Bi-directional multi-scale cascaded segmentation network based on transformer-guided feature Aggregation for medical images

Abstract

convolutional neural networks (CNNs) show great potential in medical image segmentation tasks, and can provide reliable basis for disease diagnosis and clinical research. However, CNNs exhibit general limitations on modeling explicit long-range relation, and existing cures, resorting to building deep encoders along with aggressive downsampling operations, leads to loss of localized details. Transformer has naturally excellent ability to model the global features and long-range correlations of the input information, which is strongly complementary to the inductive bias of CNNs. In this paper, a novel Bi-directional Multi-scale Cascaded Segmentation Network, BMCS-Net, is proposed to improve the performance of medical segmentation tasks by aggregating these features obtained from Transformers and CNNs branches. Specifically, a novel feature integration technique, termed as Two-stream Cascaded Feature Aggregation (TCFA) module, is designed to fuse features in two-stream branches, and solve the problem of gradual dilution of global information in the network. Besides, a Multi-Scale Expansion-Aware (MSEA) module based on the convolution of feature perception and expansion is introduced to capture context information, and further compensate for the loss of details. Extensive experiments demonstrated that BMCS-Net has an excellent performance on both skin and Polyp segmentation datasets.