TranSEFusionNet: Deep fusion network for colorectal polyp segmentation

Abstract

Fully convolutional neural (FCN) networks like U-Net have been the state-of-the-art methods in colorectal polyp segmentation. However, U-Net still has some limitations in modelling remote semantic information, especially since the semantic information at different levels can vary greatly, making it difficult to utilize this information fully. To address these issues, we propose a new network architecture called TranSEFusionNet that utilizes the Transformer’s global modelling capability to better focus on global contextual semantic information. In addition, we added two feature fusion modules, a spatial feature fusion module (SFM) and an edge feature fusion module (EFM), to the network. SEF with a skip connection can improve the accuracy of passing deep features to shallow features. EFM in the output part of each decoder layer improves the recognition of edge ambiguous features by refining the semantic information of the network. We validate the model’s performance on five publicly available colorectal polyp datasets, and the experiments show that TranSEFusionNet has higher segmentation accuracy. To measure the generalization ability of TranSEFusionNet, we further applied the model to the cell nuclei dataset, which further verifies the performance of our model.Code: https://github.com/Linaaalin/TranSEFusionNet.