NMNet: Learning Multi-level semantic information from scale extension domain for improved medical image segmentation

Abstract

Medical image segmentation methods based on encoder-decoder network structure have gained great success. However, these methods inevitably cause feature loss due to the pooling operation on the features during the encoding stage. Furthermore, there exists semantic gaps caused by the difference between low-level features and high-level features in the encoder-decoder network structure. By fusing the contextual features through simple skip-connections, it will limit the segmentation performance. To address these problems, this paper presents a new network for medical image segmentation, termed as NMNet, which mainly consists of a reverse encoder-decoder major structure with new attention modules. Specifically, in this network, we first design an N-shaped reverse encoder-decoder medical image segmentation structure (NNet), which can effectively reduce the impact of feature loss during the encoding process by performing feature representation compensation from the scale extension domain. Then, we build a Multi-scale Cross-attention Mechanism (MSC) in the skip-connections, which can enhance low-level features to bridge the semantic gaps. Extensive experiments on three benchmark datasets show that our NMNet performs favorably against most state-of-the-art methods under different evaluation metrics.