Label-aware Attention Network with Multi-scale Boosting for Medical Image Segmentation

Abstract

Deep medical image segmentation calls for features with strong discrimination and rich scales due to ambiguous background distraction and large variations in object sizes and shapes. In this paper, we propose two modules to obtain these features. First, existing encoders tend to extract similar foreground/background features at blurry boundaries due to mixed-label feature aggregation. To enhance the discrimination of these features, a Label-Aware Attention (LAA) module is presented to reconstruct them by fusing same-label local features. The fusion is guided by local attention maps based on label-aware affinity learning. Second, instead of relying on a single encoder for scale context mining, we propose a Multi-scale Feature Boosting (MFB) module that applies parallel convolution with different receptive fields for scale embedding and integrates an additional backbone for cross-encoder scale reference. Combining LAA and MFB, a new encoder–decoder based framework is presented, where MFBs act as encoder blocks to recursively extract features with rich scale context, while LAA operates in the decoder layer to enhance the label-aware discriminativeness of features. Extensive experiments on three standard medical segmentation datasets demonstrate the effectiveness of the proposed framework.