Transformer-based multi-attention hybrid networks for skin lesion segmentation

Abstract

High-precision segmentation of skin lesions is essential for early diagnosis of skin cancer and improved patient survival. However, this task becomes challenging due to the irregularity of skin lesion regions, significant differences in color shades, ambiguity of boundaries, and other complex interfering factors. In this study, a transformer-based multi-attention hybrid network, TMAHU-Net, is proposed to cope with the complexity of skin lesion regions. The model introduces an innovative hybrid module that fully combines the advantages of CNNs and transformers to capture both global and local feature information. We also employ a deeply separable convolutional attention module to dynamically assign attention weights, which enhances the channel and spatial dimensions of feature information. To improve contextual relevance, we introduce an extended gated external attention module. Finally, a multi-scale aggregation module is used to fuse feature information at different scales to compensate for the differences in processing images with different resolutions. The model performs well in dealing with irregular, color shades, fuzzy boundaries, and complex interference lesion regions, which provides strong support for doctors to accurately determine complex lesion regions. Critically evaluated on three publicly available datasets, ISIC2016, ISIC2017, and PH2, TMAHU-Net achieved significant improvements over state-of-the-art methods, with F1 scores and IoU scores increasing by 1.01% and 1.01%, 1.22% and 1.40%, and 0.92% and 1.50%, respectively.