Accurate Medical Image Segmentation Research Articles

DA-TransUNet: integrating spatial and channel dual attention with transformer U-net for medical image segmentation

Accurate medical image segmentation is critical for disease quantification and treatment evaluation. While traditional U-Net architectures and their transformer-integrated variants excel in automated segmentation tasks. Existing models also struggle with parameter efficiency and computational complexity, often due to the extensive use of Transformers. However, they lack the ability to harness the image’s intrinsic position and channel features. Research employing Dual Attention mechanisms of position and channel have not been specifically optimized for the high-detail demands of medical images. To address these issues, this study proposes a novel deep medical image segmentation framework, called DA-TransUNet, aiming to integrate the Transformer and dual attention block (DA-Block) into the traditional U-shaped architecture. Also, DA-TransUNet tailored for the high-detail requirements of medical images, optimizes the intermittent channels of Dual Attention (DA) and employs DA in each skip-connection to effectively filter out irrelevant information. This integration significantly enhances the model’s capability to extract features, thereby improving the performance of medical image segmentation. DA-TransUNet is validated in medical image segmentation tasks, consistently outperforming state-of-the-art techniques across 5 datasets. In summary, DA-TransUNet has made significant strides in medical image segmentation, offering new insights into existing techniques. It strengthens model performance from the perspective of image features, thereby advancing the development of high-precision automated medical image diagnosis. The codes and parameters of our model will be publicly available at https://github.com/SUN-1024/DA-TransUnet.

Attention-Based Two-Branch Hybrid Fusion Network for Medical Image Segmentation

Accurate segmentation of medical images is vital for disease detection and treatment. Convolutional Neural Networks (CNN) and Transformer models are widely used in medical image segmentation due to their exceptional capabilities in image recognition and segmentation. However, CNNs often lack an understanding of the global context and may lose spatial details of the target, while Transformers struggle with local information processing, leading to reduced geometric detail of the target. To address these issues, this research presents a Global-Local Fusion network model (GLFUnet) based on the U-Net framework and attention mechanisms. The model employs a dual-branch network that utilizes ConvNeXt and Swin Transformer to simultaneously extract multi-level features from pathological images. It enhances ConvNeXt’s local feature extraction with spatial and global attention up-sampling modules, while improving Swin Transformer’s global context dependency with channel attention. The Attention Feature Fusion module and skip connections efficiently merge local detailed and global coarse features from CNN and Transformer branches at various scales. The fused features are then progressively restored to the original image resolution for pixel-level prediction. Comprehensive experiments on datasets of stomach and liver cancer demonstrate GLFUnet’s superior performance and adaptability in medical image segmentation, holding promise for clinical analysis and disease diagnosis.

DECTNet: Dual Encoder Network combined convolution and Transformer architecture for medical image segmentation.

Automatic and accurate segmentation of medical images plays an essential role in disease diagnosis and treatment planning. Convolution neural networks have achieved remarkable results in medical image segmentation in the past decade. Meanwhile, deep learning models based on Transformer architecture also succeeded tremendously in this domain. However, due to the ambiguity of the medical image boundary and the high complexity of physical organization structures, implementing effective structure extraction and accurate segmentation remains a problem requiring a solution. In this paper, we propose a novel Dual Encoder Network named DECTNet to alleviate this problem. Specifically, the DECTNet embraces four components, which are a convolution-based encoder, a Transformer-based encoder, a feature fusion decoder, and a deep supervision module. The convolutional structure encoder can extract fine spatial contextual details in images. Meanwhile, the Transformer structure encoder is designed using a hierarchical Swin Transformer architecture to model global contextual information. The novel feature fusion decoder integrates the multi-scale representation from two encoders and selects features that focus on segmentation tasks by channel attention mechanism. Further, a deep supervision module is used to accelerate the convergence of the proposed method. Extensive experiments demonstrate that, compared to the other seven models, the proposed method achieves state-of-the-art results on four segmentation tasks: skin lesion segmentation, polyp segmentation, Covid-19 lesion segmentation, and MRI cardiac segmentation.

Medical Image Segmentation with Dual-Encoding and Multi-Level Feature Adaptive Fusion

Purpose: Accurate segmentation of medical images is critical for disease diagnosis, surgical planning and prognostic assessment. TransUNet, a hybrid CNN-Transformer-based method, extracts local features using CNN and compensates for the lack of long-range dependencies through a self-attention mechanism. However, the initial focus on extracting local features from specific regions impacts the generation of subsequent global features, thus constraining the model’s capacity to effectively capture a broader range of semantic information. Effective integration of local and global features plays a pivotal role in achieving precise and dense prediction. Therefore, we propose a novel hybrid CNN-Transformer-based method aimed at enhancing medical image segmentation. Approach: In this study, a dual-encoder parallel structure is used to enhance the feature representation of the input image. By introducing a multi-scale adaptive feature fusion module, a fine fusion of local features across perceptual domains is realized in the decoding process. The generalized convolutional block attention module helps to increase cross-channel interactions in layers with more channels, thus enabling the fusion of local features and global representations at different resolutions during the decoding process. Results: The proposed method achieves average DSC scores of 79.98%, 84.83% and 85.78% on the Synapse, ISIC2017 and Pediatric Pyelonephritis datasets, respectively. These scores are 2.5%, 0.56% and 0.42% higher than those of TransUNet. The best performance of 91.66% is observed on the ACDC dataset, representing improvements of 2.46% and 7.24% compared to HiFormer and DAE-Former, respectively. Conclusions: The experimental results show that the proposed model has a significant competitive advantage in terms of ACDC image segmentation performance.

Semantic-Based Optimization of Deep Learning for Efficient Real-Time Medical Image Segmentation

In response to the critical need for advanced solutions in medical imaging segmentation, particularly for real-time applications in diagnostics and treatment planning, this study introduces SM-UNet. This novel deep learning architecture efficiently addresses the challenge of real-time, accurate medical image segmentation by integrating convolutional neural network (CNN) with multilayer perceptron (MLP). The architecture uniquely combines an initial convolutional encoder for detailed feature extraction, MLP module for capturing long-range dependencies, and a decoder that merges global features with high-resolution CNN map. Further optimization is achieved through a tokenization approach, significantly reducing computational demands. Its superior performance is confirmed by evaluations on standard datasets, showing interaction times drastically lower than comparable networks—between 1/6 to 1/10, and 1/25 compared to SOTA models. These advancements underscore SM-UNet's potential as a groundbreaking tool for facilitating real-time, precise medical diagnostics and treatment strategies.

MpMsCFMA-Net: Multi-path Multi-scale Context Feature Mixup and Aggregation Network for medical image segmentation

Automatic and accurate medical image segmentation is a crucial step for clinical diagnosis and treatment planning of diseases. The advanced convolutional neural network (CNN) approaches based on the encoder–decoder structure have achieved state-of-the-art performances in many different medical image segmentation tasks. However, existing networks have insufficient capability of extracting the context information in each encoding stage, so they cannot effectively perceive multi-scale objects in images. In addition, the continuous down-sampling and convolution operations in the encoding stage lead to much loss of the detailed information, resulting in poor segmentation performance. In this paper, we propose a Multi-path Multi-scale Context Feature MixUp and Aggregation Network (named MpMsCFMA-Net) which fuses and aggregates multi-path features with multi-scale context information to address these issues. Based on the encoder–decoder structure, we first design the encoder to encode the semantic and detailed information of input images and introduce multi-scale context extraction module in each encoding stage. Furthermore, we design multiple features mixup module between the encoder and the decoder, aiming at providing different levels of global context information for the decoder by reconstructing skip-connection. Finally, we introduce the decoder with deeper features aggregation to better fuse multi-scale context information across layers. Experimental results on four public medical image datasets confirm that our proposed network achieves promising results and outperforms other state-of-the-art methods in most of evaluation metrics. The source code will be publicly available at https://github.com/tricksterANDthug/MpMsCFMA-Net.

HmsU-Net: A hybrid multi-scale U-net based on a CNN and transformer for medical image segmentation

Accurate medical image segmentation is of great significance for subsequent diagnosis and analysis. The acquisition of multi-scale information plays an important role in segmenting regions of interest of different sizes. With the emergence of Transformers, numerous networks adopted hybrid structures incorporating Transformers and CNNs to learn multi-scale information. However, the majority of research has focused on the design and composition of CNN and Transformer structures, neglecting the inconsistencies in feature learning between Transformer and CNN. This oversight has resulted in the hybrid network's performance not being fully realized. In this work, we proposed a novel hybrid multi-scale segmentation network named HmsU-Net, which effectively fused multi-scale features. Specifically, HmsU-Net employed a parallel design incorporating both CNN and Transformer architectures. To address the inconsistency in feature learning between CNN and Transformer within the same stage, we proposed the multi-scale feature fusion module. For feature fusion across different stages, we introduced the cross-attention module. Comprehensive experiments conducted on various datasets demonstrate that our approach surpasses current state-of-the-art methods.

Advanced Medical Image Segmentation Enhancement: A Particle-Swarm-Optimization-Based Histogram Equalization Approach

Accurate medical image segmentation is paramount for precise diagnosis and treatment in modern healthcare. This research presents a comprehensive study of the efficacy of particle swarm optimization (PSO) combined with histogram equalization (HE) preprocessing for medical image segmentation, focusing on lung CT scan and chest X-ray datasets. Best-cost values reveal the PSO algorithm’s performance, with HE preprocessing demonstrating significant stabilization and enhanced convergence, particularly for complex lung CT scan images. Evaluation metrics, including accuracy, precision, recall, F1-score/Dice, specificity, and Jaccard, show substantial improvements with HE preprocessing, emphasizing its impact on segmentation accuracy. Comparative analyses against alternative methods, such as Otsu, Watershed, and K-means, confirm the competitiveness of the PSO-HE approach, especially for chest X-ray images. The study also underscores the positive influence of preprocessing on image clarity and precision. These findings highlight the promise of the PSO-HE approach for advancing the accuracy and reliability of medical image segmentation and pave the way for further research and method integration to enhance this critical healthcare application.

δARD loss for low-contrast medical image segmentation

Medical image segmentation is essential to image-based disease analysis and has proven to be significantly helpful for doctors to make decisions. Due to the low-contrast of some medical images, the accurate segmentation of medical images has always been a challenging problem. The experiment found that UNet with current loss functions cannot capture subtle information in target contours or regions in low-contrast medical images, which are crucial for subsequent disease diagnosis. We propose a robust loss by incorporating the difference in average radial derivative (ARD), length and region area to further help the network to achieve more accurate segmentation results. We evaluated the proposed loss function using UNet as the base segmentation network compared to five conventional loss functions on one private and four public medical image datasets. Experimental results illustrate that UNet with the proposed loss function can achieve the best segmentation performance, even better than the outstanding deep learning models with original loss functions. Furthermore, three representative datasets were chosen to validate the effectiveness of the proposed δARD loss function with seven different models. The experiments revealed δARD loss’s plug-and-play feature and its robustness over multiple models and datasets.

Towards Accurate Medical Image Segmentation With Gradient-Optimized Dice Loss

Towards Accurate Medical Image Segmentation With Gradient-Optimized Dice Loss

An Efficient and Rapid Medical Image Segmentation Network.

Accurate medical image segmentation is an essential part of the medical image analysis process that provides detailed quantitative metrics. In recent years, extensions of classical networks such as UNet have achieved state-of-the-art performance on medical image segmentation tasks. However, the high model complexity of these networks limits their applicability to devices with constrained computational resources. To alleviate this problem, we propose a shallow hierarchical Transformer for medical image segmentation, called SHFormer. By decreasing the number of transformer blocks utilized, the model complexity of SHFormer can be reduced to an acceptable level. To improve the learned attention while keeping the structure lightweight, we propose a spatial-channel connection module. This module separately learns attention in the spatial and channel dimensions of the feature while interconnecting them to produce more focused attention. To keep the decoder lightweight, the MLP-D module is proposed to progressively fuse multi-scale features in which channels are aligned using Multi-Layer Perceptron (MLP) and spatial information is fused by convolutional blocks. We first validated the performance of SHFormer on the ISIC-2018 dataset. Compared to the latest network, SHFormer exhibits comparable performance with 15 times fewer parameters, 30 times lower computational complexity and 5 times higher inference efficiency. To test the generalizability of SHFormer, we introduced the polyp dataset for additional testing. SHFormer achieves comparable segmentation accuracy to the latest network while having lower computational overhead.

Cross-domain attention-guided generative data augmentation for medical image analysis with limited data

Data augmentation is widely applied to medical image analysis tasks in limited datasets with imbalanced classes and insufficient annotations. However, traditional augmentation techniques cannot supply extra information, making the performance of diagnosis unsatisfactory. GAN-based generative methods have thus been proposed to obtain additional useful information to realize more effective data augmentation; but existing generative data augmentation techniques mainly encounter two problems: (i) Current generative data augmentation lacks of the capability in using cross-domain differential information to extend limited datasets. (ii) The existing generative methods cannot provide effective supervised information in medical image segmentation tasks. To solve these problems, we propose an attention-guided cross-domain tumor image generation model (CDA-GAN) with an information enhancement strategy. The CDA-GAN can generate diverse samples to expand the scale of datasets, improving the performance of medical image diagnosis and treatment tasks. In particular, we incorporate channel attention into a CycleGAN-based cross-domain generation network that captures inter-domain information and generates positive or negative samples of brain tumors. In addition, we propose a semi-supervised spatial attention strategy to guide spatial information of features at the pixel level in tumor generation. Furthermore, we add spectral normalization to prevent the discriminator from mode collapse and stabilize the training procedure. Finally, to resolve an inapplicability problem in the segmentation task, we further propose an application strategy of using this data augmentation model to achieve more accurate medical image segmentation with limited data. Experimental studies on two public brain tumor datasets (BraTS and TCIA) show that the proposed CDA-GAN model greatly outperforms the state-of-the-art generative data augmentation in both practical medical image classification tasks and segmentation tasks; e.g. CDA-GAN is 0.50%, 1.72%, 2.05%, and 0.21% better than the best SOTA baseline in terms of ACC, AUC, Recall, and F1, respectively, in the classification task of BraTS, while its improvements w.r.t. the best SOTA baseline in terms of Dice, Sens, HD95, and mIOU, in the segmentation task of TCIA are 2.50%, 0.90%, 14.96%, and 4.18%, respectively.

MTAN: A semi-supervised learning model for kidney tumor segmentation.

Medical image segmentation is crucial in disease diagnosis and treatment planning. Deep learning (DL) techniques have shown promise. However, optimizing DL models requires setting numerous parameters, and demands substantial labeled datasets, which are labor-intensive to create. This study proposes a semi-supervised model that can utilize labeled and unlabeled data to accurately segment kidneys, tumors, and cysts on CT images, even with limited labeled samples. An end-to-end semi-supervised learning model named MTAN (Mean Teacher Attention N-Net) is designed to segment kidneys, tumors, and cysts on CT images. The MTAN model is built on the foundation of the AN-Net architecture, functioning dually as teachers and students. In its student role, AN-Net learns conventionally. In its teacher role, it generates objects and instructs the student model on their utilization to enhance learning quality. The semi-supervised nature of MTAN allows it to effectively utilize unlabeled data for training, thus improving performance and reducing overfitting. We evaluate the proposed model using two CT image datasets (KiTS19 and KiTS21). In the KiTS19 dataset, MTAN achieved segmentation results with an average Dice score of 0.975 for kidneys and 0.869 for tumors, respectively. Moreover, on the KiTS21 dataset, MTAN demonstrates its robustness, yielding average Dice scores of 0.977 for kidneys, 0.886 for masses, 0.861 for tumors, and 0.759 for cysts, respectively. The proposed MTAN model presents a compelling solution for accurate medical image segmentation, particularly in scenarios where the labeled data is scarce. By effectively utilizing the unlabeled data through a semi-supervised learning approach, MTAN mitigates overfitting concerns and achieves high-quality segmentation results. The consistent performance across two distinct datasets, KiTS19 and KiTS21, underscores model's reliability and potential for clinical reference.

Exploring a frequency-domain attention-guided cascade U-Net: Towards spatially tunable segmentation of vasculature

Developing fully automatic and highly accurate medical image segmentation methods is critically important for vascular disease diagnosis and treatment planning. Although advances in convolutional neural networks (CNNs) have spawned an array of automatic segmentation models converging to saturated high performance, none have explored whether CNNs can achieve (spatially) tunable segmentation. As a result, we propose multiple attention modules from a frequency-domain perspective to construct a unified CNN architecture for segmenting vasculature with desired (spatial) scales. The proposed CNN architecture is named frequency-domain attention-guided cascaded U-Net (FACU-Net). Specifically, FACU-Net contains two innovative components: (1) a frequency-domain-based channel attention module that adaptively tunes channel-wise feature responses and (2) a frequency-domain-based spatial attention module that enables the deep network to concentrate on foreground regions of interest (ROIs) effectively.Furthermore, we devised a novel frequency-domain-based content attention module to enhance or weaken the high (spatial) frequency information, allowing us to strengthen or eliminate vessels of interest. Extensive experiments using clinical data from patients with intracranial aneurysms (IA) and abdominal aortic aneurysms (AAA) demonstrated that the proposed FACU-Net met its design goal. In addition, we further investigated the association between varying (spatial) frequency components and the desirable vessel size/scale attributes. In summary, our preliminary findings are encouraging, and further developments may lead to deployable image segmentation models that are spatially tunable for clinical applications.

A deep model towards accurate boundary location and strong generalization for medical image segmentation

Accurate medical image segmentation plays a crucial role in computer-assisted diagnosis and monitoring. However, due to the complexity of medical images and the limitations of image acquisition, most of the current segmentation models still face the challenges of imprecise boundary positioning and poor generalization ability, which greatly affect their application in clinical practices. To address these issues, we propose a new deep learning-based medical image semantic segmentation network (BLG-Net). To improve the accuracy of object boundary location, a Local–global module (Logo) is proposed. It uses the combination of a self-attention mechanism and a cross-attention mechanism to model intra-sample and inter-sample long-range dependencies from local and global perspectives, respectively, to help recover the spatial structures. To further enhance the generalization ability of the BLG-Net, we develop a generalization enhancement module (GEM), which can adaptively adjust the classifier weights of the network according to the features of the test images extracted during the test process to alleviate the distribution shift issues. Besides, we design a multi-granularity loss (MGL) function, which can guide the network to learn the object region and its boundary details at the pixel, patch, and map levels, respectively. Extensive experiments on five medical image segmentation tasks demonstrate the improvements of our method. We further verify the model’s generalization ability to unseen data on two other popular datasets with different modalities.

FAFuse: A Four-Axis Fusion framework of CNN and Transformer for medical image segmentation

Medical image segmentation is crucial for accurate diagnosis and treatment in the medical field. In recent years, convolutional neural networks (CNNs) and Transformers have been frequently adopted as network architectures in medical image segmentation. The convolution operation is limited in modeling long-range dependencies because it can only extract local information through the limited receptive field. In comparison, Transformers demonstrate excellent capability in modeling long-range dependencies but are less effective in capturing local information. Hence, effectively modeling long-range dependencies while preserving local information is essential for accurate medical image segmentation. In this paper, we propose a four-axis fusion framework called FAFuse, which can exploit the advantages of CNN and Transformer. As the core component of our FAFuse, a Four-Axis Fusion module (FAF) is proposed to efficiently fuse global and local information. FAF combines Four-Axis attention (height, width, main diagonal, and counter diagonal axial attention), a multi-scale convolution, and a residual structure with a depth-separable convolution and a Hadamard product. Furthermore, we also introduce deep supervision to enhance gradient flow and improve overall performance. Our approach achieves state-of-the-art segmentation accuracy on three publicly available medical image segmentation datasets. The code is available at https://github.com/cczu-xiao/FAFuse.

Self-supervised Semantic Segmentation: Consistency over Transformation.

Accurate medical image segmentation is of utmost importance for enabling automated clinical decision procedures. However, prevailing supervised deep learning approaches for medical image segmentation encounter significant challenges due to their heavy dependence on extensive labeled training data. To tackle this issue, we propose a novel self-supervised algorithm, , which integrates a robust framework based on the proposed Inception Large Kernel Attention (I-LKA) modules. This architectural enhancement makes it possible to comprehensively capture contextual information while preserving local intricacies, thereby enabling precise semantic segmentation. Furthermore, considering that lesions in medical images often exhibit deformations, we leverage deformable convolution as an integral component to effectively capture and delineate lesion deformations for superior object boundary definition. Additionally, our self-supervised strategy emphasizes the acquisition of invariance to affine transformations, which is commonly encountered in medical scenarios. This emphasis on robustness with respect to geometric distortions significantly enhances the model's ability to accurately model and handle such distortions. To enforce spatial consistency and promote the grouping of spatially connected image pixels with similar feature representations, we introduce a spatial consistency loss term. This aids the network in effectively capturing the relationships among neighboring pixels and enhancing the overall segmentation quality. The approach iteratively learns pixel-level feature representations for image content clustering in an end-to-end manner. Our experimental results on skin lesion and lung organ segmentation tasks show the superior performance of our method compared to the SOTA approaches.

A Novel Deep Learning Model for Medical Image Segmentation with Convolutional Neural Network and Transformer.

Accurate segmentation of medical images is essential for clinical decision-making, and deep learning techniques have shown remarkable results in this area. However, existing segmentation models that combine transformer and convolutional neural networks often use skip connections in U-shaped networks, which may limit their ability to capture contextual information in medical images. To address this limitation, we propose a coordinated mobile and residual transformer UNet (MRC-TransUNet) that combines the strengths of transformer and UNet architectures. Our approach uses a lightweight MR-ViT to address the semantic gap and a reciprocal attention module to compensate for the potential loss of details. To better explore long-range contextual information, we use skip connections only in the first layer and add MR-ViT and RPA modules in the subsequent downsampling layers. In our study, we evaluated the effectiveness of our proposed method on three different medical image segmentation datasets, namely, breast, brain, and lung. Our proposed method outperformed state-of-the-art methods in terms of various evaluation metrics, including the Dice coefficient and Hausdorff distance. These results demonstrate that our proposed method can significantly improve the accuracy of medical image segmentation and has the potential for clinical applications. Illustration of the proposed MRC-TransUNet. For the input medical images, we first subject them to an intrinsic downsampling operation and then replace the original jump connection structure using MR-ViT. The output feature representations at different scales are fused by the RPA module. Finally, an upsampling operation is performed to fuse the features to restore them to the same resolution as the input image.

H2Former: An Efficient Hierarchical Hybrid Transformer for Medical Image Segmentation.

Accurate medical image segmentation is of great significance for computer aided diagnosis. Although methods based on convolutional neural networks (CNNs) have achieved good results, it is weak to model the long-range dependencies, which is very important for segmentation task to build global context dependencies. The Transformers can establish long-range dependencies among pixels by self-attention, providing a supplement to the local convolution. In addition, multi-scale feature fusion and feature selection are crucial for medical image segmentation tasks, which is ignored by Transformers. However, it is challenging to directly apply self-attention to CNNs due to the quadratic computational complexity for high-resolution feature maps. Therefore, to integrate the merits of CNNs, multi-scale channel attention and Transformers, we propose an efficient hierarchical hybrid vision Transformer (H2Former) for medical image segmentation. With these merits, the model can be data-efficient for limited medical data regime. The experimental results show that our approach exceeds previous Transformer, CNNs and hybrid methods on three 2D and two 3D medical image segmentation tasks. Moreover, it keeps computational efficiency in model parameters, FLOPs and inference time. For example, H2Former outperforms TransUNet by 2.29% in IoU score on KVASIR-SEG dataset with 30.77% parameters and 59.23% FLOPs.

Multiscale progressive text prompt network for medical image segmentation

The accurate segmentation of medical images is a crucial step in obtaining reliable morphological statistics. However, training a deep neural network for this task requires a large amount of labeled data to ensure high-accuracy results. To address this issue, we propose using progressive text prompts as prior knowledge to guide the segmentation process. Our model consists of two stages. In the first stage, we perform contrastive learning on natural images to pretrain a powerful prior prompt encoder (PPE). This PPE leverages text prior prompts to generate multimodality features. In the second stage, medical image and text prior prompts are sent into the PPE inherited from the first stage to achieve the downstream medical image segmentation task. A multiscale feature fusion block (MSFF) combines the features from the PPE to produce multiscale multimodality features. These two progressive features not only bridge the semantic gap but also improve prediction accuracy. Finally, an UpAttention block refines the predicted results by merging the image and text features. This design provides a simple and accurate way to leverage multiscale progressive text prior prompts for medical image segmentation. Compared with using only images, our model achieves high-quality results with low data annotation costs. Moreover, our model not only has excellent reliability and validity on medical images but also performs well on natural images. The experimental results on different image datasets demonstrate that our model is effective and robust for image segmentation.