Multi-organ Segmentation Research Articles

NCCT-to-CECT synthesis with contrast-enhanced knowledge and anatomical perception for multi-organ segmentation in non-contrast CT images

Contrast-enhanced computed tomography (CECT) is constantly used for delineating organs-at-risk (OARs) in radiation therapy planning. The delineated OARs are needed to transfer from CECT to non-contrast CT (NCCT) for dose calculation. Yet, the use of iodinated contrast agents (CA) in CECT and the dose calculation errors caused by the spatial misalignment between NCCT and CECT images pose risks of adverse side effects. A promising solution is synthesizing CECT images from NCCT scans, which can improve the visibility of organs and abnormalities for more effective multi-organ segmentation in NCCT images. However, existing methods neglect the difference between tissues induced by CA and lack the ability to synthesize the details of organ edges and blood vessels. To address these issues, we propose a contrast-enhanced knowledge and anatomical perception network (CKAP-Net) for NCCT-to-CECT synthesis. CKAP-Net leverages a contrast-enhanced knowledge learning network to capture both similarities and dissimilarities in domain characteristics attributable to CA. Specifically, a CA-based perceptual loss function is introduced to enhance the synthesis of CA details. Furthermore, we design a multi-scale anatomical perception transformer that utilizes multi-scale anatomical information from NCCT images, enabling the precise synthesis of tissue details. Our CKAP-Net is evaluated on a multi-center abdominal NCCT-CECT dataset, a head an neck NCCT-CECT dataset, and an NCMRI-CEMRI dataset. It achieves a MAE of 25.96 ± 2.64, a SSIM of 0.855 ± 0.017, and a PSNR of 32.60 ± 0.02 for CECT synthesis, and a DSC of 81.21 ± 4.44 for segmentation on the internal dataset. Extensive experiments demonstrate that CKAP-Net outperforms state-of-the-art CA synthesis methods and has better generalizability across different datasets.

Dynamic U-Net for multi-organ nucleus segmentation

Dynamic U-Net for multi-organ nucleus segmentation

Annotation-free multi-organ anomaly detection in abdominal CT using free-text radiology reports: A multi-centre retrospective study

Artificial intelligence (AI) systems designed to detect abnormalities in abdominal computed tomography (CT) could reduce radiologists' workload and improve diagnostic processes. However, development of such models has been hampered by the shortage of large expert-annotated datasets. Here, we used information from free-text radiology reports, rather than manual annotations, to develop a deep-learning-based pipeline for comprehensive detection of abdominal CT abnormalities. In this multicentre retrospective study, we developed a deep-learning-based pipeline to detect abnormalities in the liver, gallbladder, pancreas, spleen, and kidneys. Abdominal CT exams and related free-text reports obtained during routine clinical practice collected from three institutions were used for training and internal testing, while data collected from six institutions were used for external testing. A multi-organ segmentation model and an information extraction schema were used to extract specific organ images and disease information, CT images and radiology reports, respectively, which were used to train a multiple-instance learning model for anomaly detection. Its performance was evaluated using the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, and F1 score against radiologists' ground-truth labels. We trained the model for each organ on images selected from 66,684 exams (39,255 patients) and tested it on 300 (295 patients) and 600 (596 patients) exams for internal and external validation, respectively. In the external test cohort, the overall AUC for detecting organ abnormalities was 0.886. Whereas models trained on human-annotated labels performed better with the same number of exams, those trained on larger datasets with labels auto-extracted via the information extraction schema significantly outperformed human-annotated label-derived models. Using disease information from routine clinical free-text radiology reports allows development of accurate anomaly detection models without requiring manual annotations. This approach is applicable to various anatomical sites and could streamline diagnostic processes. Japan Science and Technology Agency.

Dual convolution-transformer UNet (DCT-UNet) for organs at risk and clinical target volume segmentation in MRI for cervical cancer brachytherapy

Objective. MRI is the standard imaging modality for high-dose-rate brachytherapy of cervical cancer. Precise contouring of organs at risk (OARs) and high-risk clinical target volume (HR-CTV) from MRI is a crucial step for radiotherapy planning and treatment. However, conventional manual contouring has limitations in terms of accuracy as well as procedural time. To overcome these, we propose a deep learning approach to automatically segment OARs (bladder, rectum, and sigmoid colon) and HR-CTV from female pelvic MRI.Approach. In the proposed pipeline, a coarse multi-organ segmentation model first segments all structures, from which a region of interest is computed for each structure. Then, each organ is segmented using an organ-specific fine segmentation model separately trained for each organ. To account for variable sizes of HR-CTV, a size-adaptive multi-model approach was employed. For coarse and fine segmentations, we designed a dual convolution-transformer UNet (DCT-UNet) which uses dual-path encoder consisting of convolution and transformer blocks. To evaluate our model, OAR segmentations were compared to the clinical contours drawn by the attending radiation oncologist. For HR-CTV, four sets of contours (clinical + three additional sets) were obtained to produce a consensus ground truth as well as for inter/intra-observer variability analysis.Main results. DCT-UNet achieved dice similarity coefficient (mean ± SD) of 0.932 ± 0.032 (bladder), 0.786 ± 0.090 (rectum), 0.663 ± 0.180 (sigmoid colon), and 0.741 ± 0.076 (HR-CTV), outperforming other state-of-the-art models. Notably, the size-adaptive multi-model significantly improved HR-CTV segmentation compared to a single-model. Furthermore, significant inter/intra-observer variability was observed, and our model showed comparable performance to all observers. Computation time for the entire pipeline per subject was 12.59 ± 0.79 s, which is significantly shorter than the typical manual contouring time of >15 min.Significance. These experimental results demonstrate that our model has great utility in cervical cancer brachytherapy by enabling fast and accurate automatic segmentation, and has potential in improving consistency in contouring. DCT-UNet source code is available athttps://github.com/JHU-MICA/DCT-UNet.

A systematic literature review on pancreas segmentation from traditional to non-supervised techniques in abdominal medical images

Abdominal organs play a significant role in regulating various functional systems. Any impairment in its functioning can lead to cancerous diseases. Diagnosing these diseases mainly relies on radiologists’ subjective assessment, which varies according to professional abilities and clinical experience. Computer-Aided Diagnosis (CAD) system is designed to assist clinicians in identifying various pathological changes. Hence, automatic pancreas segmentation is a vital input to the CAD system in the diagnosis of cancer at its early stages. Automatic segmentation is achieved through traditional methods like atlas-based and statistical models, and nowadays, it is achieved through artificial intelligence approaches like machine learning and deep learning using various imaging modalities. This study investigates and analyses the various state-of-the-art multi-organ and pancreas segmentation approaches to identify the research gaps and future perspectives for the research community. The objective is achieved by framing the research questions using the PICOC framework and then selecting 140 research articles using a systematic process through the Covidence tool to conclude the answers to the respective questions. The literature search has been conducted on five databases of original studies published from 2003 to 2023. Initially, the literature analysis is presented in terms of publication, and the comparative analysis of the current study is presented with existing review studies. Then, existing studies are analyzed, focusing on semi-automatic and automatic multi-organ segmentation and pancreas segmentation, using various learning methods. Finally, the various critical issues, the research gaps and the future perspectives of segmentation methods based on published evidence are summarized.

Automated segmentation pipeline for segmentation and characterising metabolic dysregulation in cancer cachexia

Cancer cachexia disrupts metabolic homeostasis across various organs. Positron Emission Tomography (PET) is valuable for investigating cachexia in preclinical models and patients[1]. However, manually segmenting multiple organs is laborious and hinders quantitative analysis of whole-body PET. This study establishes an automated pipeline for segmenting and characterizing organs in preclinical PET/MRI. Acute metabolic changes were induced in non-tumour bearing mice through three approaches: a single dose of Growth Differentiation Factor-15 (GDF-15; n=3 treated, control), daily dexamethasone for 24 days (n=4 treated, control), and 20 hour fasting (n=2 fasted, n=2 fed). [18F]flurodeoxy-glucose PET was used to image whole-body glucose utilisation. To overcome manual segmentation limitations, an automated multi-organ segmentation method was developed for preclinical MRI using nnU-net, an artificial intelligence architecture[2,3]. The nnU-net was trained on 18 preclinical MRI scans with 21 manual tissue annotations. Evaluation on 8 additional mice revealed high accuracy (Dice Coefficients: 0.72-0.91 for 17 tissues). Adipose tissues had lower accuracy but retained good specificity. Due to low accuracy, pancreas segmentation was excluded. All mice used for training and testing the nnU-net were imaged with the same protocol, sourced from the described datasets, with additional scans from different time points. After segmenting the scans, we measured mean Standardized Uptake Values (quantitative measure of metabolic activity) for each tissue. Significant differences (p<0.05) were observed following the various interventions (GDF-15, dexamethasone, fasting) compared to control. Network analysis provided insights into potential inter-organ interactions, while Linear Discriminant Analysis successfully distinguished between each metabolic state. This study presents a novel method for automated segmentation and analysis within preclinical PET/MRI. This method significantly improves efficiency and provides new analysis of total-body metabolic phenotypes. We are currently applying this approach to explore metabolic changes associated with cancer cachexia, aiming to improve our understanding of systemic metabolic dysregulation in both preclinical models and clinical settings. Please click on the 'PDF' for the full abstract!

UNet-based multi-organ segmentation in photon counting CT using virtual monoenergetic images.

Multi-organ segmentation aids in disease diagnosis, treatment, and radiotherapy. The recently emerged photon counting detector-based CT (PCCT) provides spectral information of the organs and the background tissue and may improve segmentationperformance. We propose UNet-based multi-organ segmentation in PCCT using virtual monoenergetic images (VMI) to exploit spectral informationeffectively. The proposed method consists of the following steps: Noise reduction in bin-wise images, image-based material decomposition, generating VMIs, and deep learning-based segmentation. VMIs are synthesized for various x-ray energies using basis images. The UNet-based networks (3D UNet, Swin UNETR) were used for segmentation, and dice similarity coefficients (DSC) and 3D visualization of the segmented result were evaluation indicators. We validated the proposed method for the liver, pancreas, and spleen segmentation using abdominal phantoms from 55 subjects for dual- and quad-energy bins. We compared it to the conventional PCCT-based segmentation, which uses only the (noise-reduced) bin-wise images. The experiments were conducted on two cases by adjusting the doselevels. The proposed method improved the training stability for most cases. With the proposed method, the average DSC for the three organs slightly increased from 0.933 to 0.95, and the standard deviation decreased from 0.066 to 0.047, for example, in the low dose case (using VMIs v.s. bin-wise images from dual-energy bins; 3D UNet). The proposed method using VMIs improves training stability for multi-organ segmentation in PCCT, particularly when the number of energy bins issmall.

Multi-Organ Foundation Model for Universal Ultrasound Image Segmentation with Task Prompt and Anatomical Prior.

Semantic segmentation of ultrasound (US) images with deep learning has played a crucial role in computer-aided disease screening, diagnosis and prognosis. However, due to the scarcity of US images and small field of view, resulting segmentation models are tailored for a specific single organ and may lack robustness, overlooking correlations among anatomical structures of multiple organs. To address these challenges, we propose the Multi-Organ FOundation (MOFO) model for universal US image segmentation. The MOFO is optimized jointly from multiple organs across various anatomical regions to overcome the data scarcity and explore correlations between multiple organs. The MOFO extracts organ-invariant representations from US images. Simultaneously, the task prompt is employed to refine organ-specific representations for segmentation predictions. Moreover, the anatomical prior is incorporated to enhance the consistency of the anatomical structures. A multi-organ US database, comprising 7039 images from 10 organs across various regions of the human body, has been established to evaluate our model. Results demonstrate that the MOFO outperforms single-organ methods in terms of the Dice coefficient, 95% Hausdorff distance and average symmetric surface distance with statistically sufficient margins. Our experiments in multi-organ universal segmentation for US images serve as a pioneering exploration of improving segmentation performance by leveraging semantic and anatomical relationships within US images of multiple organs.

A Two-Phase Methodology leveraging X-DenseNet for Multi-Organ Segmentation in Abdominal CT Scans

Introduction: Multi organ segmentation (MOS) is crucial for various medical applications such as disease diagnosis, treatment planning, and surgery. Manual segmentation of CT organs is inefficient, laborious and time-consuming, necessitating the development of automated techniques. To address these challenges, automatic techniques based on deep learning models are explored in this field. These techniques require improvements to deal with patient-to-patient variability in organ size, location, and shape. Despite the transformative potential of deep learning techniques, research studies often inadequately address low contrast and overlapping structures within CT scans. This paper proposes a novel two phase methodology to integrate deep learning models and image enhancement strategies. Methods: Our framework contributes in two stages named (a) Optimized Contrast Limited Adaptive Histogram Equalization-Weighted Grey Wolf Optimization (optiCLAHE-Weighted GWO), where the organs contrast is enhanced by employing weighted GWO for CLAHE clip limit selection (b) X-DenseNet architecture, where segmented regions of interest are produced with X-DenseNet model from the enhanced CT. Results: For validation of proposed multi organ segmentation (PMOS) technique, experimentation and comparative analysis with existing models has been conducted on FLARE 22 challenge dataset. The Dice Score (DSC) of liver, aorta and spleen for proposed Multi organ segmentation (PMOS) approach is 90.4%, 99.38% and 98.33% respectively. The mean of Precision (Pre), Accuracy (Acc), F-score and DSC of the proposed approach are 95.05%, 95.55%, 95.29% and 96.06% respectively.

Head and neck automatic multi-organ segmentation on Dual-Energy Computed Tomography

Background and purposeDeep-learning-based automatic segmentation is widely used in radiation oncology to delineate organs-at-risk. Dual-energy CT (DECT) allows the reconstruction of enhanced contrast images that could help with manual and auto-delineation. This paper presents a performance evaluation of a commercial auto-segmentation software on image series generated by a DECT. Material and methodsDifferent types of DECT images from seventy four head-and-neck (HN) patients were retrieved, including polyenergetic images at different voltages [80 kV reconstructed with a kernel corresponding to the commercial algorithm DirectDensity™ (PEI80-DD), 80 kV (PEI80), 120 kV-mixed (PEI120)] and a virtual-monoenergetic image at 40 keV (VMI40). Delineations used for treatment planning were considered as ground truth (GT) and were compared with the auto-segmentations performed on the 4 DECT images. A blinded qualitative evaluation of 3 structures (thyroid, left parotid, left nodes level II) was carried out. Performance metrics were calculated for thirteen HN structures to evaluate the auto-contours including dice similarity coefficient (DSC), 95th percentile Hausdorff distance (95HD) and mean surface distance (MSD). ResultsWe observed a high rate of low scores for PEI80-DD and VMI40 auto-segmentations on the thyroid and for GT and VMI40 contours on the nodes level II. All images received excellent scores for the parotid glands. The metrics comparison between GT and auto-segmented contours revealed that PEI80-DD had the highest DSC scores, significantly outperforming other reconstructed images for all organs (p < 0.05). ConclusionsThe results indicate that the auto-contouring system cannot generalize to images derived from DECT acquisition. It is therefore crucial to identify which organs benefit from these acquisitions to adapt the training datasets accordingly.

LIT-Unet: a lightweight and effective model for medical image segmentation.

This study aimed to design a simple and efficient automatic segmentation model for medical images, so as to facilitate doctors to make more accurate diagnosis and treatment plan. A hybrid lightweight network LIT-Unet with symmetric encoder-decoder U-shaped architecture is proposed. Synapse multi-organ segmentation dataset and automated cardiac diagnosis challenge (ACDC) dataset were used to test the segmentation performance of the method. Two indexes, Dice similarity coefficient (DSC ↑) and 95% Hausdorff distance (HD95 ↓), were used to evaluate and compare the segmentation ability with the current advanced methods. Ablation experiments were conducted to demonstrate the lightweight nature and effectiveness of our model. For Synapse dataset, our model achieves a higher DSC score (80.40%), an improvement of 3.8% over the typical hybrid model (TransUnet). The 95 HD value is low at 20.67%. For ACDC dataset, LIT-Unet achieves the optimal average DSC (%) of 91.84 compared with other networks listed. Compared to patch expanding, the DSC of our model is intuitively improved by 1.62% with the help of deformable token merging (DTM). These results show that the proposed hierarchical LIT-Unet can achieve significant accuracy and is expected to provide a reliable basis for clinical diagnosis and treatment.

A 3D U-NET BASED ON EARLY FUSION MODEL: IMPROVEMENT, COMPARATIVE ANALYSIS WITH STATE-OF-THE-ART MODELS AND FINE-TUNING

Multi-organ segmentation is the process of identifying and separating multiple organs in medical images. This segmentation allows for the detection of structural abnormalities by examining the morphological structure of organs. Carrying out the process quickly and precisely has become an important issue in today's conditions. In recent years, researchers have used various technologies for the automatic segmentation of multiple organs. In this study, improvements were made to increase the multi-organ segmentation performance of the 3D U-Net based fusion model combining HSV and grayscale color spaces and compared with state-of-the-art models. Training and testing were performed on the MICCAI 2015 dataset published at Vanderbilt University, which contains 3D abdominal CT images in NIfTI format. The model's performance was evaluated using the Dice similarity coefficient. In the tests, the liver organ showed the highest Dice score. Considering the average Dice score of all organs, and comparing it with other models, it has been observed that the fusion approach model yields promising results.

ASF-LKUNet: Adjacent-scale fusion U-Net with large kernel for multi-organ segmentation

In the multi-organ segmentation task of medical images, there are some challenging issues such as the complex background, blurred boundaries between organs, and the larger scale difference in volume. Due to the local receptive fields of conventional convolution operations, it is difficult to obtain desirable results by directly using them for multi-organ segmentation. While Transformer-based models have global information, there is a significant dependency on hardware because of the high computational demands. Meanwhile, the depthwise convolution with large kernel can capture global information and have less computational requirements. Therefore, to leverage the large receptive field and reduce model complexity, we propose a novel CNN-based approach, namely adjacent-scale fusion U-Net with large kernel (ASF-LKUNet) for multi-organ segmentation. We utilize a u-shaped encoder–decoder as the base architecture of ASF-LKUNet. In the encoder path, we design the large kernel residual block, which combines the large and small kernels and can simultaneously capture the global and local features. Furthermore, for the first time, we propose an adjacent-scale fusion and large kernel GRN channel attention that incorporates the low-level details with the high-level semantics by the adjacent-scale feature and then adaptively focuses on the more global and meaningful channel information. Extensive experiments and interpretability analysis are made on the Synapse multi-organ dataset (Synapse) and the ACDC cardiac multi-structure dataset (ACDC). Our proposed ASF-LKUNet achieves 88.41% and 89.45% DSC scores on the Synapse and ACDC datasets, respectively, with 17.96M parameters and 29.14 GFLOPs. These results show that our method achieves superior performance with favorable lower complexity against ten competing approaches.ASF-LKUNet is superior to various competing methods and has less model complexity. Code and the trained models have been released on GitHub.

Segmentation prompts classification: A nnUNet-based 3D transfer learning framework with ROI tokenization and cross-task attention for esophageal cancer T-stage diagnosis

The computer-aided diagnosis system for esophageal cancer (EC) holds vital significance in EC diagnosis and treatment making, with a primary focus on accurate segmentation of EC-related organs and classification of EC’s T-stage. Above two tasks are closely related and crucial in assisting surgeon segment and diagnose cancer early. Note that this paradigm is still at its infancy and limited by closely related open issues: (1) how to link the complementary relationship between these two tasks and improve the originally poor performance? and (2) how to determine whether the tumor has invaded the surrounding muscle layers from CT images? Aiming at these issues, this study develops nn-TransEC, a 3D transfer learning framework that builds upon nnU-Net and synergizes segmentation and classification. nn-TransEC focuses on prompting fine-grained classification of EC’s T-stage with the aid of prior segmentation, which is implemented in two parts: (1) A nnUNet-configured multi-task learning network (nn-MTNet) is designed for complementary segmentation of EC-related organs and classification of EC’s T-stages with cross-task attention gates and transfer learning. (2) A knowledge-embedded ROI tokenization method (KRT) is defined to mimic the diagnostic workflow of doctors for classifying EC’s T-stage. KRT is implemented by cropping the most concerned regions from entire CT volume based on prior segmentation. Experiments have been conducted on a private dataset collected from 169 patients with confirmed EC through pathological diagnosis. Our proposed nn-TransEC is compared against the state-of-the-art counterparts (e.g., nnU-Net and nnFormer), and results demonstrate that: nn-TransEC excels in all compared methods in multi-organ segmentation and classification of EC’s T-stages, with 3D Dice of EC and average AUC of T-stages reaching 0.844 and 0.941, respectively. In contrast, the state-of-the-art method nnFormer achieves 0.814 and 0.927, respectively. Meanwhile, nn-TransEC also outperforms state-of-the-art multi-task learning models in joint segmentation and classification, with Hausdorff Distance of EC and average precision of T-stages reaching 8.497 and 0.845, respectively. In contrast, the state-of-the-art method TransMT-Net achieves 12.206 and 0.730, respectively.

MA-SAM: Modality-agnostic SAM adaptation for 3D medical image segmentation

The Segment Anything Model (SAM), a foundation model for general image segmentation, has demonstrated impressive zero-shot performance across numerous natural image segmentation tasks. However, SAM’s performance significantly declines when applied to medical images, primarily due to the substantial disparity between natural and medical image domains. To effectively adapt SAM to medical images, it is important to incorporate critical third-dimensional information, i.e., volumetric or temporal knowledge, during fine-tuning. Simultaneously, we aim to harness SAM’s pre-trained weights within its original 2D backbone to the fullest extent. In this paper, we introduce a modality-agnostic SAM adaptation framework, named as MA-SAM, that is applicable to various volumetric and video medical data. Our method roots in the parameter-efficient fine-tuning strategy to update only a small portion of weight increments while preserving the majority of SAM’s pre-trained weights. By injecting a series of 3D adapters into the transformer blocks of the image encoder, our method enables the pre-trained 2D backbone to extract third-dimensional information from input data. We comprehensively evaluate our method on five medical image segmentation tasks, by using 11 public datasets across CT, MRI, and surgical video data. Remarkably, without using any prompt, our method consistently outperforms various state-of-the-art 3D approaches, surpassing nnU-Net by 0.9%, 2.6%, and 9.9% in Dice for CT multi-organ segmentation, MRI prostate segmentation, and surgical scene segmentation respectively. Our model also demonstrates strong generalization, and excels in challenging tumor segmentation when prompts are used. Our code is available at: https://github.com/cchen-cc/MA-SAM.

Semi-supervised abdominal multi-organ segmentation by object-redrawing.

Multi-organ segmentation is a critical task in medical imaging, with wide-ranging applications in both clinical practice and research. Accurate delineation of organs from high-resolution 3D medical images, such as CT scans, is essential for radiation therapy planning, enhancing treatment outcomes, and minimizing radiation toxicity risks. Additionally, it plays a pivotal role in quantitative image analysis, supporting various medical research studies. Despite its significance, manual segmentation of multiple organs from 3D images is labor-intensive and prone to low reproducibility due to high interoperator variability. Recent advancements in deep learning have led to several automated segmentation methods, yet many rely heavily on labeled data and human anatomy expertise. In this study, our primary objective is to address the limitations of existing semi-supervised learning (SSL) methods for abdominal multi-organ segmentation. We aim to introduce a novel SSL approach that leverages unlabeled data to enhance the performance of deep neural networks in segmenting abdominal organs. Specifically, we propose a method that incorporates a redrawing network into the segmentation process to correct errors and improve accuracy. Our proposed method comprises three interconnected neural networks: a segmentation network for image segmentation, a teacher network for consistency regularization, and a redrawing network for object redrawing. During training, the segmentation network undergoes two rounds of optimization: basic training and readjustment. We adopt the Mean-Teacher model as our baseline SSL approach, utilizing labeled and unlabeled data. However, recognizing significant errors in abdominal multi-organ segmentation using this method alone, we introduce the redrawing network to generate redrawn images based on CT scans, preserving original anatomical information. Our approach is grounded in the generative process hypothesis, encompassing segmentation, drawing, and assembling stages. Correct segmentation is crucial for generating accurate images. In the basic training phase, the segmentation network is trained using both labeled and unlabeled data, incorporating consistency learning to ensure consistent predictions before and after perturbations. The readjustment phase focuses on reducing segmentation errors by optimizing the segmentation network parameters based on the differences between redrawn and original CT images. We evaluated our method using two publicly available datasets: the beyond the cranial vault (BTCV) segmentation dataset (training: 44, validation: 6) and the abdominal multi-organ segmentation (AMOS) challenge 2022 dataset (training:138, validation:16). Our results were compared with state-of-the-art SSL methods, including MT and dual-task consistency (DTC), using the Dice similarity coefficient (DSC) as an accuracy metric. On both datasets, our proposed SSL method consistently outperformed other methods, including supervised learning, achieving superior segmentation performance for various abdominal organs. These findings demonstrate the effectiveness of our approach, even with a limited number of labeled data. Our novel semi-supervised learning approach for abdominal multi-organ segmentation addresses the challenges associated with this task. By integrating a redrawing network and leveraging unlabeled data, we achieve remarkable improvements in accuracy. Our method demonstrates superior performance compared to existing SSL and supervised learning methods. This approach holds great promise in enhancing the precision and efficiency of multi-organ segmentation in medical imaging applications.

An efficient segment anything model for the segmentation of medical images

This paper introduces the efficient medical-images-aimed segment anything model (EMedSAM), addressing the high computational demands and limited adaptability of using SAM for medical image segmentation tasks. We present a novel, compact image encoder, DD-TinyViT, designed to enhance segmentation efficiency through an innovative parameter tuning method called med-adapter. The lightweight DD-TinyViT encoder is derived from the well-known ViT-H using a decoupled distillation approach.The segmentation and recognition capabilities of EMedSAM for specific structures are improved by med-adapter, which dynamically adjusts the model parameters specifically for medical imaging. We conducted extensive testing on EMedSAM using the public FLARE 2022 dataset and datasets from the First Hospital of Zhejiang University School of Medicine. The results demonstrate that our model outperforms existing state-of-the-art models in both multi-organ and lung segmentation tasks.

IEA-Net: Internal and External Dual-Attention Medical Segmentation Network with High-Performance Convolutional Blocks.

Currently, deep learning is developing rapidly in the field of image segmentation, and medical image segmentation is one of the key applications in this field. Conventional CNN has achieved great success in general medical image segmentation tasks, but it has feature loss in the feature extraction part and lacks the ability to explicitly model remote dependencies, which makes it difficult to adapt to the task of human organ segmentation. Although methods containing attention mechanisms have made good progress in the field of semantic segmentation, most of the current attention mechanisms are limited to a single sample, while the number of samples of human organ images is large, ignoring the correlation between the samples is not conducive to image segmentation. In order to solve these problems, an internal and external dual-attention segmentation network (IEA-Net) is proposed in this paper, and the ICSwR (interleaved convolutional system with residual) module and the IEAM module are designed in this network. The ICSwR contains interleaved convolution and hopping connection, which are used for the initial extraction of the features in the encoder part. The IEAM module (internal and external dual-attention module) consists of the LGGW-SA (local-global Gaussian-weighted self-attention) module and the EA module, which are in a tandem structure. The LGGW-SA module focuses on learning local-global feature correlations within individual samples for efficient feature extraction. Meanwhile, the EA module is designed to capture inter-sample connections, addressing multi-sample complexities. Additionally, skip connections will be incorporated into each IEAM module within both the encoder and decoder to reduce feature loss. We tested our method on the Synapse multi-organ segmentation dataset and the ACDC cardiac segmentation dataset, and the experimental results show that the proposed method achieves better performance than other state-of-the-art methods.

Dynamic Loss Weighting for Multiorgan Segmentation in Medical Images.

Deep neural networks often suffer from performance inconsistency for multiorgan segmentation in medical images; some organs are segmented far worse than others. The main reason might be organs with different levels of learning difficulty for segmentation mapping, due to variations such as size, texture complexity, shape irregularity, and imaging quality. In this article, we propose a principled class-reweighting algorithm, termed dynamic loss weighting, which dynamically assigns a larger loss weight to organs if they are discriminated as more difficult to learn according to the data and network's status, for forcing the network to learn from them more to maximally promote the performance consistency. This new algorithm uses an extra autoencoder to measure the discrepancy between the segmentation network's output and the ground truth and dynamically estimates the loss weight of organs per the contribution of the organ to the new updated discrepancy. It can capture the variation in organs' learning difficult during training, and it is neither sensitive to data's property nor dependent on human priors. We evaluate this algorithm in two multiorgan segmentation tasks: abdominal organs and head-neck structures, on publicly available datasets, with positive results obtained from extensive experiments which confirm the validity and effectiveness. Source codes are available at: https://github.com/YouyiSong/Dynamic-Loss-Weighting.

TransUNet: Rethinking the U-Net architecture design for medical image segmentation through the lens of transformers

Medical image segmentation is crucial for healthcare, yet convolution-based methods like U-Net face limitations in modeling long-range dependencies. To address this, Transformers designed for sequence-to-sequence predictions have been integrated into medical image segmentation. However, a comprehensive understanding of Transformers’ self-attention in U-Net components is lacking. TransUNet, first introduced in 2021, is widely recognized as one of the first models to integrate Transformer into medical image analysis. In this study, we present the versatile framework of TransUNet that encapsulates Transformers’ self-attention into two key modules: (1) a Transformer encoder tokenizing image patches from a convolution neural network (CNN) feature map, facilitating global context extraction, and (2) a Transformer decoder refining candidate regions through cross-attention between proposals and U-Net features. These modules can be flexibly inserted into the U-Net backbone, resulting in three configurations: Encoder-only, Decoder-only, and Encoder+Decoder. TransUNet provides a library encompassing both 2D and 3D implementations, enabling users to easily tailor the chosen architecture. Our findings highlight the encoder’s efficacy in modeling interactions among multiple abdominal organs and the decoder’s strength in handling small targets like tumors. It excels in diverse medical applications, such as multi-organ segmentation, pancreatic tumor segmentation, and hepatic vessel segmentation. Notably, our TransUNet achieves a significant average Dice improvement of 1.06% and 4.30% for multi-organ segmentation and pancreatic tumor segmentation, respectively, when compared to the highly competitive nn-UNet, and surpasses the top-1 solution in the BrasTS2021 challenge. 2D/3D Code and models are available at https://github.com/Beckschen/TransUNet and https://github.com/Beckschen/TransUNet-3D, respectively.