Abdominal Organ Segmentation Research Articles

Partial Label Multi-organ Segmentation based on Local Feature Enhancement.

The automatic segmentation of abdominal organs from CT images is essential for surgical planning of abdominal diseases. However, each medical institution only annotates some organs according to its own clinical practice. This brings the partial annotation problem to multi-center abdominal multi-organ segmentation. To address this issue, we introduce a 3D local feature enhanced multi-head segmentation network for multi-organ segmentation of abdominal regions in multiple partially labeled datasets. More specifically, our proposed architecture consists of two branches, the global branch with 3D Transformer and U-Net fusion named 3D TransUNet as the backbone, and the local 3D U-Net branch that provides additional abdominal organ structure information to the global branch to generate more accurate segmentation results. We evaluate our method on four publicly available CT datasets with four different partial label. Our experiments show that the proposed approach provides better accuracy and robustness, with 93.01% average Dice-score-coefficient (DSC) and 3.489 mm Hausdorff Distance (HD) outperforming three existing state-of-the-art methods.

CI-UNet: Application of Segmentation of Medical Images of the Human Torso

The study of human torso medical image segmentation is significant for computer-aided diagnosis of human examination, disease tracking, and disease prevention and treatment. In this paper, two application tasks are designed for torso medical images: the abdominal multi-organ segmentation task and the spine segmentation task. For this reason, this paper proposes a net-work model CI-UNet improve the accuracy of edge segmentation. CI-UNet is a U-shaped network structure consisting of encoding and decoding networks. Firstly, it replaces UNet’s double convolutional backbone network with a VGG16 network loaded with Transfer Learning. It feeds image information from two adjacent layers in the VGG16 network into the decoding grid via information aggregation blocks. Secondly, Polarized Self-Attention is added at the decoding network and the hopping connection, which allows the network to focus on the compelling features of the image. Finally, the image information is decoded by convolution and Up-sampling several times to obtain the segmentation results. CI-UNet was tested in the abdominal multi-organ segmentation task using the Chaos (Combined CT-MR Healthy Abdominal Organ Segmentation) open challenge dataset and compared with UNet, Attention UNet, PSPNet, DeepLabv3+ prediction networks, and dedicated network for MRI images. The experimental results showed that the average intersegmental union (mIoU) and average pixel accuracy (mPA) of organ segmentation were 82.33% and 90.10%, respectively, higher than the above comparison network. Meanwhile, we used CI-UNet for the spine dataset of the Guizhou branch of Beijing Jishuitan Hospital. The average intersegmental union (mIoU) and average pixel accuracy (mPA) of organ segmentation were 87.97% and 93.48%, respectively, which were approved by the physicians for both tasks.

Robust and efficient abdominal CT segmentation using shape constrained multi-scale attention network

PurposeAlthough many deep learning-based abdominal multi-organ segmentation networks have been proposed, the various intensity distributions and organ shapes of the CT images from multi-center, multi-phase with various diseases introduce new challenges for robust abdominal CT segmentation. To achieve robust and efficient abdominal multi-organ segmentation, a new two-stage method is presented in this study. MethodsA binary segmentation network is used for coarse localization, followed by a multi-scale attention network for the fine segmentation of liver, kidney, spleen, and pancreas. To constrain the organ shapes produced by the fine segmentation network, an additional network is pre-trained to learn the shape features of the organs with serious diseases and then employed to constrain the training of the fine segmentation network. ResultsThe performance of the presented segmentation method was extensively evaluated on the multi-center data set from the Fast and Low GPU Memory Abdominal oRgan sEgmentation (FLARE) challenge, which was held in conjunction with International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2021. Dice Similarity Coefficient (DSC) and Normalized Surface Dice (NSD) were calculated to quantitatively evaluate the segmentation accuracy and efficiency. An average DSC and NSD of 83.7% and 64.4% were achieved, and our method finally won the second place among more than 90 participating teams. ConclusionsThe evaluation results on the public challenge demonstrate that our method shows promising performance in robustness and efficiency, which may promote the clinical application of the automatic abdominal multi-organ segmentation.

A dynamic few-shot learning framework for medical image stream mining based on self-training

Few-shot semantic segmentation (FSS) has been widely used in the field of information medicine and intelligent diagnosis. Due to the high cost of medical data collection and the privacy protection of patients, labeled medical images are difficult to obtain. Compared with other semantic segmentation dataset which can be automatically generated in a large scale, the medical image data tend to be continually generated. Most of the existing FSS techniques require abundant annotated semantic classes for pre-training and cannot deal with its dynamic nature of medical data stream. To deal with this issue, we propose a dynamic few-shot learning framework for medical semantic segmentation, which can fully utilize the features of newly-collected/generated data stream. We introduce a new pseudo-label generation strategy for continuously generating pseudo-labels and avoiding model collapse during self-training. Furthermore, an efficient consistency regularization strategy is proposed to fully utilize the limited data. The proposed framework is iteratively trained on three tasks: abdominal organ segmentation for CT and MRI, and cardiac segmentation for MRI. Experiments results demonstrate significant performance gain on medical data stream mining compared with the baseline method.

Bridging Feature Gaps to Improve Multi-Organ Segmentation on Abdominal Magnetic Resonance Image.

Accurate segmentation of abdominal organs on MRI is crucial for computer-aided surgery and computer-aided diagnosis. Most state-of-the-art methods for MRI segmentation employ an encoder-decoder structure, with skip connections concatenating shallow features from the encoder and deep features from the decoder. In this work, we noticed that simply concatenating shallow and deep features was insufficient for segmentation due to the feature gap between shallow features and deep features. To mitigate this problem, we quantified the feature gap from spatial and semantic aspects and proposed a spatial loss and a semantic loss to bridge the feature gap. The spatial loss enhanced spatial details in deep features, and the semantic loss introduced semantic information into shallow features. The proposed method successfully aggregated the complementary information between shallow and deep features by formulating and bridging the feature gap. Experiments on two abdominal MRI datasets demonstrated the effectiveness of the proposed method, which improved the segmentation performance over a baseline with nearly zero additional parameters. Particularly, the proposed method has advantages for segmenting organs with blurred boundaries or in a small scale, achieving superior performance than state-of-the-art methods.

Automatic abdominal segmentation using novel 3D self-adjustable organ aware deep network in CT images

CT scan is an important reference means of disease diagnosis in practice. Automatic segmentation of organ regions can save a lot of time and labor costs, and allow doctors to produce more intuitive observations of the organization of the human body. However, automatic multi-organ segmentation in CT images remains challenging due to the complicated anatomical structures and low tissue contrast in CT images. Traditional segmentation methods are relatively inefficient for organ segmentation with large abdominal deformation, small volume, and blurry tissue boundaries, and the traditional network architectures are rarely designed to meet the requirements of lightweight and efficient clinical practice. In this paper, we propose a novel segmentation network named Self-Adjustable Organ Attention U-Net (SOA-Net) to overcome these limitations. To be a pragmatic solution for effective segmentation method, the SOA-Net includes multi-branches feature attention (MBFA) module and the feature attention aggregation (FAA) module. These two modules have multiple branches with different kernel sizes to capture different scales feature information based on multiple scales of the target organs. An adjustable attention is used on these branches to generate different sizes of the receptive fields in the fusion layer. On the whole, SOA-Net is a 3D self-adjustable organ aware deep network which can adaptively adjust their attention and receptive field sizes based on multiple scales of the target organs to realize the efficient segmentation of multiple abdominal organs. We evaluate our method on AbdomenCT-1K and AMOS2022 datasets and the final experiments proved that our model achieves the best segmentation performance compared with the state-of-the-art segmentation networks. (Our code will be publicly available soon).

The Dresden Surgical Anatomy Dataset for Abdominal Organ Segmentation in Surgical Data Science

Laparoscopy is an imaging technique that enables minimally-invasive procedures in various medical disciplines including abdominal surgery, gynaecology and urology. To date, publicly available laparoscopic image datasets are mostly limited to general classifications of data, semantic segmentations of surgical instruments and low-volume weak annotations of specific abdominal organs. The Dresden Surgical Anatomy Dataset provides semantic segmentations of eight abdominal organs (colon, liver, pancreas, small intestine, spleen, stomach, ureter, vesicular glands), the abdominal wall and two vessel structures (inferior mesenteric artery, intestinal veins) in laparoscopic view. In total, this dataset comprises 13195 laparoscopic images. For each anatomical structure, we provide over a thousand images with pixel-wise segmentations. Annotations comprise semantic segmentations of single organs and one multi-organ-segmentation dataset including segments for all eleven anatomical structures. Moreover, we provide weak annotations of organ presence for every single image. This dataset markedly expands the horizon for surgical data science applications of computer vision in laparoscopic surgery and could thereby contribute to a reduction of risks and faster translation of Artificial Intelligence into surgical practice.

A Unified Visual Information Preservation Framework for Self-supervised Pre-Training in Medical Image Analysis.

Recent advances in self-supervised learning (SSL) in computer vision are primarily comparative, whose goal is to preserve invariant and discriminative semantics in latent representations by comparing siamese image views. However, the preserved high-level semantics do not contain enough local information, which is vital in medical image analysis (e.g., image-based diagnosis and tumor segmentation). To mitigate the locality problem of comparative SSL, we propose to incorporate the task of pixel restoration for explicitly encoding more pixel-level information into high-level semantics. We also address the preservation of scale information, a powerful tool in aiding image understanding but has not drawn much attention in SSL. The resulting framework can be formulated as a multi-task optimization problem on the feature pyramid. Specifically, we conduct multi-scale pixel restoration and siamese feature comparison in the pyramid. In addition, we propose non-skip U-Net to build the feature pyramid and develop sub-crop to replace multi-crop in 3D medical imaging. The proposed unified SSL framework (PCRLv2) surpasses its self-supervised counterparts on various tasks, including brain tumor segmentation (BraTS 2018), chest pathology identification (ChestX-ray, CheXpert), pulmonary nodule detection (LUNA), and abdominal organ segmentation (LiTS), sometimes outperforming them by large margins with limited annotations. Codes and models are available at https://github.com/RL4M/PCRLv2.

SACNet: Shuffling atrous convolutional U‐Net for medical image segmentation

Medical images exhibit multi-granularity and high obscurity along boundaries. As representative work, the U-Net and its variants exhibit two shortcomings on medical image segmentation: (a) they expand the range of reception fields by applying addition or concatenate operators to features with different reception fields, which disrupts the distribution of the essential feature of objects; (b) they utilize the downsampling or atrous convolution to characterize multi-granular features of objects, which can obtain a large range of reception fields but leads to blur boundaries of objects. A Shuffling Atrous Convolutional U-Net (SACNet) for circumventing those issues is proposed. The significant component of SACNet is the Shuffling Atrous Convolution (SAC) module, which fuses different atrous convolutional layers together by using a shuffle concatenate operation, so that the features from the same channel (which correspond to the same attribute of objects) are merged together. Besides the SAC modules, SACNet utilizes an EP module during the fine and medium levels to enhance the boundaries of objects, and utilizes a Transformer module during the coarse level to capture an overall correlation of pixels. Experiments on three medical image segmentation tasks: abdominal organ, cardiac, and skin lesion segmentation demonstrate that, SACNet outperforms several state-of-the-art methods and facilitates easy transplant to other semantic segmentation tasks.

Automated imaging-based abdominal organ segmentation and quality control in 20,000 participants of the UK Biobank and German National Cohort Studies

Large epidemiological studies such as the UK Biobank (UKBB) or German National Cohort (NAKO) provide unprecedented health-related data of the general population aiming to better understand determinants of health and disease. As part of these studies, Magnetic Resonance Imaging (MRI) is performed in a subset of participants allowing for phenotypical and functional characterization of different organ systems. Due to the large amount of imaging data, automated image analysis is required, which can be performed using deep learning methods, e. g. for automated organ segmentation. In this paper we describe a computational pipeline for automated segmentation of abdominal organs on MRI data from 20,000 participants of UKBB and NAKO and provide results of the quality control process. We found that approx. 90% of data sets showed no relevant segmentation errors while relevant errors occurred in a varying proportion of data sets depending on the organ of interest. Image-derived features based on automated organ segmentations showed relevant deviations of varying degree in the presence of segmentation errors. These results show that large-scale, deep learning-based abdominal organ segmentation on MRI data is feasible with overall high accuracy, but visual quality control remains an important step ensuring the validity of down-stream analyses in large epidemiological imaging studies.

WORD: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from CT image.

Whole abdominal organ segmentation is important in diagnosing abdomen lesions, radiotherapy, and follow-up. However, oncologists' delineating all abdominal organs from 3D volumes is time-consuming and very expensive. Deep learning-based medical image segmentation has shown the potential to reduce manual delineation efforts, but it still requires a large-scale fine annotated dataset for training, and there is a lack of large-scale datasets covering the whole abdomen region with accurate and detailed annotations for the whole abdominal organ segmentation. In this work, we establish a new large-scale Whole abdominal ORgan Dataset (WORD) for algorithm research and clinical application development. This dataset contains 150 abdominal CT volumes (30495 slices). Each volume has 16 organs with fine pixel-level annotations and scribble-based sparse annotations, which may be the largest dataset with whole abdominal organ annotation. Several state-of-the-art segmentation methods are evaluated on this dataset. And we also invited three experienced oncologists to revise the model predictions to measure the gap between the deep learning method and oncologists. Afterwards, we investigate the inference-efficient learning on the WORD, as the high-resolution image requires large GPU memory and a long inference time in the test stage. We further evaluate the scribble-based annotation-efficient learning on this dataset, as the pixel-wise manual annotation is time-consuming and expensive. The work provided a new benchmark for the abdominal multi-organ segmentation task, and these experiments can serve as the baseline for future research and clinical application development.

Fast and Low-GPU-memory abdomen CT organ segmentation: The FLARE challenge.

Automatic segmentation of abdominal organs in CT scans plays an important role in clinical practice. However, most existing benchmarks and datasets only focus on segmentation accuracy, while the model efficiency and its accuracy on the testing cases from different medical centers have not been evaluated. To comprehensively benchmark abdominal organ segmentation methods, we organized the first Fast and Low GPU memory Abdominal oRgan sEgmentation (FLARE) challenge, where the segmentation methods were encouraged to achieve high accuracy on the testing cases from different medical centers, fast inference speed, and low GPU memory consumption, simultaneously. The winning method surpassed the existing state-of-the-art method, achieving a 19× faster inference speed and reducing the GPU memory consumption by 60% with comparable accuracy. We provide a summary of the top methods, make their code and Docker containers publicly available, and give practical suggestions on building accurate and efficient abdominal organ segmentation models. The FLARE challenge remains open for future submissions through a live platform for benchmarking further methodology developments at https://flare.grand-challenge.org/.

AbdomenCT-1K: Is Abdominal Organ Segmentation a Solved Problem?

With the unprecedented developments in deep learning, automatic segmentation of main abdominal organs seems to be a solved problem as state-of-the-art (SOTA) methods have achieved comparable results with inter-rater variability on many benchmark datasets. However, most of the existing abdominal datasets only contain single-center, single-phase, single-vendor, or single-disease cases, and it is unclear whether the excellent performance can generalize on diverse datasets. This paper presents a large and diverse abdominal CT organ segmentation dataset, termed AbdomenCT-1K, with more than 1000 (1K) CT scans from 12 medical centers, including multi-phase, multi-vendor, and multi-disease cases. Furthermore, we conduct a large-scale study for liver, kidney, spleen, and pancreas segmentation and reveal the unsolved segmentation problems of the SOTA methods, such as the limited generalization ability on distinct medical centers, phases, and unseen diseases. To advance the unsolved problems, we further build four organ segmentation benchmarks for fully supervised, semi-supervised, weakly supervised, and continual learning, which are currently challenging and active research topics. Accordingly, we develop a simple and effective method for each benchmark, which can be used as out-of-the-box methods and strong baselines. We believe the AbdomenCT-1K dataset will promote future in-depth research towards clinical applicable abdominal organ segmentation methods.

AbdomenNet: deep neural network for abdominal organ segmentation in epidemiologic imaging studies

BackgroundWhole-body imaging has recently been added to large-scale epidemiological studies providing novel opportunities for investigating abdominal organs. However, the segmentation of these organs is required beforehand, which is time consuming, particularly on such a large scale.MethodsWe introduce AbdomentNet, a deep neural network for the automated segmentation of abdominal organs on two-point Dixon MRI scans. A pre-processing pipeline enables to process MRI scans from different imaging studies, namely the German National Cohort, UK Biobank, and Kohorte im Raum Augsburg. We chose a total of 61 MRI scans across the three studies for training an ensemble of segmentation networks, which segment eight abdominal organs. Our network presents a novel combination of octave convolutions and squeeze and excitation layers, as well as training with stochastic weight averaging.ResultsOur experiments demonstrate that it is beneficial to combine data from different imaging studies to train deep neural networks in contrast to training separate networks. Combining the water and opposed-phase contrasts of the Dixon sequence as input channels, yields the highest segmentation accuracy, compared to single contrast inputs. The mean Dice similarity coefficient is above 0.9 for larger organs liver, spleen, and kidneys, and 0.71 and 0.74 for gallbladder and pancreas, respectively.ConclusionsOur fully automated pipeline provides high-quality segmentations of abdominal organs across population studies. In contrast, a network that is only trained on a single dataset does not generalize well to other datasets.

Anomaly detection-inspired few-shot medical image segmentation through self-supervision with supervoxels.

Recent work has shown that label-efficient few-shot learning through self-supervision can achieve promising medical image segmentation results. However, few-shot segmentation models typically rely on prototype representations of the semantic classes, resulting in a loss of local information that can degrade performance. This is particularly problematic for the typically large and highly heterogeneous background class in medical image segmentation problems. Previous works have attempted to address this issue by learning additional prototypes for each class, but since the prototypes are based on a limited number of slices, we argue that this ad-hoc solution is insufficient to capture the background properties. Motivated by this, and the observation that the foreground class (e.g., one organ) is relatively homogeneous, we propose a novel anomaly detection-inspired approach to few-shot medical image segmentation in which we refrain from modeling the background explicitly. Instead, we rely solely on a single foreground prototype to compute anomaly scores for all query pixels. The segmentation is then performed by thresholding these anomaly scores using a learned threshold. Assisted by a novel self-supervision task that exploits the 3D structure of medical images through supervoxels, our proposed anomaly detection-inspired few-shot medical image segmentation model outperforms previous state-of-the-art approaches on two representative MRI datasets for the tasks of abdominal organ segmentation and cardiac segmentation.

Accelerating 2D Abdominal Organ Segmentation with Active Learning.

Abdominal computed tomography CT imaging enables assessment of body habitus and organ health. Quantification of these health factors necessitates semantic segmentation of key structures. Deep learning efforts have shown remarkable success in automating segmentation of abdominal CT, but these methods largely rely on 3D volumes. Current approaches are not applicable when single slice imaging is used to minimize radiation dose. For 2D abdominal organ segmentation, lack of 3D context and variety in acquired image levels are major challenges. Deep learning approaches for 2D abdominal organ segmentation benefit by adding more images with manual annotation, but annotation is resource intensive to acquire given the large quantity and the requirement of expertise. Herein, we designed a gradient based active learning annotation framework by meta-parameterizing and optimizing the exemplars to dynamically select the 'hard cases' to achieve better results with fewer annotated slices to reduce the annotation effort. With the Baltimore Longitudinal Study on Aging (BLSA) cohort, we evaluated the performance with starting from 286 subjects and added 50 more subjects iteratively to 586 subjects in total. We compared the amount of data required to add to achieve the same Dice score between using our proposed method and the random selection in terms of Dice. When achieving 0.97 of the maximum Dice, the random selection needed 4.4 times more data compared with our active learning framework. The proposed framework maximizes the efficacy of manual efforts and accelerates learning.

Liver, kidney and spleen segmentation from CT scans and MRI with deep learning: A survey

Deep Learning approaches for automatic segmentation of organs from CT scans and MRI are providing promising results, leading towards a revolution in the radiologists’ workflow. Precise delineations of abdominal organs boundaries reveal fundamental for a variety of purposes: surgical planning, volumetric estimation (e.g. Total Kidney Volume – TKV – assessment in Autosomal Dominant Polycystic Kidney Disease – ADPKD), diagnosis and monitoring of pathologies. Fundamental imaging techniques exploited for these tasks are Computed Tomography (CT) and Magnetic Resonance Imaging (MRI), which enable clinicians to perform 3D analyses of all Regions of Interests (ROIs). In the realm of existing methods for segmentation and classification of these zones, Convolutional Neural Networks (CNNs) are emerging as the reference approach. In the last five years an enormous research effort has been done about the possibility of applying CNNs in Medical Imaging, resulting in more than 8000 documents on Scopus and more than 80000 results on Google Scholar. The high accuracy provided by those systems cannot be denied as motivation of all obtained results, though there are still problems to be addressed with. In this survey, major article databases, as Scopus, for instance, were systematically investigated for different kinds of Deep Learning approaches in segmentation of abdominal organs with a particular focus on liver, kidney and spleen. In this work, approaches are accurately classified, both by relevance of each organ (for instance, segmentation of liver has specific properties, if compared to other organs) and by type of computational approach, as well as the architecture of the employed network. For this purpose, a case study of segmentation for each of these organs is presented.

Disentangled representation and cross-modality image translation based unsupervised domain adaptation method for abdominal organ segmentation.

Existing medical image segmentation models tend to achieve satisfactory performance when the training and test data are drawn from the same distribution, while they often produce significant performance degradation when used for the evaluation of cross-modality data. To facilitate the deployment of deep learning models in real-world medical scenarios and to mitigate the performance degradation caused by domain shift, we propose an unsupervised cross-modality segmentation framework based on representation disentanglement and image-to-image translation. Our approach is based on a multimodal image translation framework, which assumes that the latent space of images can be decomposed into a content space and a style space. First, image representations are decomposed into the content and style codes by the encoders and recombined to generate cross-modality images. Second, we propose content and style reconstruction losses to preserve consistent semantic information from original images and construct content discriminators to match the content distributions between source and target domains. Synthetic images with target domain style and source domain anatomical structures are then utilized for training of the segmentation model. We applied our framework to the bidirectional adaptation experiments on MRI and CT images of abdominal organs. Compared to the case without adaptation, the Dice similarity coefficient (DSC) increased by almost 30 and 25% and average symmetric surface distance (ASSD) dropped by 13.3 and 12.2, respectively. The proposed unsupervised domain adaptation framework can effectively improve the performance of cross-modality segmentation, and minimize the negative impact of domain shift. Furthermore, the translated image retains semantic information and anatomical structure. Our method significantly outperforms several competing methods.

Supervised and Semi-supervised Methods for Abdominal Organ Segmentation: A Review

Supervised and Semi-supervised Methods for Abdominal Organ Segmentation: A Review

Abdominal multi-organ segmentation with cascaded convolutional and adversarial deep networks

Abdominal anatomy segmentation is crucial for numerous applications from computer-assisted diagnosis to image-guided surgery. In this context, we address fully-automated multi-organ segmentation from abdominal CT and MR images using deep learning. The proposed model extends standard conditional generative adversarial networks. Additionally to the discriminator which enforces the model to create realistic organ delineations, it embeds cascaded partially pre-trained convolutional encoder-decoders as generator. Encoder fine-tuning from a large amount of non-medical images alleviates data scarcity limitations. The network is trained end-to-end to benefit from simultaneous multi-level segmentation refinements using auto-context. Employed for healthy liver, kidneys and spleen segmentation, our pipeline provides promising results by outperforming state-of-the-art encoder-decoder schemes. Followed for the Combined Healthy Abdominal Organ Segmentation (CHAOS) challenge organized in conjunction with the IEEE International Symposium on Biomedical Imaging 2019, it gave us the first rank for three competition categories: liver CT, liver MR and multi-organ MR segmentation. Combining cascaded convolutional and adversarial networks strengthens the ability of deep learning pipelines to automatically delineate multiple abdominal organs, with good generalization capability. The comprehensive evaluation provided suggests that better guidance could be achieved to help clinicians in abdominal image interpretation and clinical decision making.