NIH Pancreas Research Articles

Triple-task mutual consistency for semi-supervised 3D medical image segmentation

Semi-supervised deep learning algorithm is an effective means of medical image segmentation. Among these methods, multi-task learning with consistency regularization has achieved outstanding results. However, most of the existing methods usually simply embed the Signed Distance Map (SDM) task into the network, which underestimates the potential ability of SDM in edge awareness and leads to excessive dependence between tasks. In this work, we propose a novel triple-task mutual consistency (TTMC) framework to enhance shape and edge awareness capabilities, and overcome the task dependence problem underestimated in previous work. Specifically, we innovatively construct the Signed Attention Map (SAM), a novel fusion image with attention mechanism, and use it as an auxiliary task for segmentation to enhance the edge awareness ability. Then we implement a triple-task deep network, which jointly predicts the voxel-wise classification map, the Signed Distance Map and the Signed Attention Map. In our proposed framework, an optimized differentiable transformation layer associates SDM with voxel-wise classification map and SAM prediction, while task-level consistency regularization utilizes unlabeled data in an unsupervised manner. Evaluated on the public Left Atrium dataset and NIH Pancreas dataset, our proposed framework achieves significant performance gains by effectively utilizing unlabeled data, outperforming recent state-of-the-art semi-supervised segmentation methods. Code is available at https://github.com/Saocent/TTMC.

Strongly representative semantic-guided segmentation network for pancreatic and pancreatic tumors

Accurate and reliable segmentation of the pancreas and its lesions on computed tomography (CT) images is crucial in medical imaging for preoperative diagnosis, surgical planning, and postoperative monitoring. However, there are limited studies that address simultaneous segmentation of the pancreas and pancreatic tumors. Moreover, existing studies have not fully utilized the feature potential of the original images and have neglected the exploration of semantic information with strong representation. To overcome these limitations, we propose the Strongly Representative Semantic-guided Segmentation Network (SRSNet). Specifically, we employ intermediate semantic information to generate strongly representative high-resolution pre-segmented images, effectively reducing channel redundancy across different resolutions. We utilize various mechanisms to extract distinct representative features, and with the guidance of these features, SRSNet effectively supplements high-resolution detailed information for features of different resolutions, provides auxiliary features for the pixel decision phase of the network, and detects large-scale changes in the pancreas and pancreatic tumors. Additionally, we design a loss function that enhances SRSNet’s sensitivity to boundary pixels and attenuates the effect of class imbalance. Our method is evaluated on Task07 Pancreas and NIH Pancreas datasets. In the experiment of combined pancreas and tumor segmentation in the MSD dataset, we achieved Dice, Recall, Precision, and MIoU scores of 78.60%, 79.64%, 81.72%, and 71.47%, respectively. Extensive experiments demonstrate that our algorithm not only outperforms state-of-the-art algorithms for pancreas segmentation but also exhibits excellent performance for pancreas and pancreatic tumor segmentation.

Incorporating multi-stage spatial visual cues and active localization offset for pancreas segmentation

Accurately segmenting pancreas or pancreatic tumor from limited computed tomography (CT) scans plays an essential role in making a precise diagnosis and planning the surgical procedure for clinicians. Although deep convolutional neural networks (DCNNs) have greatly advanced in automatic organ segmentation, there are still many challenges in solving the pancreas segmentation problem with small region and complex background. Many researchers have developed a coarse-to-fine scheme, which employ prediction from the coarse stage as a smaller input region for the fine stage. Despite this scheme effectiveness, most existing approaches handle two stages individually, and fail to identify the reliability of coarse stage predictions. In this work, we present a novel coarse-to-fine framework based on spatial contextual cues and active localization offset. The novelty lies in carefully designed two modules: Spacial Visual Cues Fusion (SVCF) and Active Localization OffseT (ALOT). The SVCF combines the correlations between all pixels in an image to optimize the rough and uncertain pixel prediction at the coarse stage, while ALOT dynamically adjusts the localization as the coarse stage iteration. These two modules work together to optimize the coarse stage results and provide high-quality input for the fine stage, thereby achieving inspiring target segmentation. Empirical results on NIH pancreas segmentation and MSD pancreatic tumor segmentation dataset show that our framework yields state-of-the-art results. The code will make available at https://github.com/PinkGhost0812/SANet.

Extension–contraction transformation network for pancreas segmentation in abdominal CT scans

Accurate and automatic pancreas segmentation from abdominal computed tomography (CT) scans is crucial for the diagnosis and prognosis of pancreatic diseases. However, the pancreas accounts for a relatively small portion of the scan and presents high anatomical variability and low contrast, making traditional automated segmentation methods fail to generate satisfactory results. In this paper, we propose an extension–contraction transformation network (ECTN) and deploy it into a cascaded two-stage segmentation framework for accurate pancreas segmenting. This model can enhance the perception of 3D context by distinguishing and exploiting the extension and contraction transformation of the pancreas between slices. It consists of an encoder, a segmentation decoder, and an extension–contraction (EC) decoder. The EC decoder is responsible for predicting the inter-slice extension and contraction transformation of the pancreas by feeding the extension and contraction information generated by the segmentation decoder; meanwhile, its output is combined with the output of the segmentation decoder to reconstruct and refine the segmentation results. Quantitative evaluation is performed on NIH Pancreas Segmentation (Pancreas-CT) dataset using 4-fold cross-validation. We obtained average Precision of 86.59±6.14% , Recall of 85.11±5.96%, Dice similarity coefficient (DSC) of 85.58±3.98%. and Jaccard Index (JI) of 74.99±5.86%. The performance of our method outperforms several baseline and state-of-the-art methods.

CMFCUNet: cascaded multi-scale feature calibration UNet for pancreas segmentation

Segmenting the pancreas from abdominal CT scans is challenging since it often takes up a relatively small region. Researchers suggested leveraging coarse-to-fine approaches to cope with this challenge. However, the coarse-scaled segmentation and the fine-scaled segmentation are either trained separately utilizing the coordinates located by the coarse-scaled segmentation mask to crop the fine-scaled segmentation input, or trained jointly utilizing the coarse-scaled segmentation mask to enhance the fine-scaled segmentation input. We argued that these two solutions are complementary to some extent and can promote each other to improve the performance of pancreas segmentation. In addition, the backbone in the coarse-scaled segmentation and fine-scaled segmentation is mostly based on UNet or UNet-like networks, where the multi-scale features transmitted from the encoder to the decoder have not been explored for vertical calibration before. In this paper, we propose a cascaded multi-scale feature calibration UNet (CMFCUNet) for pancreas segmentation where the multi-scale features in the backbone of each scaled segmentation are calibrated vertically in a pixel-wise fashion. Besides, the coarse-scaled segmentation and the fine-scaled segmentation are connected by leveraging a designed dual enhancement module (DEM). Experiments are first conducted on the public NIH pancreas dataset. First, when leveraging CMFCUNet, our method increased by over 3% on the Jaccard index (JI) and nearly 1% on dice similarity coefficient (DSC) which surpasses all existing pancreas segmentation approaches. In addition, our experiments demonstrate that CMFCUNet improved the coarse-to-fine segmentation framework and outperformed the mainstream coarse-to-fine pancreas segmentation approaches. Furthermore, we also conducted ablation studies to analyze the effectiveness of the backbone (MFCUNet) and the DEM. In addition to the experiments on the NIH dataset, we also experimentally demonstrate the excellent generalization of our method on the MSD pancreas dataset.

SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentation.

Automated segmentation in medical image analysis is a challenging task that requires a large amount of manually labeled data. However, most existing learning-based approaches usually suffer from limited manually annotated medical data, which poses a major practical problem for accurate and robust medical image segmentation. In addition, most existing semi-supervised approaches are usually not robust compared with the supervised counterparts, and also lack explicit modeling of geometric structure and semantic information, both of which limit the segmentation accuracy. In this work, we present SimCVD, a simple contrastive distillation framework that significantly advances state-of-the-art voxel-wise representation learning. We first describe an unsupervised training strategy, which takes two views of an input volume and predicts their signed distance maps of object boundaries in a contrastive objective, with only two independent dropout as mask. This simple approach works surprisingly well, performing on the same level as previous fully supervised methods with much less labeled data. We hypothesize that dropout can be viewed as a minimal form of data augmentation and makes the network robust to representation collapse. Then, we propose to perform structural distillation by distilling pair-wise similarities. We evaluate SimCVD on two popular datasets: the Left Atrial Segmentation Challenge (LA) and the NIH pancreas CT dataset. The results on the LA dataset demonstrate that, in two types of labeled ratios (i.e., 20% and 10%), SimCVD achieves an average Dice score of 90.85% and 89.03% respectively, a 0.91% and 2.22% improvement compared to previous best results. Our method can be trained in an end-to-end fashion, showing the promise of utilizing SimCVD as a general framework for downstream tasks, such as medical image synthesis, enhancement, and registration.

Learning a Discriminative Feature Attention Network for pancreas CT segmentation

Accurate pancreas segmentation is critical for the diagnosis and management of diseases of the pancreas. It is challenging to precisely delineate pancreas due to the highly variations in volume, shape and location. In recent years, coarse-to-fine methods have been widely used to alleviate class imbalance issue and improve pancreas segmentation accuracy. However, cascaded methods could be computationally intensive and the refined results are significantly dependent on the performance of its coarse segmentation results. To balance the segmentation accuracy and computational efficiency, we propose a Discriminative Feature Attention Network for pancreas segmentation, to effectively highlight pancreas features and improve segmentation accuracy without explicit pancreas location. The final segmentation is obtained by applying a simple yet effective post-processing step. Two experiments on both public NIH pancreas CT dataset and abdominal BTCV multi-organ dataset are individually conducted to show the effectiveness of our method for 2D pancreas segmentation. We obtained average Dice Similarity Coefficient (DSC) of 82.82±6.09%, average Jaccard Index (JI) of 71.13± 8.30% and average Symmetric Average Surface Distance (ASD) of 1.69 ± 0.83 mm on the NIH dataset. Compared to the existing deep learning-based pancreas segmentation methods, our experimental results achieve the best average DSC and JI value.

Learning multi-level structural information for small organ segmentation

Deep neural networks have achieved great success in medical image segmentation problems such as liver, kidney, the accuracy of which already exceeds the human level. However, small organ segmentation (e.g., pancreas) is still a challenging task. To tackle such problems, extracting and aggregating multi-scale robust features become essentially important. In this paper, we develop a multi-level structural loss by integrating the region, boundary, and pixel-wise information to supervise feature fusion and precise segmentation. The novel pixel-wise term can provide information complementary to the region and boundary loss, which helps to discover more local information from the image. We further develop a multi-branch network with a saliency guidance module to better aggregate the three levels of features. The coarse-to-fine segmentation architecture is adopted to use the prediction on the coarse stage to obtain the bounding box for the fine stage. Comprehensive evaluations are performed on three benchmark datasets, i.e., the NIH pancreas, ISICDM pancreas, and MSD spleen dataset, showing that our models can achieve significant increases in segmentation accuracy compared to several state-of-the-art pancreas and spleen segmentation methods. Furthermore, the ablation study demonstrates the multi-level structural features help both the training stability and the convergence of the coarse-to-fine approach.

Parallel Multi‐Scale Network with Attention Mechanism for Pancreas Segmentation

In this paper, we address the task of segmenting small organs (i.e., the pancreas) from abdominal CT scans. As the target often occupies a relatively small region in the input image, deep neural networks can be easily confused by complex and variable backgrounds. We propose a method that uses a parallel multi‐scale network with an attention mechanism for pancreas segmentation, which can better grasp the balance between the semantic segmentation, classification, and localization tasks. We use a parallel network to connect the feature maps between different bottleneck layers, which contain rich semantic information and complete spatial information. We apply an attention module to enhance the key features of semantic information. Then, we fuse the two modules and apply the fused module as attention information on the feature map to ensure the full fusion between contextual semantic information and spatial information, thereby improving segmentation accuracy. We conduct extensive experiments on the NIH pancreas segmentation data set. In particular, our model achieves a mean coefficient Dice of 86.6. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

Multi-scale U-like network with attention mechanism for automatic pancreas segmentation.

In recent years, the rapid development of deep neural networks has made great progress in automatic organ segmentation from abdominal CT scans. However, automatic segmentation for small organs (e.g., the pancreas) is still a challenging task. As an inconspicuous and small organ in the abdomen, the pancreas has a high degree of anatomical variability and is indistinguishable from the surrounding organs and tissues, which usually leads to a very vague boundary. Therefore, the accuracy of pancreatic segmentation is sometimes below satisfaction. In this paper, we propose a 2.5D U-net with an attention mechanism. The proposed network includes 2D convolutional layers and 3D convolutional layers, which means that it requires less computational resources than 3D segmentation models while it can capture more spatial information along the third dimension than 2D segmentation models. Then We use a cascaded framework to increase the accuracy of segmentation results. We evaluate our network on the NIH pancreas dataset and measure the segmentation accuracy by the Dice similarity coefficient (DSC). Experimental results demonstrate a better performance compared with state-of-the-art methods.

Automatic Pancreas Segmentation in CT Images With Distance-Based Saliency-Aware DenseASPP Network.

Pancreas identification and segmentation is an essential task in the diagnosis and prognosis of pancreas disease. Although deep neural networks have been widely applied in abdominal organ segmentation, it is still challenging for small organs (e.g. pancreas) that present low contrast, highly flexible anatomical structure and relatively small region. In recent years, coarse-to-fine methods have improved pancreas segmentation accuracy by using coarse predictions in the fine stage, but only object location is utilized and rich image context is neglected. In this paper, we propose a novel distance-based saliency-aware model, namely DSD-ASPP-Net, to fully use coarse segmentation to highlight the pancreas feature and boost accuracy in the fine segmentation stage. Specifically, a DenseASPP (Dense Atrous Spatial Pyramid Pooling) model is trained to learn the pancreas location and probability map, which is then transformed into saliency map through geodesic distance-based saliency transformation. In the fine stage, saliency-aware modules that combine saliency map and image context are introduced into DenseASPP to develop the DSD-ASPP-Net. The architecture of DenseASPP brings multi-scale feature representation and achieves larger receptive field in a denser way, which overcome the difficulties brought by variable object sizes and locations. Our method was evaluated on both public NIH pancreas dataset and local hospital dataset, and achieved an average Dice-Sørensen Coefficient (DSC) value of 85.49±4.77% on the NIH dataset, outperforming former coarse-to-fine methods.

View adaptive learning for pancreas segmentation

Pancreatic segmentation is a fundamental step in computer-aided diagnosis of the pancreatic cancer. Although 3D U-Net has been dominantly used for this task, it still suffers from limited ability to represent the 3D context in volumetric data. In this paper, we propose the view adaptive 3D U-Net (VA-3DUNet) method for pancreas segmentation in contrasted-enhanced abdominal computed tomography (CT) volumes. Adopting the location-to-segmentation strategy, we first train a 3D U-Net for pancreas localization, and then train another 3D U-Net in a view adaptive way to segment the pancreas in the volume of interest (VOI) determined in the localization step. Such view adaptive training enables the 3D U-Net to perceive each volumetric data from the axial, coronal, and sagittal views simultaneously and hence improves its ability to represent the 3D context. We evaluated the proposed VA-3DUNet method against four state-of-the-art methods on the NIH pancreas segmentation dataset and achieved an average Dice similarity coefficient of 86.19%, which is higher than that achieved by those competing methods. Our results demonstrate the effectiveness of the view adaptive training and the satisfactory performance of the proposed VA-3DUNet method in pancreas segmentation.

Inter-Slice Context Residual Learning for 3D Medical Image Segmentation.

Automated and accurate 3D medical image segmentation plays an essential role in assisting medical professionals to evaluate disease progresses and make fast therapeutic schedules. Although deep convolutional neural networks (DCNNs) have widely applied to this task, the accuracy of these models still need to be further improved mainly due to their limited ability to 3D context perception. In this paper, we propose the 3D context residual network (ConResNet) for the accurate segmentation of 3D medical images. This model consists of an encoder, a segmentation decoder, and a context residual decoder. We design the context residual module and use it to bridge both decoders at each scale. Each context residual module contains both context residual mapping and context attention mapping, the formal aims to explicitly learn the inter-slice context information and the latter uses such context as a kind of attention to boost the segmentation accuracy. We evaluated this model on the MICCAI 2018 Brain Tumor Segmentation (BraTS) dataset and NIH Pancreas Segmentation (Pancreas-CT) dataset. Our results not only demonstrate the effectiveness of the proposed 3D context residual learning scheme but also indicate that the proposed ConResNet is more accurate than six top-ranking methods in brain tumor segmentation and seven top-ranking methods in pancreas segmentation.

An Uncertainty-Driven GCN Refinement Strategy for Organ Segmentation

Organ segmentation in CT volumes is an important pre-processing step in many computer assisted intervention and diagnosis methods. In recent years, convolutional neural networks have dominated the state of the art in this task. However, since this problem presents a challenging environment due to high variability in the organ’s shape and similarity between tissues, the generation of false negative and false positive regions in the output segmentation is a common issue. Recent works have shown that the uncertainty analysis of the model can provide us with useful information about potential errors in the segmentation. In this context, we proposed a segmentation refinement method based on uncertainty analysis and graph convolutional networks. We employ the uncertainty levels of the convolutional network in a particular input volume to formulate a semi-supervised graph learning problem that is solved by training a graph convolutional network. To test our method we refine the initial output of a 2D U-Net. We validate our framework with the NIH pancreas dataset and the spleen dataset of the medical segmentation decathlon. We show that our method outperforms the state-of-the-art CRF refinement method by improving the dice score by 1% for the pancreas and 2% for spleen, with respect to the original U-Net’s prediction. Finally, we perform a sensitivity analysis on the parameters of our proposal and discuss the applicability to other CNN architectures, the results, and current limitations of the model for future work in this research direction. For reproducibility purposes, we make our code publicly available at https://github.com/rodsom22/gcn_refinement

Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation.

Although having achieved great success in medical image segmentation, deep learning-based approaches usually require large amounts of well-annotated data, which can be extremely expensive in the field of medical image analysis. Unlabeled data, on the other hand, is much easier to acquire. Semi-supervised learning and unsupervised domain adaptation both take the advantage of unlabeled data, and they are closely related to each other. In this paper, we propose uncertainty-aware multi-view co-training (UMCT), a unified framework that addresses these two tasks for volumetric medical image segmentation. Our framework is capable of efficiently utilizing unlabeled data for better performance. We firstly rotate and permute the 3D volumes into multiple views and train a 3D deep network on each view. We then apply co-training by enforcing multi-view consistency on unlabeled data, where an uncertainty estimation of each view is utilized to achieve accurate labeling. Experiments on the NIH pancreas segmentation dataset and a multi-organ segmentation dataset show state-of-the-art performance of the proposed framework on semi-supervised medical image segmentation. Under unsupervised domain adaptation settings, we validate the effectiveness of this work by adapting our multi-organ segmentation model to two pathological organs from the Medical Segmentation Decathlon Datasets. Additionally, we show that our UMCT-DA model can even effectively handle the challenging situation where labeled source data is inaccessible, demonstrating strong potentials for real-world applications.

Global context and boundary structure-guided network for cross-modal organ segmentation

In multi-modal medical images such as X-ray and CT, automated organ segmentation is often disturbed by the frequent existence of shape variation, intensity non-uniformities and blurs. Introducing supplementary information helps with eliminating the negative effects of these factors. Medical priors are often leveraged to supplement for specific tasks in previous works, but the specificity of priors hinders these methods from generalizing to cross-modal problems. In this paper we propose Global Context and Boundary Structure-guided Network (GCBSN), which utilizes global context and boundary structure to assist cross-modal organ segmentation. Specifically, we innovatively employ the global context from all spatial regions to guide our deformable convolution. Therefore, more suitable receptive fields can be generated for irregular-shaped targets. Also, we extract the global context contained in each class from the coarse segmentation to assist classifying the areas with non-uniform intensity. Moreover, we design a novel loss that weights more on the errors of positions nearby organ boundaries, and this loss function can avoid errors brought by border blurs. The cross-modal performance of GCBSN is evaluated on two datasets of different modals, i.e., X-Ray images and CT slices. On the 3D NIH Pancreas Dataset, GCBSN outperforms the baseline by 8.72% in terms of the dice coefficient (DC). On the 2D Japanese Society of Radiological Technology Dataset, the mean DC of GCBSN is 98.07% for lung and 94.91% for heart and it surpasses other state-of-the-art methods.

Deep pancreas segmentation with uncertain regions of shadowed sets

Pancreas segmentation is a challenging task in medical image analysis especially for the patients with pancreatic cancer. First, the images often have poor contrast and blurred boundaries. Second, there exist large variations in gray scale, texture, location, shape and size among pancreas images. It becomes even worse with cases of pancreatic cancer. Besides, as an inevitable phenomenon, some of the slices have disconnected topology in pancreas part. All these problems lead to high segmentation uncertainties and make the results inaccurate. Existing pancreas segmentation methods rarely achieve sufficiently accurate and robust results especially for cancer cases. To tackle these problems, we propose a 2D deep learning-based method which can involve uncertainties in the process of segmentation iteratively. The proposed method describes the uncertain regions of pancreatic MRI images based on shadowed sets theory. The results are further corrected through increasing the weights of uncertain regions in iterative training. We evaluate our approach on a challenging pancreatic cancer MRI images dataset collected from the Changhai Hospital, and also validate our approach on the NIH pancreas segmentation dataset. The experimental results demonstrate that our proposed method outperforms the state-of-the-art methods in terms of the Dice similarity coefficient of 73.88% on cancer MRI dataset and 84.37% on NIH dataset respectively.

Deep multi-scale feature fusion for pancreas segmentation from CT images.

Pancreas segmentation from computed tomography (CT) images is an important step in surgical procedures such as cancer detection and radiation treatment. While manual segmentation is time-consuming and operator-dependent, current computer-assisted segmentation methods are facing challenges posed by varying shapes and sizes. To address these challenges, this paper presents a multi-scale feature fusion (MsFF) model for accurate pancreas segmentation from CT images. The proposed MsFF is built upon the well-recognized encoder-decoder framework. Firstly, in the encoder stage, the squeeze-and-excitation module is incorporated to enhance the learning of features by exploiting channel-wise independence. Secondly, a hierarchical fusion module is introduced to better utilize both low-level and high-level features to retain boundary information and make final predictions. The proposed MsFF is evaluated on the NIH pancreas dataset and outperforms the current state-of-the-art methods, by achieving a mean of 87.26% and 22.67% under the Dice Sorensen Coefficient and Volumetric Overlap Error, respectively. The experimental results confirm that the incorporation of squeeze-and-excitation and hierarchical fusion modules contributes to a net gain in the performance of our proposed MsFF.

Iterative 3D feature enhancement network for pancreas segmentation from CT images

Automatic and accurate pancreas segmentation from 3D computed tomography volumes is a crucial prerequisite for computer-aided diagnosis, intraoperative planning and guidance. However, this is a challenging task because of the high inter-subject variability in the shape and location of the pancreas, as well as the existence of the surrounding organs. In order to address the above challenges, we propose a novel iterative 3D feature enhancement network to segment pancreas accurately. Specifically, the multi-level integrated features and the individual features at different levels can be progressively enhanced in an iterative manner by leveraging the complementary information encoded in different features. Therefore, the non-pancreas information at lower layers can be suppressed, and the fine details of pancreas at higher layers can be increased. In addition, because the pancreas region occupies only a small part of the scan, in order to prevent the final predictions from being biased toward the background class, we design the Dice similarity coefficients loss function in the training phase to mitigate this issue. Meanwhile, deep supervision with auxiliary classifier is incorporated in the intermediate layers at each iteration to guide the back-propagation of gradient flows and boost the discriminative capability at lower layers. Finally, in order to verify the effectiveness of the proposed method, we evaluated our approach on the publicly available NIH pancreas segmentation dataset. Extensive experiments illustrate that the proposed method achieves better performance than the state-of-the-art algorithms, and it can be easily applied to other volumetric image segmentation tasks.