Liver Segmentation Using Modified CAG-SwinUNet with Explainability

ObjectiveLiver cancer is one of the most commonly diagnosed cancer, and energy-based tumor ablation is a widely accepted treatment. Automatic and robust segmentation of liver tumors and ablation zones would facilitate the evaluation of treatment success. The purpose of this study was to develop and evaluate an automatic deep learning based method for (1) segmentation of liver and liver tumors in both arterial and portal venous phase for pre-treatment CT, and (2) segmentation of liver and ablation zones in both arterial and portal venous phase for after ablation treatment.Materials and Methods252 CT images from 63 patients undergoing liver tumor ablation at a large University Hospital were retrospectively included; each patient had pre-treatment and post-treatment multi-phase CT images. 3D voxel-wise manual segmentation of the liver, tumors and ablation region by the radiologist provided reference standard. Deep learning models for liver and lesion segmentation were initially trained on the public Liver Tumor Segmentation Challenge (LiTS) dataset to obtain base models. Then, transfer learning was applied to adapt the base models on the clinical training-set, to obtain tumor and ablation segmentation models both for arterial and portal venous phase images. For modeling, 2D residual-attention Unet (RA-Unet) was employed for liver segmentation and a multi-scale patch-based 3D RA-Unet for tumor and ablation segmentation.ResultsOn the independent test-set, the proposed method achieved a dice similarity coefficient (DSC) of 0.96 and 0.95 for liver segmentation on arterial and portal venous phase, respectively. For liver tumors, the model on arterial phase achieved detection sensitivity of 71%, DSC of 0.64, and on portal venous phase sensitivity of 82%, DSC of 0.73. For liver tumors >0.5cm3 performance improved to sensitivity 79%, DSC 0.65 on arterial phase and, sensitivity 86%, DSC 0.72 on portal venous phase. For ablation zone, the model on arterial phase achieved detection sensitivity of 90%, DSC of 0.83, and on portal venous phase sensitivity of 90%, DSC of 0.89.ConclusionThe proposed deep learning approach can provide automated segmentation of liver tumors and ablation zones on multi-phase (arterial and portal venous) and multi-time-point (before and after treatment) CT enabling quantitative evaluation of treatment success.

Automatic segmentation of the liver and hepatic lesions from abdominal 3D computed tomography (CT) images is fundamental tasks in computer-assisted liver surgery planning. However, due to complex backgrounds, ambiguous boundaries, heterogeneous appearances and highly varied shapes of the liver, accurate liver segmentation and tumor detection are still challenging problems. To address these difficulties, we propose an automatic segmentation framework based on 3D U-net with dense connections and globally optimized refinement. Firstly, a deep U-net architecture with dense connections is trained to learn the probability map of the liver. Then the probability map goes into the following refinement step as the initial surface and prior shape. The segmentation of liver tumor is based on the similar network architecture with the help of segmentation results of liver. In order to reduce the influence of the surrounding tissues with the similar intensity and texture behavior with the tumor region, during the training procedure, I × liverlabel is the input of the network for the segmentation of liver tumor. By doing this, the accuracy of segmentation can be improved. The proposed method is fully automatic without any user interaction. Both qualitative and quantitative results reveal that the proposed approach is efficient and accurate for liver volume estimation in clinical application. The high correlation between the automatic and manual references shows that the proposed method can be good enough to replace the time-consuming and non-reproducible manual segmentation method.

Segmentation of livers and liver tumors is one of the most important steps in radiation therapy of hepatocellular carcinoma. The segmentation task is often done manually, making it tedious, labor intensive, and subject to intra-/inter- operator variations. While various algorithms for delineating organ-at-risks (OARs) and tumor targets have been proposed, automatic segmentation of livers and liver tumors remains intractable due to their low tissue contrast with respect to the surrounding organs and their deformable shape in CT images. The U-Net has gained increasing popularity recently for image analysis tasks and has shown promising results. Conventional U-Net architectures, however, suffer from three major drawbacks. First, skip connections allow for the duplicated transfer of low resolution information in feature maps to improve efficiency in learning, but this often leads to blurring of extracted image features. Secondly, high level features extracted by the network often do not contain enough high resolution edge information of the input, leading to greater uncertainty where high resolution edge dominantly affects the network's decisions such as liver and liver-tumor segmentation. Thirdly, it is generally difficult to optimize the number of pooling operations in order to extract high level global features, since the number of pooling operations used depends on the object size. To cope with these problems, we added a residual path with deconvolution and activation operations to the skip connection of the U-Net to avoid duplication of low resolution information of features. In the case of small object inputs, features in the skip connection are not incorporated with features in the residual path. Furthermore, the proposed architecture has additional convolution layers in the skip connection in order to extract high level global features of small object inputs as well as high level features of high resolution edge information of large object inputs. Efficacy of the modified U-Net (mU-Net) was demonstrated using the public dataset of Liver tumor segmentation (LiTS) challenge 2017. For liver-tumor segmentation, Dice similarity coefficient (DSC) of 89.72 %, volume of error (VOE) of 21.93 %, and relative volume difference (RVD) of - 0.49 % were obtained. For liver segmentation, DSC of 98.51 %, VOE of 3.07 %, and RVD of 0.26 % were calculated. For the public 3D Image Reconstruction for Comparison of Algorithm Database (3Dircadb), DSCs were 96.01 % for the liver and 68.14 % for liver-tumor segmentations, respectively. The proposed mU-Net outperformed existing state-of-art networks.

The accurate segmentation of liver and liver tumors from CT images can assist radiologists in decision-making and treatment planning. The contours of liver and liver tumors are currently obtained by manual labeling, which is time-consuming and subjective. Computer-aided segmentation methods have been widely used in the segmentation of liver and liver tumors. However, due to the diversity of shape, volume, and image intensity, the segmentation is still a difficult task. In this study, we present a Spatial Feature Fusion Convolutional Network (SFF-Net) to automatically segment liver and liver tumors from CT images. First, we extract side-outputs at each convolutional block in SFF-Net to make full use of multiscale features. Second, skip-connections are added in the down-sampling phase, therefore, the spatial information can be efficiently transferred to later layers. Third, we present feature fusion blocks (FFBs) to merge spatial features and high-level semantic features from early layers and later layers, respectively. Finally, a fully connected 3D conditional random fields (CRFs) is applied to refine the liver and liver tumor segmentation results. We test our method on the MICCAI 2017 Liver Tumor Segmentation (LiTS) challenge dataset. The Dice Global (DG) score, Dice per case (DC) score, Volume Overlap Error (VOE), Average Symmetric Surface Distance (ASSD), and tumor precision score are calculated to evaluate the liver and liver tumor segmentation accuracies. For the liver segmentation, DG is 0.955; DC is 0.937; VOE is 0.106; and ASSD is 3.678. For the tumor segmentation, DG is 0.746; DC is 0.592; VOE is 0.416; ASSD is 1.585 and the tumor precision score is 0.369. The SFF-Net learns more spatial information by adding skip-connections and feature fusion blocks. The experiments validate that our method can accurately segment liver and liver tumors from CT images.

Accurate liver segmentation from computed tomography (CT) scans is essential for liver cancer diagnosis and liver surgery planning. Convolutional neural network (CNN)-based models have limited segmentation performance due to their localized receptive fields. Hybrid models incorporating CNNs and transformers that can capture long-range dependencies have shown promising performance in liver segmentation with the cost of high model complexity. Therefore, a new network architecture named G-UNETR++ is proposed to improve accuracy in liver segmentation with moderate model complexity. Two gradient-based encoders that take the second-order partial derivatives (the first two elements from the last column of the Hessian matrix of a CT scan) as inputs are proposed to learn the 3D geometric features such as the boundaries between different organs and tissues. In addition, a hybrid loss function that combines dice loss, cross-entropy loss, and Hausdorff distance loss is designed to address class imbalance and improve segmentation performance in challenging cases. The proposed method was evaluated on three public datasets, the Liver Tumor Segmentation (LiTS) dataset, the 3D Image Reconstruction for Comparison of Algorithms Database (3D-IRCADb), and the Segmentation of the Liver Competition 2007 (Sliver07) dataset, and achieved 97.38%, 97.50%, and 97.32% in terms of the dice similarity coefficient for liver segmentation on the three datasets, respectively. The proposed method outperformed the other state-of-the-art models on the three datasets, which demonstrated the strong effectiveness, robustness, and generalizability of the proposed method in liver segmentation.

Automated liver tumor segmentation from computed tomography (CT) images is a necessary prerequisite in the interventions of hepatic abnormalities and surgery planning. However, accurate liver tumor segmentation remains challenging due to the large variability of tumor sizes and inhomogeneous texture. Recent advances based on fully convolutional network (FCN) for medical image segmentation drew on the success of learning discriminative pyramid features. In this paper, we propose a decoupled pyramid correlation network (DPC-Net) that exploits attention mechanisms to fully leverage both low- and high-level features embedded in FCN to segment livertumor. We first design a powerful pyramid feature encoder (PFE) to extract multilevel features from input images. Then we decouple the characteristics of features concerning spatial dimension (i.e., height, width, depth) and semantic dimension (i.e., channel). On top of that, we present two types of attention modules, spatial correlation (SpaCor) and semantic correlation (SemCor) modules, to recursively measure the correlation of multilevel features. The former selectively emphasizes global semantic information in low-level features with the guidance of high-level ones. The latter adaptively enhance spatial details in high-level features with the guidance of low-levelones. We evaluate the DPC-Net on MICCAI 2017 LiTS Liver Tumor Segmentation (LiTS) challenge data set. Dice similarity coefficient (DSC) and average symmetric surface distance (ASSD) are employed for evaluation. The proposed method obtains a DSC of 76.4% and an ASSD of 0.838 mm for liver tumor segmentation, outperforming the state-of-the-art methods. It also achieves a competitive result with a DSC of 96.0% and an ASSD of 1.636 mm for liversegmentation. The experimental results show promising performance of DPC-Net for liver and tumor segmentation from CT images. Furthermore, the proposed SemCor and SpaCor can effectively model the multilevel correlation from both semantic and spatial dimensions. The proposed attention modules are lightweight and can be easily extended to other multilevel methods in an end-to-endmanner.

The segmentation of the liver is a difficult process due to the changes in shape, border, and density that occur in each section in computed tomography (CT) images. In this study, the Adding Inception Module-Unet (AIM-Unet) model, which is a hybridization of convolutional neural networks-based Unet and Inception models, is proposed for computer-assisted automatic segmentation of the liver and liver tumors from CT scans of the abdomen. Experimental studies were carried out on four different liver CT image datasets, one of which was prepared for this study and three of which were open (CHAOS, LIST, and 3DIRCADb). The results obtained using the proposed method and the segmentation results marked by the specialist were compared with the Dice similarity coefficient (DSC), Jaccard similarity coefficient (JSC), and accuracy (ACC) measurement parameters. In this study, we obtained the best DSC, JSC, and ACC liver segmentation performance metrics on the CHAOS dataset as 97.86%, 96.10%, and 99.75%, respectively, of the AIM-Unet model we propose, which is trained separately on three datasets (LiST, CHAOS, and our dataset) containing liver images. Additionally, 75.6% and 65.5% of the DSC tumor segmentation metrics were calculated on the proposed model LiST and 3DIRCADb datasets, respectively. In addition, the segmentation success results on the datasets with the AIM-Unet model were compared with the previous studies. With these results, it has been seen that the method proposed in this study can be used as an auxiliary tool in the decision-making processes of physicians for liver segmentation and detection of liver tumors. This study is useful for medical images, and the developed model can be easily developed for applications in different organs and other medical fields.

Introduction: Currently, the incidence of liver cancer is on the rise annually. Precise identification of liver tumors is crucial for clinicians to strategize the treatment and combat liver cancer. Thus far, liver tumor contours have been derived through labor-intensive and subjective manual labeling. Computers have gained widespread application in the realm of liver tumor segmentation. Nonetheless, liver tumor segmentation remains a formidable challenge owing to the diverse range of volumes, shapes, and image intensities encountered. Methods: In this article, we introduce an innovative solution called the attention connect network (AC-Net) designed for automated liver tumor segmentation. Building upon the U-shaped network architecture, our approach incorporates 2 critical attention modules: the axial attention module (AAM) and the vision transformer module (VTM), which replace conventional skip-connections to seamlessly integrate spatial features. The AAM facilitates feature fusion by computing axial attention across feature maps, while the VTM operates on the lowest resolution feature maps, employing multihead self-attention, and reshaping the output into a feature map for subsequent concatenation. Furthermore, we employ a specialized loss function tailored to our approach. Our methodology begins with pretraining AC-Net using the LiTS2017 dataset and subsequently fine-tunes it using computed tomography (CT) and magnetic resonance imaging (MRI) data sourced from Hubei Cancer Hospital. Results: The performance metrics for AC-Net on CT data are as follows: dice similarity coefficient (DSC) of 0.90, Jaccard coefficient (JC) of 0.82, recall of 0.92, average symmetric surface distance (ASSD) of 4.59, Hausdorff distance (HD) of 11.96, and precision of 0.89. For AC-Net on MRI data, the metrics are DSC of 0.80, JC of 0.70, recall of 0.82, ASSD of 7.58, HD of 30.26, and precision of 0.84. Conclusion: The comparative experiments highlight that AC-Net exhibits exceptional tumor recognition accuracy when tested on the Hubei Cancer Hospital dataset, demonstrating highly competitive performance for practical clinical applications. Furthermore, the ablation experiments provide conclusive evidence of the efficacy of each module proposed in this article. For those interested, the code for this research article can be accessed at the following GitHub repository: https://github.com/killian-zero/py_tumor-segmentation.git.

Fast Automated Liver Delineation from Computational Tomography Angiography

PurposeLiver is one of the organs with a high incidence of tumors in the human body. Malignant liver tumors seriously threaten human life and health. The difficulties of liver tumor segmentation from computed tomography (CT) image are: (a) The contrast between the liver tumors and healthy tissues in CT images is low and the boundary is blurred; (b) The image of liver tumor is complex and diversified in size, shape, and location.MethodsTo solve the above problems, this paper focused on the human liver and liver tumor segmentation algorithm based on convolutional neural network (CNN), and specially designed a three‐dimensional dual path multiscale convolutional neural network (TDP‐CNN). To balance the performance of segmentation and requirement of computational resources, the dual path was used in the network, then the feature maps from both paths were fused at the end of the paths. To refine the segmentation results, we used conditional random fields (CRF) to eliminate the false segmentation points in the segmentation results to improve the accuracy.ResultsIn the experiment, we used the public dataset liver tumor segmentation (LiTS) to analyze the segmentation results qualitatively and quantitatively. Ground truth segmentation of liver and liver tumor was manually labeled by an experienced radiologist. Quantitative metrics were Dice, Hausdorff distance, and average distance. For the segmentation results of liver tumor, Dice was 0.689, Hausdorff distance was 7.69, and the average distance was 1.07; for the segmentation results of the liver, Dice was 0.965, Hausdorff distance was 29.162, and the average distance was 0.197. Compared with other liver and liver tumor segmentation algorithms in Medical Image Computing and Intervention (MICCAI) 2017 competition, our method of liver segmentation ranked first, and liver tumor segmentation ranked second.ConclusionsThe experimental results showed that the proposed algorithm had good performance in both liver and liver tumor segmentation.

PurposeAccurate segmentation of liver and liver tumors is critical for radiotherapy. Liver tumor segmentation, however, remains a difficult and relevant problem in the field of medical image processing because of the various factors like complex and variable location, size, and shape of liver tumors, low contrast between tumors and normal tissues, and blurred or difficult-to-define lesion boundaries. In this paper, we proposed a neural network (S-Net) that can incorporate attention mechanisms to end-to-end segmentation of liver tumors from CT images.MethodsFirst, this study adopted a classical coding-decoding structure to realize end-to-end segmentation. Next, we introduced an attention mechanism between the contraction path and the expansion path so that the network could encode a longer range of semantic information in the local features and find the corresponding relationship between different channels. Then, we introduced long-hop connections between the layers of the contraction path and the expansion path, so that the semantic information extracted in both paths could be fused. Finally, the application of closed operation was used to dissipate the narrow interruptions and long, thin divide. This eliminated small cavities and produced a noise reduction effect.ResultsIn this paper, we used the MICCAI 2017 liver tumor segmentation (LiTS) challenge dataset, 3DIRCADb dataset and doctors’ manual contours of Hubei Cancer Hospital dataset to test the network architecture. We calculated the Dice Global (DG) score, Dice per Case (DC) score, volumetric overlap error (VOE), average symmetric surface distance (ASSD), and root mean square error (RMSE) to evaluate the accuracy of the architecture for liver tumor segmentation. The segmentation DG for tumor was found to be 0.7555, DC was 0.613, VOE was 0.413, ASSD was 1.186 and RMSE was 1.804. For a small tumor, DG was 0.3246 and DC was 0.3082. For a large tumor, DG was 0.7819 and DC was 0.7632.ConclusionS-Net obtained more semantic information with the introduction of an attention mechanism and long jump connection. Experimental results showed that this method effectively improved the effect of tumor recognition in CT images and could be applied to assist doctors in clinical treatment.

Liver and liver tumor segmentations are essential in computer‐aided systems for diagnosing liver tumors. These systems must operate on multiphase computed tomography (CT) images instead of a single phase for accurate diagnosis for clinical applications. We have proposed a framework that can perform segmentation from quadriphasic CT data. The liver was segmented using a fine‐tuned SegNet model and the liver tumor was segmented using the K‐means clustering method coupled with a power‐law transformation‐based image enhancement technique. The best values for liver segmentation achieved were: Dice Coefficient = 96.46 ± 0.48%, Jaccard Index = 93.16 ± 0.89%, volumetric overlap error = 6.84 ± 0.89% and average symmetric surface distance = 0.59 ± 0.3 mm and the results for liver tumor delineation were Dice Coefficient = 85.07 ± 4.5%, Jaccard Index = 74.29 ± 6.8%, volumetric overlap error = 25.71 ± 6.8% and average symmetric surface distance = 1.14 ± 1.3 mm. The proposed liver segmentation method based on deep learning is fully automatic, robust, and effective for all phases. The image enhancement technique has shown promising results and aided in better liver tumor segmentation. The liver tumors were segmented satisfactorily; however, improvements concerning false positive reduction can further increase the accuracy.

Developing an effective liver and liver tumor segmentation model from CT scans is very important for the success of liver cancer diagnosis, surgical planning and cancer treatment. In this work, we propose a two-stage framework for 2D liver and tumor segmentation. The first stage is a coarse liver segmentation network, while the second stage is an edge enhanced network (E\(^2\)Net) for more accurate liver and tumor segmentation. E\(^2\)Net explicitly models complementary objects (liver and tumor) and their edge information within the network to preserve the organ and lesion boundaries. We introduce an edge prediction module in E\(^2\)Net and design an edge distance map between liver and tumor boundaries, which is used as an extra supervision signal to train the edge enhanced network. We also propose a deep cross feature fusion module to refine multi-scale features from both objects and their edges. E\(^2\)Net is more easily and efficiently trained with a small labeled dataset, and it can be trained/tested on the original 2D CT slices (resolve resampling error issue in 3D models). The proposed framework has shown superior performance on both liver and liver tumor segmentation compared to several state-of-the-art 2D, 3D and 2D/3D hybrid frameworks.

The liver is an essential metabolic organ of the human body, and malignant liver tumors seriously affect and threaten human life. The segmentation algorithm for liver and liver tumors is one of the essential branches of computer-aided diagnosis. This paper proposed a two-stage liver and tumor segmentation algorithm based on the convolutional neural network (CNN). In the present study, we used two stages to segment the liver and tumors: liver localization and tumor segmentation. In the liver localization stage, the network segments the liver region, adopts the encoding–decoding structure and long-distance feature fusion operation, and utilizes the shallow features’ spatial information to improve liver identification. In the tumor segmentation stage, based on the liver segmentation results of the first two steps, a CNN model was designed to accurately identify the liver tumors by using the 2D image features and 3D spatial features of the CT image slices. At the same time, we use the attention mechanism to improve the segmentation performance of small liver tumors. The proposed algorithm was tested on the public data set Liver Tumor Segmentation Challenge (LiTS). The Dice coefficient of liver segmentation was 0.967, and the Dice coefficient of tumor segmentation was 0.725. The proposed algorithm can accurately segment the liver and liver tumors in CT images. Compared with other state-of-the-art algorithms, the segmentation results of the proposed algorithm rank the highest in the Dice coefficient.

Background/Objectives: Accurate liver and tumor detection and segmentation are crucial in diagnosis of early-stage liver malignancies. As opposed to manual interpretation, which is a difficult and time-consuming process, accurate tumor detection using a computer-aided diagnosis system can save both time and human efforts. Methods: We propose a cascaded encoder-decoder technique based on self-organized neural networks, which is a recent variant of operational neural networks (ONNs), for accurate segmentation and identification of liver tumors. The first encoder-decoder CNN segments the liver. For generating the liver region of interest, the segmented liver mask is placed over the input computed tomography (CT) image and then fed to the second Self-ONN model for tumor segmentation. For further investigation the other three distinct encoder-decoder architectures U-Net, feature pyramid networks (FPNs), and U-Net++, have also been investigated by altering the backbone at the encoders utilizing ResNet and DenseNet variants for transfer learning. Results: For the liver segmentation task, Self-ONN with a ResNet18 backbone has achieved a dice similarity coefficient score of 98.182% and an intersection over union of 97.436%. Tumor segmentation with Self-ONN with the DenseNet201 encoder resulted in an outstanding DSC of 92.836% and IoU of 91.748%. Conclusions: The suggested method is capable of precisely locating liver tumors of various sizes and shapes, including tiny infection patches that were said to be challenging to find in earlier research.