95th Percentile Hausdorff Distance Research Articles

Automated segmentation and volume prediction in pediatric Wilms’ tumor CT using nnu-net

BackgroundRadiologic volumetric evaluation of Wilms’ tumor (WT) is an important indicator to guide treatment decisions. However, due to the heterogeneity of the tumors, radiologists have main-guard differences in diagnosis that can lead to misdiagnosis and poor treatment. The aim of this study was to explore whether CT-based outlining of WT foci can be automated using deep learning.MethodsWe included CT intravenous phase images of 105 patients with WT and double-blind outlining of lesions by two radiologists. Then, we trained an automatic segmentation model using nnUnet. The Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95) were used to assess the performance. Next, we optimized the automatic segmentation results based on the ratio of the three-dimensional diameter of the lesion to improve the performance of volumetric assessment.ResultsThe DSC and HD95 was 0.83 ± 0.22 and 10.50 ± 8.98 mm. The absolute difference and percentage difference in tumor size was 72.27 ± 134.84 cm3 and 21.08% ± 30.46%. After optimization according to our method, it decreased to 40.22 ± 96.06 cm3 and 10.16% ± 9.70%.ConclusionWe introduce a novel method that enhances the accuracy of predicting WT volume by integrating AI automated outlining and 3D tumor diameters. This approach surpasses the accuracy of using AI outcomes alone and has the potential to enhance the clinical evaluation of pediatric patients with WT. By intertwining AI outcomes with clinical data, this method becomes more interpretive and offers promising applications beyond Wilms tumor, extending to other pediatric diseases.

Estimating Surgical Urethral Length on Intraoperative Robot-Assisted Prostatectomy Images using Artificial Intelligence Anatomy Recognition.

Objective To construct a Convolutional Neural Network (CNN) model that can recognize and delineate anatomic structures on intraoperative video frames of robot-assisted radical prostatectomy (RARP) and to use these annotations to predict the surgical urethral length (SUL). Background Urethral dissection during RARP impacts patient urinary incontinence (UI) outcomes, and requires extensive training. Large differences exist between incontinence outcomes of different urologists and hospitals. Also, surgeon experience and education are critical towards optimal outcomes. Therefore new approaches are warranted. SUL is associated with UI. Artificial intelligence (AI) surgical image segmentation using a CNN could automate SUL estimation and contribute towards future AI-assisted RARP and surgeon guidance. Methods Eighty-eight intraoperative RARP videos between June 2009 and September 2014 were collected from a single center. 264 frames were annotated according to: prostate, urethra, ligated plexus and catheter. Thirty annotated images from different RARP videos were used as a test dataset. The Dice coefficient (DSC) and 95th percentile Hausdorff distance (Hd95) were used to determine model performance. SUL was calculated using the catheter as a reference. Results The DSC of the best performing model were 0.735 and 0.755 for the catheter and urethra classes respectively, with a Hd95 of 29.27 and 72.62 respectively. The model performed moderately on the ligated plexus and prostate. The predicted SUL showed a mean difference of 0.64 - 1.86mm difference versus human annotators, but with significant deviation (SD 3.28 - 3.56). Conclusion This study shows that an AI image segmentation model can predict vital structures during RARP urethral dissection with moderate to fair accuracy. SUL estimation derived from it showed large deviations and outliers when compared to human annotators, but with a very small mean difference (<2mm). This is a promising development for further research on AI-assisted RARP. Keywords Prostate cancer, Anatomy recognition, Artificial intelligence, Continence, Urethral length.

Impact of bias field correction on 0.35 T pelvic MR images: evaluation on generative adversarial network-based OARs' auto-segmentation and visual grading assessment.

Magnetic resonance imaging (MRI)-guided radiotherapy enables adaptive treatment plans based on daily anatomical changes and accurate organ visualization. However, the bias field artifact can compromise image quality, affecting diagnostic accuracy and quantitative analyses. This study aims to assess the impact of bias field correction on 0.35 T pelvis MRIs by evaluating clinical anatomy visualization and generative adversarial network (GAN) auto-segmentation performance. 3D simulation MRIs from 60 prostate cancer patients treated on MR-Linac (0.35 T) were collected and preprocessed with the N4ITK algorithm for bias field correction. A 3D GAN architecture was trained, validated, and tested on 40, 10, and 10 patients, respectively, to auto-segment the organs at risk (OARs) rectum and bladder. The GAN was trained and evaluated either with the original or the bias-corrected MRIs. The Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95th) were computed for the segmented volumes of each patient. The Wilcoxon signed-rank test assessed the statistical difference of the metrics within OARs, both with and without bias field correction. Five radiation oncologists blindly scored 22 randomly chosen patients in terms of overall image quality and visibility of boundaries (prostate, rectum, bladder, seminal vesicles) of the original and bias-corrected MRIs. Bennett's S score and Fleiss' kappa were used to assess the pairwise interrater agreement and the interrater agreement among all the observers, respectively. In the test set, the GAN trained and evaluated on original and bias-corrected MRIs showed DSC/HD95th of 0.92/5.63mm and 0.92/5.91mm for the bladder and 0.84/10.61mm and 0.83/9.71mm for the rectum. No statistical differences in the distribution of the evaluation metrics were found neither for the bladder (DSC: p = 0.07; HD95th: p = 0.35) nor for the rectum (DSC: p = 0.32; HD95th: p = 0.63). From the clinical visual grading assessment, the bias-corrected MRI resulted mostly in either no change or an improvement of the image quality and visualization of the organs' boundaries compared with the original MRI. The bias field correction did not improve the anatomy visualization from a clinical point of view and the OARs' auto-segmentation outputs generated by the GAN.

Transfer learning for auto-segmentation of 17 organs-at-risk in the head and neck: Bridging the gap between institutional and public datasets.

Auto-segmentation of organs-at-risk (OARs) in the head and neck (HN) on computed tomography (CT) images is a time-consuming component of the radiation therapy pipeline that suffers from inter-observer variability. Deep learning (DL) has shown state-of-the-art results in CT auto-segmentation, with larger and more diverse datasets showing better segmentation performance. Institutional CT auto-segmentation datasets have been small historically (n<50) due to the time required for manual curation of images and anatomical labels. Recently, large public CT auto-segmentation datasets (n>1000 aggregated) have become available through online repositories such as The Cancer Imaging Archive. Transfer learning is a technique applied when training samples are scarce, but a large dataset from a closely related domain is available. The purpose of this study was to investigate whether a large public dataset could be used in place of an institutional dataset (n>500), or to augment performance via transfer learning, when building HN OAR auto-segmentation models for institutional use. Auto-segmentation models were trained on a large public dataset (public models) and a smaller institutional dataset (institutional models). The public models were fine-tuned on the institutional dataset using transfer learning (transfer models). We assessed both public model generalizability and transfer model performance by comparison with institutional models. Additionally, the effect of institutional dataset size on both transfer and institutional models was investigated. All DL models used a high-resolution, two-stage architecture based on the popular 3D U-Net. Model performance was evaluated using five geometric measures: the dice similarity coefficient (DSC), surface DSC, 95th percentile Hausdorff distance, mean surface distance (MSD), and added path length. For a small subset of OARs (left/right optic nerve, spinal cord, left submandibular), the public models performed significantly better (p<0.05) than, or showed no significant difference to, the institutional models under most of the metrics examined. For the remaining OARs, the public models were inferior to the institutional models, although performance differences were small (DSC≤0.03, MSD<0.5mm) for seven OARs (brainstem, left/right lens, left/right parotid, mandible, right submandibular). The transfer models performed significantly better than the institutional models for seven OARs (brainstem, right lens, left/right optic nerve, left/right parotid, spinal cord) with a small margin of improvement (DSC≤0.02, MSD<0.4mm). When numbers of institutional training samples were limited, public and transfer models outperformed the institutional models for most OARs (brainstem, left/right lens, left/right optic nerve, left/right parotid, spinal cord, and left/right submandibular). Training auto-segmentation models with public data alone was suitable for a small number of OARs. Using only public data incurred a small performance deficit for most other OARs, when compared with institutional data alone, but may be preferable over time-consuming curation of a large institutional dataset. When a large institutional dataset was available, transfer learning with models pretrained on a large public dataset provided a modest performance improvement for several OARs. When numbers of institutional samples were limited, using the public dataset alone, or as a pretrained model, was beneficial for most OARs.

NnU-Net versus mesh growing algorithm as a tool for the robust and timely segmentation of neurosurgical 3D images in contrast-enhanced T1 MRI scans

PurposeThis study evaluates the nnU-Net for segmenting brain, skin, tumors, and ventricles in contrast-enhanced T1 (T1CE) images, benchmarking it against an established mesh growing algorithm (MGA).MethodsWe used 67 retrospectively collected annotated single-center T1CE brain scans for training models for brain, skin, tumor, and ventricle segmentation. An additional 32 scans from two centers were used test performance compared to that of the MGA. The performance was measured using the Dice-Sørensen coefficient (DSC), intersection over union (IoU), 95th percentile Hausdorff distance (HD95), and average symmetric surface distance (ASSD) metrics, with time to segment also compared.ResultsThe nnU-Net models significantly outperformed the MGA (p < 0.0125) with a median brain segmentation DSC of 0.971 [95CI: 0.945–0.979], skin: 0.997 [95CI: 0.984–0.999], tumor: 0.926 [95CI: 0.508–0.968], and ventricles: 0.910 [95CI: 0.812–0.968]. Compared to the MGA’s median DSC for brain: 0.936 [95CI: 0.890, 0.958], skin: 0.991 [95CI: 0.964, 0.996], tumor: 0.723 [95CI: 0.000–0.926], and ventricles: 0.856 [95CI: 0.216–0.916]. NnU-Net performance between centers did not significantly differ except for the skin segmentations Additionally, the nnU-Net models were faster (mean: 1139 s [95CI: 685.0–1616]) than the MGA (mean: 2851 s [95CI: 1482–6246]).ConclusionsThe nnU-Net is a fast, reliable tool for creating automatic deep learning-based segmentation pipelines, reducing the need for extensive manual tuning and iteration. The models are able to achieve this performance despite a modestly sized training set. The ability to create high-quality segmentations in a short timespan can prove invaluable in neurosurgical settings.

Auto-segmentation of pelvic organs at risk on 0.35T MRI using 2D and 3D Generative Adversarial Network models

PurposeManual recontouring of targets and Organs At Risk (OARs) is a time-consuming and operator-dependent task. We explored the potential of Generative Adversarial Networks (GAN) to auto-segment the rectum, bladder and femoral heads on 0.35T MRIs to accelerate the online MRI-guided-Radiotherapy (MRIgRT) workflow. Methods3D planning MRIs from 60 prostate cancer patients treated with 0.35T MR-Linac were collected. A 3D GAN architecture and its equivalent 2D version were trained, validated and tested on 40, 10 and 10 patients respectively. The volumetric Dice Similarity Coefficient (DSC) and 95th percentile Hausdorff Distance (HD95th) were computed against expert drawn ground-truth delineations. The networks were also validated on an independent external dataset of 16 patients. ResultsIn the internal test set, the 3D and 2D GANs showed DSC/HD95th of 0.83/9.72 mm and 0.81/10.65 mm for the rectum, 0.92/5.91 mm and 0.85/15.72 mm for the bladder, and 0.94/3.62 mm and 0.90/9.49 mm for the femoral heads. In the external test set, the performance was 0.74/31.13 mm and 0.72/25.07 mm for the rectum, 0.92/9.46 mm and 0.88/11.28 mm for the bladder, and 0.89/7.00 mm and 0.88/10.06 mm for the femoral heads. The 3D and 2D GANs required on average 1.44 s and 6.59 s respectively to generate the OARs’ volumetric segmentation for a single patient. ConclusionsThe proposed 3D GAN auto-segments pelvic OARs with high accuracy on 0.35T, in both the internal and the external test sets, outperforming its 2D equivalent in both segmentation robustness and volume generation time.

Lesion segmentation on 18F-fluciclovine PET/CT images using deep learning.

A novel radiotracer, 18F-fluciclovine (anti-3-18F-FACBC), has been demonstrated to be associated with significantly improved survival when it is used in PET/CT imaging to guide postprostatectomy salvage radiotherapy for prostate cancer. We aimed to investigate the feasibility of using a deep learning method to automatically detect and segment lesions on 18F-fluciclovine PET/CT images. We retrospectively identified 84 patients who are enrolled in Arm B of the Emory Molecular Prostate Imaging for Radiotherapy Enhancement (EMPIRE-1) trial. All 84 patients had prostate adenocarcinoma and underwent prostatectomy and 18F-fluciclovine PET/CT imaging with lesions identified and delineated by physicians. Three different neural networks with increasing levels of complexity (U-net, Cascaded U-net, and a cascaded detection segmentation network) were trained and tested on the 84 patients with a fivefold cross-validation strategy and a hold-out test, using manual contours as the ground truth. We also investigated using both PET and CT or using PET only as input to the neural network. Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95), center-of-mass distance (CMD), and volume difference (VD) were used to quantify the quality of segmentation results against ground truth contours provided by physicians. All three deep learning methods were able to detect 144/155 lesions and 153/155 lesions successfully when PET+CT and PET only, respectively, served as input. Quantitative results demonstrated that the neural network with the best performance was able to segment lesions with an average DSC of 0.68 ± 0.15 and HD95 of 4 ± 2mm. The center of mass of the segmented contours deviated from physician contours by approximately 2mm on average, and the volume difference was less than 1 cc. The novel network proposed by us achieves the best performance compared to current networks. The addition of CT as input to the neural network contributed to more cases of failure (DSC = 0), and among those cases of DSC > 0, it was shown to produce no statistically significant difference with the use of only PET as input for our proposed method. Quantitative results demonstrated the feasibility of the deep learning methods in automatically segmenting lesions on 18F-fluciclovine PET/CT images. This indicates the great potential of 18F-fluciclovine PET/CT combined with deep learning for providing a second check in identifying lesions as well as saving time and effort for physicians in contouring.

Deep learning-based clinical-radiomics nomogram for preoperative prediction of lymph node metastasis in patients with rectal cancer: a two-center study.

Precise preoperative evaluation of lymph node metastasis (LNM) is crucial for ensuring effective treatment for rectal cancer (RC). This research aims to develop a clinical-radiomics nomogram based on deep learning techniques, preoperative magnetic resonance imaging (MRI) and clinical characteristics, enabling the accurate prediction of LNM in RC. Between January 2017 and May 2023, a total of 519 rectal cancer cases confirmed by pathological examination were retrospectively recruited from two tertiary hospitals. A total of 253 consecutive individuals were selected from Center I to create an automated MRI segmentation technique utilizing deep learning algorithms. The performance of the model was evaluated using the dice similarity coefficient (DSC), the 95th percentile Hausdorff distance (HD95), and the average surface distance (ASD). Subsequently, two external validation cohorts were established: one comprising 178 patients from center I (EVC1) and another consisting of 88 patients from center II (EVC2). The automatic segmentation provided radiomics features, which were then used to create a Radscore. A predictive nomogram integrating the Radscore and clinical parameters was constructed using multivariate logistic regression. Receiver operating characteristic (ROC) curve analysis and decision curve analysis (DCA) were employed to evaluate the discrimination capabilities of the Radscore, nomogram, and subjective evaluation model, respectively. The mean DSC, HD95 and ASD were 0.857 ± 0.041, 2.186 ± 0.956, and 0.562 ± 0.194 mm, respectively. The nomogram, which incorporates MR T-stage, CEA, CA19-9, and Radscore, exhibited a higher area under the ROC curve (AUC) compared to the Radscore and subjective evaluation in the training set (0.921 vs. 0.903 vs. 0.662). Similarly, in both external validation sets, the nomogram demonstrated a higher AUC than the Radscore and subjective evaluation (0.908 vs. 0.735 vs. 0.640, and 0.884 vs. 0.802 vs. 0.734). The application of the deep learning method enables efficient automatic segmentation. The clinical-radiomics nomogram, utilizing preoperative MRI and automatic segmentation, proves to be an accurate method for assessing LNM in RC. This approach has the potential to enhance clinical decision-making and improve patient care. Research registry, identifier 9158, https://www.researchregistry.com/browse-the-registry#home/registrationdetails/648e813efffa4e0028022796/.

Deep learning models for preoperative T-stage assessment in rectal cancer using MRI: exploring the impact of rectal filling.

The objective of this study was twofold: firstly, to develop a convolutional neural network (CNN) for automatic segmentation of rectal cancer (RC) lesions, and secondly, to construct classification models to differentiate between different T-stages of RC. Additionally, it was attempted to investigate the potential benefits of rectal filling in improving the performance of deep learning (DL) models. A retrospective study was conducted, including 317 consecutive patients with RC who underwent MRI scans. The datasets were randomly divided into a training set (n = 265) and a test set (n = 52). Initially, an automatic segmentation model based on T2-weighted imaging (T2WI) was constructed using nn-UNet. The performance of the model was evaluated using the dice similarity coefficient (DSC), the 95th percentile Hausdorff distance (HD95), and the average surface distance (ASD). Subsequently, three types of DL-models were constructed: Model 1 trained on the total training dataset, Model 2 trained on the rectal-filling dataset, and Model 3 trained on the non-filling dataset. The diagnostic values were evaluated and compared using receiver operating characteristic (ROC) curve analysis, confusion matrix, net reclassification index (NRI), and decision curve analysis (DCA). The automatic segmentation showed excellent performance. The rectal-filling dataset exhibited superior results in terms of DSC and ASD (p = 0.006 and 0.017). The DL-models demonstrated significantly superior classification performance to the subjective evaluation in predicting T-stages for all test datasets (all p < 0.05). Among the models, Model 1 showcased the highest overall performance, with an area under the curve (AUC) of 0.958 and an accuracy of 0.962 in the filling test dataset. This study highlighted the utility of DL-based automatic segmentation and classification models for preoperative T-stage assessment of RC on T2WI, particularly in the rectal-filling dataset. Compared with subjective evaluation, the models exhibited superior performance, suggesting their noticeable potential for enhancing clinical diagnosis and treatment practices.

Estimation of right lobe graft weight for living donor liver transplantation using deep learning-based fully automatic computed tomographic volumetry

This study aimed at developing a fully automatic technique for right lobe graft weight estimation using deep learning algorithms. The proposed method consists of segmentation of the full liver region from computed tomography (CT) images, classification of the entire liver region into the right and left lobes, and estimation of the right lobe graft weight from the CT-measured right lobe graft volume using a volume-to-weight conversion formula. The first two steps were performed with a transformer-based deep learning model. To train and evaluate the model, a total of 248 CT datasets (188 for training, 40 for validation, and 20 for testing and clinical evaluation) were used. The Dice similarity coefficient (DSC), mean surface distance (MSD), and the 95th percentile Hausdorff distance (HD95) were used for evaluating the segmentation accuracy of the full liver region and the right liver lobe. The correlation coefficient (CC), percentage error (PE), and percentage absolute error (PAE) were used for the clinical evaluation of the estimated right lobe graft weight. The proposed method achieved high accuracy in segmentation for DSC, MSD, and HD95 (95.9% ± 1.0%, 1.2 ± 0.4 mm, and 5.2 ± 1.9 mm for the entire liver region; 92.4% ± 2.7%, 2.0 ± 0.7 mm, and 8.8 ± 2.9 mm for the right lobe) and in clinical evaluation for CC, PE, and PAE (0.859, − 1.8% ± 9.6%, and 8.6% ± 4.7%). For the right lobe graft weight estimation, the present study underestimated the graft weight by − 1.8% on average. A mean difference of − 21.3 g (95% confidence interval: − 55.7 to 13.1, p = 0.211) between the estimated graft weight and the actual graft weight was achieved in this study. The proposed method is effective for clinical application.

Towards deep-learning (DL) based fully automated target delineation for rectal cancer neoadjuvant radiotherapy using a divide-and-conquer strategy: a study with multicenter blind and randomized validation

PurposeManual clinical target volume (CTV) and gross tumor volume (GTV) delineation for rectal cancer neoadjuvant radiotherapy is pivotal but labor-intensive. This study aims to propose a deep learning (DL)-based workflow towards fully automated clinical target volume (CTV) and gross tumor volume (GTV) delineation for rectal cancer neoadjuvant radiotherapy.Materials & methodsWe retrospectively included 141 patients with Stage II-III mid-low rectal cancer and randomly grouped them into training (n = 121) and testing (n = 20) cohorts. We adopted a divide-and-conquer strategy to address CTV and GTV segmentation using two separate DL models with DpuUnet as backend-one model for CTV segmentation in the CT domain, and the other for GTV in the MRI domain. The workflow was validated using a three-level multicenter-involved blind and randomized evaluation scheme. Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (95HD) metrics were calculated in Level 1, four-grade expert scoring was performed in Level 2, and head-to-head Turing test in Level 3.ResultsFor the DL-based CTV contours over the testing cohort, the DSC and 95HD (mean ± SD) were 0.85 ± 0.06 and 7.75 ± 6.42 mm respectively, and 96.4% cases achieved clinical viable scores (≥ 2). The positive rate in the Turing test was 52.3%. For GTV, the DSC and 95HD were 0.87 ± 0.07 and 4.07 ± 1.67 mm respectively, and 100% of the DL-based contours achieved clinical viable scores (≥ 2). The positive rate in the Turing test was 52.0%.ConclusionThe proposed DL-based workflow exhibited promising accuracy and excellent clinical viability towards automated CTV and GTV delineation for rectal cancer neoadjuvant radiotherapy.

Development and Validation of a Deep Learning-Based Auto-Delineation of Target Volume and Organs at Risk in Pancreatic Cancer Radiotherapy.

The delineation of the clinical target volume (CTV), gross target volume (GTV) and organs at risk (OARs) is a crucial and laborious in pancreatic cancer radiotherapy. In this work, we propose and evaluate a three-dimensional (3D) novel convolutional neural network (CNN) for automatic and accurate CTV, GTV and OARs in pancreatic cancer. A total of 120 computed tomography (CT) scans patients with pancreatic cancer were collected. A novel 3D CNN network, called ResUNet3D, was developed to achieve auto-delineation. 96 patients chosen randomly were used for training, 12 patients for validation, and 12 patients for testing. Meanwhile, the Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95%) were used to assess the performance. The DSC values for the test data were 80.9±8.6%, 77.5±5.6%, 94.5±1.3%, 66.2±13.4%, 73.6±7.6%, 79.0±8.7%, 94.1±1.9%, 94.6±1.4%, 87.3±5.8% for CTV, GTV, liver, duodenum, spinal cord, bowel, kidney left, kidney right, stomach. The corresponding HD95% values were 10.7±6.9mm, 7.8±5.7mm, 11.6±5.6mm, 18.6±5.6mm, 2.7±0.7mm, 17.7±8.6mm, 3.9±1.4mm, 3.7±1.9mm, 13.4±5.7mm, respectively. The average delineation time for one patient's CT images was within 5 seconds. The experimental results demonstrate that the CTV, GTV and OARs delineated for pancreatic cancer by ResUNet3D achieved a close agreement with the ground truth. ResUNet3D could significantly reduce the radiation oncologists' contouring time.

Offline Verification of Daily Adaptive Therapy for Locally Advanced Lung Cancer

Daily adaptive therapy (AdaptT) for locally advanced lung cancer has been an intriguing field with the adoption of cone beam CT (CBCT). We have utilized AdaptT for lung cancer. In this workflow, contours for targets and organs at risk from the original CT simulation dataset are deformed and registered to the CBCT for each fraction. A physicist reviews and edits the deformed contours online without MD presence and a new plan is treated daily. Treating physicians are required to attend 1 fraction per week. We sought to evaluate the accuracy of these contours for targets and OARs in comparison to a thoracic physician contouring offline with the same CBCT dataset. The impetus of this study was to evaluate the model of physics directed contouring and ensure that gross tumor volume (GTV) misses are not occurring during treatment, defined as Planning Target Volumes (PTV's) derived from AdaptT failing to encompass the GTV redrawn offline by the physician. Six consecutive patients were treated from 2021-2022 with AdaptT utilizing processes as described above. One CBCT per week was re-contoured by a physician for 5-6 CBCT's per case. The GTVs, heart, and lungs were redrawn without registration. The physician was blinded to the structures approved for that fraction and from clinical information outside of the CT simulation data. The clinical target volume expansions were 5 mm shaved from normal tissue barriers and a uniform 5 mm expansion to form the PTV. The Dice Similarity Coefficient (DSE), mean surface distance (MSD), 95th percentile Hausdorff distance (95HD) for contours were calculated and the True Positive Fraction (TPF) of the redrawn GTVs within the treated PTVs were calculated. Thirty CBCTs from 6 patients were contoured. The TPF was 1 for each redrawn GTV, indicating each GTV drawn by the physician offline would have been encompassed within the PTV for treatment. The median 95HF for the targets was below 5 mm (Table 1) and the MSD for targets was close to 1.07 mm. The DSC for Heart and Lungs approached 1.0, indicating close similarity. The average time the physician spent on contouring was 9.8 minutes compared to 9.6 minutes with AdaptT. All redrawn GTVs were within the treated PTV and there was close agreement of normal structures. We evaluated the current model employing daily physics overview with weekly physician input and it appears safe and feasible. Further prospective study is needed to standardize comparison of contour quality and validate clinical outcomes of this approach.

A Cascaded Deep Learning-Based Cardiac Substructures Segmentation Frame and on Non-Gated Non-Enhanced Planning CT Scans in Breast Cancer Patients

To develop a deep learning-based segmentation frame for cardiac substructures especially coronary arteries (CAs) on non-gated non-enhanced planning computed tomography (CT) scans in breast cancer (BC) patients. Non-gated non-enhanced CT scans of 39 BC patients receiving adjuvant radiotherapy (RT) were collected. Cardiac substructures were manually labelled, including four chambers, left main (LM), left anterior descending (LAD), left circumflex (LCX) and right coronary artery (RCA). The training, validation, and test sample is 28, 7 and 4, respectively. A cascaded network, using nnUNet as the backbone, is proposed to use chambers as prior information to constrain the segmentation of CAs. The mean Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95) and average symmetrical surface distance (ASSD) were used as geometric metrics. Dosimetric parameters of cardiac substructures was calculated based on the segmentation frame and manually labeled contouring, respectively. The data of cardiac examination including ultrasonography, electrocardiogram before and during the follow-up after RT were retrospectively collected. The cardiac event was any symptomatic heart disease or new-onset abnormality in the cardiac examination after RT. The mean DSC of heart, atriums and ventricles of the proposed frame was 0.93, 0.90, and 0.93, respectively. As shown in Table 1, compared with direct segmentation (as baseline), the proposed frame had a better performance in terms of HD95, ASSD, and the mean dose (Dmean) absolute error for all CAs. Compared to the dosimetric parameters of the heart collected based on the manual labelled contours, the relative errors of D5, D95, and V15Gy for LAD was 4.3±7.8%, 11.7±5.9%, and 14.6±13.0% collected based on the direct segmentation contours and 2.4±4.4%, 3.9±3.1%, 8.5±6.9% collected based on the auto-segmented contours, respectively. Multivariate analysis showed that increased V15Gy of LAD was an independent cardiac toxicity risk factor ([HR] = 1.07, 95% CI 1-1.15, p = 0.0387). We developed a cascaded network for cardiac substructures segmentation with dosimetric validation on non-enhanced CT scans in breast cancer radiotherapy. This is the first attempt to use chambers as prior information for CAs' segmentation and had a superior stable performance. Accurate segmentation will help radiation oncologists to better evaluate DVHs based on substructures and thus to estimate cardiovascular risk. An optimized cardiac substructure-based dosimetric constrain may be proposed accordingly.

Evaluation of a Deep-Learning Auto-Segmentation Model of Cardiac Substructures

Increasing evidence has suggested that limiting dose not only to the whole heart but also to cardiac substructures can potentially reduce cardiac toxicities. Manual contouring of cardiac substructure can be challenging and time-consuming. To address this concern, we developed a deep learning (DL) model, trained on convolutional neural network algorithms in large external datasets, for auto-segmentation of cardiac substructures. This study aimed to evaluate the quality of the cardiac substructure contours generated by the DL algorithm. We identified 28 patients with esophagus or gastroesophageal junction cancer from a single institution who received radiation to the esophagus between January 2017 and December 2022. For each case, the DL-generated cardiac substructures (4 heart chambers - left/right atrium [L/RA] and L/R ventricle [L/RV], 4 coronary arteries - L common [LCA], L anterior descending [LAD], L circumflex [LCx], and R common [RCA], and great vessels - ascending aorta [AA], pulmonary artery [PA], and superior vena cava [SVC]) were modified by two radiation oncologists (RO) using the contouring atlas developed by Duane et al. Spatial overlapping of the contours were then assessed using the Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD-95), and normalized surface dice at 2 mm tolerance (NSD-2). The mean values of DSC, HD-95, and NSD-2 are shown in Table 1. Overall, the mean DSC, HD-95, and NSD-2 for the heart chambers ranged from 0.82 to 0.92, 0.40 cm to 1.52 cm, and 0.68 to 0.85, respectively. Ranges of the mean DSC, HD-95, and NSD-2 for the coronary vessels were 0.41 to 0.74, 0.18 cm to 0.98 cm, and 0.66 to 0.77, respectively. Lastly, comparison of the great vessel contours yielded the following ranges for mean DSC, HD-95, and NSD-2 respectively: 0.72 to 0.92, 0.30 cm to 1.64 cm, and 0.65 to 0.83. Our study demonstrates that auto-segmentation of cardiac substructures by DL-powered models can be comparable to manual contours for certain cardiac substructures, namely the four heart chambers and great vessels. Further improvement of the DL on contouring of coronary vessels would be needed prior to the autosegmentation model being widely adopted.

Determining the Dosimetric Accuracy of Deep Learning-Based Fully Automated Registration-Segmentation Approach for Thoracic Cancer Organs-at-Risk Contouring

For adaptive radiation therapy (ART), the contours on the planning CT (pCT) are frequently propagated to cone-beam CT (CBCT) via deformable image registration and manually edited, which is observer-dependent and time-consuming. To automate this process, we created a fully automated workflow by combining a deep learning (DL)-based pCT segmentation model with a CT-to-CBCT registration-segmentation DL model. The purpose of our research is to determine how using the proposed workflow's automatically generated contours affects thoracic organs-at-risk sparing (OAR). Seven patients with locally advanced non-small cell lung cancer who underwent treatment with intensity modulated radiation therapy were included in this study. Each patient's pCT was segmented using a published DL model that has been used for generating thoracic OAR segmentation and radiotherapy planning in the clinic since July of 2020. Next, pCT was deformably registered using a published recurrent deep registration-segmentation method. Whereas the original method's segmentation sub-network was only trained to segment esophagus, the registration sub-network was used to propagate contours for heart, esophagus, and the proximal bronchial tree (PBT). Geometric segmentation accuracy using the Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD) and dose metrics including the mean esophageal dose (MED) and D90% of the heart (D90) were computed from the total accumulated dose for the first two weeks of treatment. The esophagus had a high DSC and a low HD (0.93 and 2.85mm) and conversely, the heart had lower accuracy (DSC = 0.85, HD = 22.06mm). PBT showed relatively high performance as well, with DSC of 0.91 and HD of 2.28mm, owing to its proximity to the esophagus. The accumulated MED for manual contour was slightly lower than AI-contours (11.34 vs 11.83 Gy), suggesting reliability of the proposed workflow. The reverse is seen for the D90 of the heart (manual = 1.74 and AI-contour = 1.56 Gy), likely due to the heart not being included in the original DL framework. This study reported preliminary results on the feasibility of using a fully automated and patient-specific workflow for CBCT auto-segmentation in ART, confirming its role as a geometrically and dosimetrically accurate solution for thoracic OARs. However, because it is currently limited to the esophagus, we believe that re-training the algorithm will increase confidence in other OARs such as the heart and lungs.

Interobserver Variation in Tumor Delineation of Liver Metastases using Magnetic Resonance Imaging

Magnetic Resonance (MR-) guided stereotactic body radiotherapy enables accurate treatment of liver metastases. Despite the accuracy of treatment delivery, one of the main uncertainties is the variation in tumor delineation, which requires additional treatment margins. The aim of this study was to quantify the interobserver variation (IOV) in MRI based delineation of the gross tumor volume (GTV) of liver metastases. A cohort of 20 liver metastases treated on a MR-Linac were consecutively selected per primary tumor (colorectal (6), breast (6) and lung (8)). Planning MRI scans (T1 weighted with IV-contrast) were collected and anonymized. GTV delineation guidelines and case-specific information were provided to 8 radiation oncologists from 8 institutions. All cases were quantitatively reviewed and delineations with major violations of the guidelines were marked as outliers (such as contours including a vein or excluding obvious parts of a tumor). IOV was quantified by comparing individual delineations with the median contour and calculating the standard deviation (SD) and 95th percentile Hausdorff distance (HD95). Analyses were conducted on all delineations and on a subgroup excluding outliers per case to distinguish between accuracy of delineation and individual guideline interpretation. Sphericity was calculated from the volume and surface area of all delineations. Correlation between SD and sphericity was determined using Spearman's rs. Differences in SD between primary tumors were analyzed using the Kruskal-Wallis test. The median volume of all metastases was 6.5 cc (IQR 3.7 - 29.8 cc) and sphericity was 0.84 (IQR 0.80 - 0.87) based on all contours (Table 1). The SD was 1.6 mm and the median HD95 was 2.7 mm. In the subgroup analysis, one (in 15 cases) or two (in 5 cases) delineations were excluded. Subgroup analyses showed a SD of 1.1 mm and median HD 95 of 2.6mm. The Kruskal-Wallis test showed no difference (p = 0.59) in SD between primary tumors. There was a significant negative correlation between IOV (SD) and sphericity (rs = -0.70; p = 0.008). Table 1: Overview of SD (mm), HD95 (mm) and volume (cc) of all observers and subgroup per primary tumor and in total CONCLUSION: MRI based GTV delineation variation of liver metastases is 1.1 - 1.6 mm, which is smaller than anticipated, although significant outliers (HD95 up to 3.9 mm) were present. Use of common delineation guidelines will ensure consistency in contouring and have the potential to decrease treatment margins when taking IOV into account.

A Comprehensive Deep Learning Framework for Automatic Target Volumes Segmentation in Post-Operative Stereotactic Partial Breast Irradiation (S-PBI)

In S-PBI, accurate delineation of post-surgical tumor bed volume (TBV) and clinical target volume (CTV) are crucial tasks to achieve effective radiotherapy outcomes. However, manual contouring is labor intensive, time consuming, and largely relies on the experience of clinicians. We aimed to propose a deep learning (DL) approach which mimics physicians' contouring practice to accurately segment target volumes in post-operative breast CT images. Our approach incorporated domain knowledge into a 3D U-Net based DL model for breast target volumes (TBV and CTV) delineation. Our TBV segmentation approach was inspired by the marker-guidance procedure in manual delineation, where the visual clues provided by the markers assist physicians in defining TBV. For this purpose, a distance-transformation coupled with a Gaussian filter was adopted to convert markers' locations on the CT images to saliency maps. Subsequently, the CT images and the corresponding saliency maps formed a two-channel input for the segmentation model. For CTV segmentation, TBV was incorporated as an input in addition to the CT images, guiding the model to encode the location-related image features. The architecture allowed the network to emulate the oncologist's manual delineation where CTV is derived from TBV via a margin expansion, followed by correcting the extensions for anatomical barriers of tumor invasion (e.g., skin, chest wall). We retrospectively collected 175 prone CT images from 35 post-operative breast cancer patients who received 5-fraction partial breast irradiation (PBI) regimen on a Co-60 prone based S-PBI unit. The 35 patients were randomly split into 25, 5, and 5 for model training, validation, and testing respectively. We evaluated the performance of the developed DL model on the testing dataset by comparing the predicted volumes with the manually delineated contours (ground truth) using Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95), and average symmetric surface distance (ASD). For TBV segmentation, our model achieved mean (standard deviation) of 0.76 (±2.7), 6.76 (±1.83) mm, and 1.9 (±0.66) mm for DSC, HD95, and ASD respectively. For CTV segmentation, our model achieved 0.94 (±0.02), 2.46 (±0.5) mm, and 0.53 (±0.14) mm for DSC, HD95, and ASD respectively. The proposed auto-segmentation approach generated TBV and CTV masks in ∼11 seconds per CT volume, implying significantly improved efficiency compared to manual contouring. We developed a comprehensive DL framework mimicking clinical contouring practice for auto-segmentation of target volumes in S-PBI. The results demonstrated high levels of agreement between the predicted contours and physicians' manual contours. The approach is promising for improving the efficiency and accuracy of the on-line treatment planning workflow, such as adaptive based S-PBI.

Automated volumetric assessment of pituitary adenoma

PurposeAssessment of pituitary adenoma (PA) volume and extent of resection (EOR) through manual segmentation is time-consuming and likely suffers from poor interrater agreement, especially postoperatively. Automated tumor segmentation and volumetry by use of deep learning techniques may provide more objective and quick volumetry.MethodsWe developed an automated volumetry pipeline for pituitary adenoma. Preoperative and three-month postoperative T1-weighted, contrast-enhanced magnetic resonance imaging (MRI) with manual segmentations were used for model training. After adequate preprocessing, an ensemble of convolutional neural networks (CNNs) was trained and validated for preoperative and postoperative automated segmentation of tumor tissue. Generalization was evaluated on a separate holdout set.ResultsIn total, 193 image sets were used for training and 20 were held out for validation. At validation using the holdout set, our models (preoperative / postoperative) demonstrated a median Dice score of 0.71 (0.27) / 0 (0), a mean Jaccard score of 0.53 ± 0.21/0.030 ± 0.085 and a mean 95th percentile Hausdorff distance of 3.89 ± 1.96./12.199 ± 6.684. Pearson’s correlation coefficient for volume correlation was 0.85 / 0.22 and −0.14 for extent of resection. Gross total resection was detected with a sensitivity of 66.67% and specificity of 36.36%.ConclusionsOur volumetry pipeline demonstrated its ability to accurately segment pituitary adenomas. This is highly valuable for lesion detection and evaluation of progression of pituitary incidentalomas. Postoperatively, however, objective and precise detection of residual tumor remains less successful. Larger datasets, more diverse data, and more elaborate modeling could potentially improve performance.

A new architecture combining convolutional and transformer-based networks for automatic 3D multi-organ segmentation on CT images.

Deep learning-based networks have become increasingly popular in the field of medical image segmentation. The purpose of this research was to develop and optimize a new architecture for automatic segmentation of the prostate gland and normal organs in the pelvic, thoracic, and upper gastro-intestinal (GI) regions. We developed an architecture which combines a shifted-window (Swin) transformer with a convolutional U-Net. The network includes a parallel encoder, a cross-fusion block, and a CNN-based decoder to extract local and global information and merge related features on the same scale. A skip connection is applied between the cross-fusion block and decoder to integrate low-level semantic features. Attention gates (AGs) are integrated within the CNN to suppress features in image background regions. Our network is termed "SwinAttUNet." We optimized the architecture for automatic image segmentation. Training datasets consisted of planning-CT datasets from 300 prostate cancer patients from an institutional database and 100 CT datasets from a publicly available dataset (CT-ORG). Images were linearly interpolated and resampled to a spatial resolution of (1.0×1.0×1.5) mm3 . A volume patch (192×192×96) was used for training and inference, and the dataset was split into training (75%), validation (10%), and test (15%) cohorts. Data augmentation transforms were applied consisting of random flip, rotation, and intensity scaling. The loss function comprised Dice and cross-entropy equally weighted and summed. We evaluated Dice coefficients (DSC), 95th percentile Hausdorff Distances (HD95), and Average Surface Distances (ASD) between results of our network and ground truth data. SwinAttUNet, DSC values were 86.54±1.21, 94.15±1.17, and 87.15±1.68% and HD95 values were 5.06±1.42, 3.16±0.93, and 5.54±1.63mm for the prostate, bladder, and rectum, respectively. Respective ASD values were 1.45±0.57, 0.82±0.12, and 1.42±0.38mm. For the lung, liver, kidneys and pelvic bones, respective DSC values were: 97.90±0.80, 96.16±0.76, 93.74±2.25, and 89.31±3.87%. Respective HD95 values were: 5.13±4.11, 2.73±1.19, 2.29±1.47, and 5.31±1.25mm. Respective ASD values were: 1.88±1.45, 1.78±1.21, 0.71±0.43, and 1.21±1.11mm. Our network outperformed several existing deep learning approaches using only attention-based convolutional or Transformer-based feature strategies, as detailed in the results section. We have demonstrated that our new architecture combining Transformer- and convolution-based features is able to better learn the local and global context for automatic segmentation of multi-organ, CT-based anatomy.