95th Percentile Hausdorff Distance Research Articles

Comprehensive segmentation of gray matter structures on T1-weighted brain MRI: A Comparative Study of CNN, CNN hybrid-transformer or -Mamba architectures.

Recent advances in deep learning have shown promising results in medical image analysis and segmentation. However, most brain MRI segmentation models are limited by the size of their datasets and/or the number of structures they can identify. This study evaluates the performance of six advanced deep learning models in segmenting 122 brain structures from T1-weighted MRI scans, aiming to identify the most effective model for clinical and research applications. 1,510 T1-weighted MRIs were used to compare six deep-learning models for the segmentation of 122 distinct gray matter structures: nnU-Net, SegResNet, SwinUNETR, UNETR, U-Mamba_BOT and U-Mamba_ Enc. Each model was rigorously tested for accuracy using the Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD95). Additionally, the volume of each structure was calculated and compared between normal control (NC) and Alzheimer's Disease (AD) patients. U-Mamba_Bot achieved the highest performance with a median DSC of 0.9112 [IQR:0.8957, 0.9250]. nnU-Net achieved a median DSC of 0.9027 [IQR: 0.8847, 0.9205] and had the highest HD95 of 1.392[IQR: 1.174, 2.029]. The value of each HD95 (<3mm) indicates its superior capability in capturing detailed brain structures accurately. Following segmentation, volume calculations were performed, and the resultant volumes of normal controls and AD patients were compared. The volume changes observed in thirteen brain substructures were all consistent with those reported in existing literature, reinforcing the reliability of the segmentation outputs. This study underscores the efficacy of U-Mamba_Bot as a robust tool for detailed brain structure segmentation in T1-weighted MRI scans. The congruence of our volumetric analysis with the literature further validates the potential of advanced deep-learning models to enhance the understanding of neurodegenerative diseases such as AD. Future research should consider larger datasets to validate these findings further and explore the applicability of these models in other neurological conditions. AD = Alzheimer's Disease; ADNI = Alzheimer's Disease Neuroimaging Initiative; DSC = Dice Similarity Coefficient; HD95 = the 95th Percentile Hausdorff Distance; IQR = Interquartile Range; NC = Normal Control; SSMs = State-space Sequence Models.

Efficient application of deep learning-based elective lymph node regions delineation for pelvic malignancies.

While there are established international consensuses on the delineation of pelvic lymph node regions (LNRs), significant inter- and intra-observer variabilities persist. Contouring these clinical target volumes for irradiation in pelvic malignancies is both time-consuming and labor-intensive. The purpose of this study was to develop a deep learning model of pelvic LNRs delineation for patients with pelvic cancers. Planning computed tomography (CT) studies of 160 patients with pelvic primary malignancies (including rectal, prostate, and cervical cancer) were retrospectively collected and divided into training set (n=120) and testing set (n=40). Six pelvic LNRs, including abdominal presacral, pelvic presacral, internal iliac nodes, external iliac nodes, obturator nodes, and inguinal nodes were delineated by two radiation oncologists as ground truth (Gt) contours. The cascaded multi-heads U-net (CMU-net) was constructed based on the Gt contours from training cohort, which was subsequently verified in the testing cohort. The automatic delineation of six LNRs (Auto) was evaluated using dice similarity coefficient (DSC), average surface distance (ASD), 95th percentile Hausdorff distance (HD95), and a 7-point scale score. In the testing set, the DSC of six pelvic LNRs by CMU-net model varied from 0.851 to 0.942, ASD from 0.381 to 1.037mm, and HD95 from 2.025 to 3.697mm. No significant differences were founded in these three parameters between postoperative and preoperative cases. 95.9% and 96.2% of auto delineations by CMU-net model got a score of 1-3 by two expert radiation oncologists, respectively, meaning only minor edits needed. The CMU-net was successfully developed for automated delineation of pelvic LNRs for pelvic malignancies radiotherapy with improved contouring efficiency and highly consistent, which might justify its implementation in radiotherapy work flow.

Head and neck automatic multi-organ segmentation on Dual-Energy Computed Tomography

Background and purposeDeep-learning-based automatic segmentation is widely used in radiation oncology to delineate organs-at-risk. Dual-energy CT (DECT) allows the reconstruction of enhanced contrast images that could help with manual and auto-delineation. This paper presents a performance evaluation of a commercial auto-segmentation software on image series generated by a DECT. Material and methodsDifferent types of DECT images from seventy four head-and-neck (HN) patients were retrieved, including polyenergetic images at different voltages [80 kV reconstructed with a kernel corresponding to the commercial algorithm DirectDensity™ (PEI80-DD), 80 kV (PEI80), 120 kV-mixed (PEI120)] and a virtual-monoenergetic image at 40 keV (VMI40). Delineations used for treatment planning were considered as ground truth (GT) and were compared with the auto-segmentations performed on the 4 DECT images. A blinded qualitative evaluation of 3 structures (thyroid, left parotid, left nodes level II) was carried out. Performance metrics were calculated for thirteen HN structures to evaluate the auto-contours including dice similarity coefficient (DSC), 95th percentile Hausdorff distance (95HD) and mean surface distance (MSD). ResultsWe observed a high rate of low scores for PEI80-DD and VMI40 auto-segmentations on the thyroid and for GT and VMI40 contours on the nodes level II. All images received excellent scores for the parotid glands. The metrics comparison between GT and auto-segmented contours revealed that PEI80-DD had the highest DSC scores, significantly outperforming other reconstructed images for all organs (p < 0.05). ConclusionsThe results indicate that the auto-contouring system cannot generalize to images derived from DECT acquisition. It is therefore crucial to identify which organs benefit from these acquisitions to adapt the training datasets accordingly.

Deep learning-based segmentation for high-dose-rate brachytherapy in cervical cancer using 3D Prompt-ResUNet

Objective.To develop and evaluate a 3D Prompt-ResUNet module that utilized the prompt-based model combined with 3D nnUNet for rapid and consistent autosegmentation of high-risk clinical target volume (HRCTV) and organ at risk (OAR) in high-dose-rate brachytherapy for cervical cancer patients.Approach.We used 73 computed tomography scans and 62 magnetic resonance imaging scans from 135 (103 for training, 16 for validation, and 16 for testing) cervical cancer patients across two hospitals for HRCTV and OAR segmentation. A novel comparison of the deep learning neural networks 3D Prompt-ResUNet, nnUNet, and segment anything model-Med3D was applied for the segmentation. Evaluation was conducted in two parts: geometric and clinical assessments. Quantitative metrics included the Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95%), Jaccard index (JI), and Matthews correlation coefficient (MCC). Clinical evaluation involved interobserver comparison, 4-grade expert scoring, and a double-blinded Turing test.Main results.The Prompt-ResUNet model performed most similarly to experienced radiation oncologists, outperforming less experienced ones. During testing, the DSC, HD95% (mm), JI, and MCC value (mean ± SD) for HRCTV were 0.92 ± 0.03, 2.91 ± 0.69, 0.85 ± 0.04, and 0.92 ± 0.02, respectively. For the bladder, these values were 0.93 ± 0.05, 3.07 ± 1.05, 0.87 ± 0.08, and 0.93 ± 0.05, respectively. For the rectum, they were 0.87 ± 0.03, 3.54 ± 1.46, 0.78 ± 0.05, and 0.87 ± 0.03, respectively. For the sigmoid, they were 0.76 ± 0.11, 7.54 ± 5.54, 0.63 ± 0.14, and 0.78 ± 0.09, respectively. The Prompt-ResUNet achieved a clinical viability score of at least 2 in all evaluation cases (100%) for both HRCTV and bladder and exceeded the 30% positive rate benchmark for all evaluated structures in the Turing test.Significance.The Prompt-ResUNet architecture demonstrated high consistency with ground truth in autosegmentation of HRCTV and OARs, reducing interobserver variability and shortening treatment times.

Three contrasts in 3 min: Rapid, high-resolution, and bone-selective UTE MRI for craniofacial imaging with automated deep-learning skull segmentation.

Ultrashort echo time (UTE) MRI can be a radiation-free alternative to CT for craniofacial imaging of pediatric patients. However, unlike CT, bone-specific MR imaging is limited by long scan times, relatively low spatial resolution, and a time-consuming bone segmentation workflow. A rapid, high-resolution UTE technique for brain and skull imaging in conjunction with an automatic segmentation pipeline was developed. A dual-RF, dual-echo UTE sequence was optimized for rapid scan time (3 min) and smaller voxel size (0.65 mm3). A weighted least-squares conjugate gradient method for computing the bone-selective image improves bone specificity while retaining bone sensitivity. Additionally, a deep-learning U-Net model was trained to automatically segment the skull from the bone-selective images. Ten healthy adult volunteers (six male, age 31.5 ± 10 years) and three pediatric patients (two male, ages 12 to 15 years) were scanned at 3 T. Clinical CT for the three patients were obtained for validation. Similarities in 3D skull reconstructions relative to clinical standard CT were evaluated based on the Dice similarity coefficient and Hausdorff distance. Craniometric measurements were used to assess geometric accuracy of the 3D skull renderings. The weighted least-squares method produces images with enhanced bone specificity, suppression of soft tissue, and separation from air at the sinuses when validated against CT in pediatric patients. Dice similarity coefficient overlap was 0.86 ± 0.05, and the 95th percentile Hausdorff distance was 1.77 ± 0.49 mm between the full-skull binary masks of the optimized UTE and CT in the testing dataset. An optimized MRI acquisition, reconstruction, and segmentation workflow for craniofacial imaging was developed.

Anatomy-constrained synthesis for spleen segmentation improvement in unpaired mouse micro-CT scans with 3D CycleGAN

Objective. Auto-segmentation in mouse micro-CT enhances the efficiency and consistency of preclinical experiments but often struggles with low-native-contrast and morphologically complex organs, such as the spleen, resulting in poor segmentation performance. While CT contrast agents can improve organ conspicuity, their use complicates experimental protocols and reduces feasibility. We developed a 3D Cycle Generative Adversarial Network (CycleGAN) incorporating anatomy-constrained U-Net models to leverage contrast-enhanced CT (CECT) insights to improve unenhanced native CT (NACT) segmentation. Approach. We employed a standard CycleGAN with an anatomical loss function to synthesize virtual CECT images from unpaired NACT scans at two different resolutions. Prior to training, two U-Nets were trained to automatically segment six major organs in NACT and CECT datasets, respectively. These pretrained 3D U-Nets were integrated during the CycleGAN training, segmenting synthetic images, and comparing them against ground truth annotations. The compound loss within the CycleGAN maintained anatomical fidelity. Full image processing was achieved for low-resolution datasets, while high-resolution datasets employed a patch-based method due to GPU memory constraints. Automated segmentation was applied to original NACT and synthetic CECT scans to evaluate CycleGAN performance using the Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD95p). Main results. High-resolution scans showed improved auto-segmentation, with an average DSC increase from 0.728 to 0.773 and a reduced HD95p from 1.19 mm to 0.94 mm. Low-resolution scans benefited more from synthetic contrast, showing a DSC increase from 0.586 to 0.682 and an HD95p reduction from 3.46 mm to 1.24 mm. Significance. Implementing CycleGAN to synthesize CECT scans substantially improved the visibility of the mouse spleen, leading to more precise auto-segmentation. This approach shows the potential in preclinical imaging studies where contrast agent use is impractical.

Heart and great vessels segmentation in congenital heart disease via CNN and conditioned energy function postprocessing.

The segmentation of the heart and great vessels in CT images of congenital heart disease (CHD) is critical for the clinical assessment of cardiac anomalies and the diagnosis of CHD. However, the diverse types and abnormalities inherent in CHD present significant challenges to comprehensive heart segmentation. We proposed a novel two-stage segmentation approach, integrating a Convolutional Neural Network (CNN) with a postprocessing method with conditioned energy function for pulmonary and aorta. The initial stage employs a CNN enhanced by a gated self-attention mechanism for the segmentation of five primary heart structures and two major vessels. Subsequently, the second stage utilizes a conditioned energy function specifically tailored to refine the segmentation of the pulmonary artery and aorta, ensuring vascular continuity. Our method was evaluated on a public dataset including 110 3D CT volumes, encompassing 16 CHD variants. Compared to prevailing segmentation techniques (U-Net, V-Net, Unetr, dynUnet), our approach demonstrated improvements of 1.02, 1.04, and 1.41% in Dice Coefficient (DSC), Intersection over Union (IOU), and the 95th percentile Hausdorff Distance (HD95), respectively, for heart structure segmentation. For the two great vessels, the enhancements were 1.05, 1.07, and 1.42% in these metrics. The outcomes on the public dataset affirm the efficacy of our proposed segmentation method. Precise segmentation of the entire heart and great vessels can significantly aid in the diagnosis and treatment of CHD, underscoring the clinical relevance of our findings.

Autodelineation of Treatment Target Volume for Radiation Therapy Using Large Language Model-Aided Multimodal Learning

PurposeArtificial intelligence (AI)-aided methods have made significant progress in the auto-delineation of normal tissues. However, these approaches struggle with the auto-contouring of radiotherapy target volume. Our goal is to model the delineation of target volume as a clinical decision-making problem, resolved by leveraging large language model-aided multimodal learning approaches. Methods and MaterialsA vision-language model, termed Medformer, has been developed, employing the hierarchical vision transformer as its backbone, and incorporating large language models to extract text-rich features. The contextually embedded linguistic features are seamlessly integrated into visual features for language-aware visual encoding through the visual language attention module. Metrics, including Dice similarity coefficient (DSC), intersection over union (IOU), and 95th percentile Hausdorff distance (HD95), were used to quantitatively evaluate the performance of our model. The evaluation was conducted on an in-house prostate cancer dataset and a public oropharyngeal carcinoma (OPC) dataset, totaling 668 subjects. ResultsOur Medformer achieved a DSC of 0.81 ± 0.10 versus 0.72 ± 0.10, IOU of 0.73 ± 0.12 versus 0.65 ± 0.09, and HD95 of 9.86 ± 9.77 mm versus 19.13 ± 12.96 mm for delineation of gross tumor volume (GTV) on the prostate cancer dataset. Similarly, on the OPC dataset, it achieved a DSC of 0.77 ± 0.11 versus 0.72 ± 0.09, IOU of 0.70 ± 0.09 versus 0.65 ± 0.07, and HD95 of 7.52 ± 4.8 mm versus 13.63 ± 7.13 mm, representing significant improvements (p < 0.05). For delineating the clinical target volume (CTV), Medformer achieved a DSC of 0.91 ± 0.04, IOU of 0.85 ± 0.05, and HD95 of 2.98 ± 1.60 mm, comparable to other state-of-the-art algorithms. ConclusionsAuto-delineation of the treatment target based on multimodal learning outperforms conventional approaches that rely purely on visual features. Our method could be adopted into routine practice to rapidly contour CTV/GTV.

DeMambaNet: Deformable Convolution and Mamba Integration Network for High-Precision Segmentation of Ambiguously Defined Dental Radicular Boundaries.

The incorporation of automatic segmentation methodologies into dental X-ray images refined the paradigms of clinical diagnostics and therapeutic planning by facilitating meticulous, pixel-level articulation of both dental structures and proximate tissues. This underpins the pillars of early pathological detection and meticulous disease progression monitoring. Nonetheless, conventional segmentation frameworks often encounter significant setbacks attributable to the intrinsic limitations of X-ray imaging, including compromised image fidelity, obscured delineation of structural boundaries, and the intricate anatomical structures of dental constituents such as pulp, enamel, and dentin. To surmount these impediments, we propose the Deformable Convolution and Mamba Integration Network, an innovative 2D dental X-ray image segmentation architecture, which amalgamates a Coalescent Structural Deformable Encoder, a Cognitively-Optimized Semantic Enhance Module, and a Hierarchical Convergence Decoder. Collectively, these components bolster the management of multi-scale global features, fortify the stability of feature representation, and refine the amalgamation of feature vectors. A comparative assessment against 14 baselines underscores its efficacy, registering a 0.95% enhancement in the Dice Coefficient and a diminution of the 95th percentile Hausdorff Distance to 7.494.

Estimating Surgical Urethral Length on Intraoperative Robot-Assisted Prostatectomy Images Using Artificial Intelligence Anatomy Recognition.

Objective: To construct a convolutional neural network (CNN) model that can recognize and delineate anatomic structures on intraoperative video frames of robot-assisted radical prostatectomy (RARP) and to use these annotations to predict the surgical urethral length (SUL). Background: Urethral dissection during RARP impacts patient urinary incontinence (UI) outcomes, and requires extensive training. Large differences exist between incontinence outcomes of different urologists and hospitals. Also, surgeon experience and education are critical toward optimal outcomes. Therefore, new approaches are warranted. SUL is associated with UI. Artificial intelligence (AI) surgical image segmentation using a CNN could automate SUL estimation and contribute toward future AI-assisted RARP and surgeon guidance. Methods: Eighty-eight intraoperative RARP videos between June 2009 and September 2014 were collected from a single center. Two hundred sixty-four frames were annotated according to prostate, urethra, ligated plexus, and catheter. Thirty annotated images from different RARP videos were used as a test data set. The dice (similarity) coefficient (DSC) and 95th percentile Hausdorff distance (Hd95) were used to determine model performance. SUL was calculated using the catheter as a reference. Results: The DSC of the best performing model were 0.735 and 0.755 for the catheter and urethra classes, respectively, with a Hd95 of 29.27 and 72.62, respectively. The model performed moderately on the ligated plexus and prostate. The predicted SUL showed a mean difference of 0.64 to 1.86 mm difference vs human annotators, but with significant deviation (standard deviation = 3.28-3.56). Conclusion: This study shows that an AI image segmentation model can predict vital structures during RARP urethral dissection with moderate to fair accuracy. SUL estimation derived from it showed large deviations and outliers when compared with human annotators, but with a small mean difference (<2 mm). This is a promising development for further research on AI-assisted RARP.

SymTC: A symbiotic Transformer-CNN net for instance segmentation of lumbar spine MRI

Intervertebral disc disease, a prevalent ailment, frequently leads to intermittent or persistent low back pain, and diagnosing and assessing of this disease rely on accurate measurement of vertebral bone and intervertebral disc geometries from lumbar MR images. Deep neural network (DNN) models may assist clinicians with more efficient image segmentation of individual instances (discs and vertebrae) of the lumbar spine in an automated way, which is termed as instance image segmentation. In this work, we proposed SymTC, an innovative lumbar spine MR image segmentation model that combines the strengths of Transformer and Convolutional Neural Network (CNN). Specifically, we designed a parallel dual-path architecture to merge CNN layers and Transformer layers, and we integrated a novel position embedding into the self-attention module of Transformer, enhancing the utilization of positional information for more accurate segmentation. To further improve model performance, we introduced a new data synthesis technique to create synthetic yet realistic MR image dataset, named SSMSpine, which is made publicly available. We evaluated our SymTC and the other 16 representative image segmentation models on our private in-house dataset and public SSMSpine dataset, using two metrics, Dice Similarity Coefficient and the 95th percentile Hausdorff Distance. The results indicate that SymTC surpasses the other 16 methods, achieving the highest dice score of 96.169 % for segmenting vertebral bones and intervertebral discs on the SSMSpine dataset. The SymTC code and SSMSpine dataset are publicly available at https://github.com/jiasongchen/SymTC.

Automated segmentation and volume prediction in pediatric Wilms’ tumor CT using nnu-net

BackgroundRadiologic volumetric evaluation of Wilms’ tumor (WT) is an important indicator to guide treatment decisions. However, due to the heterogeneity of the tumors, radiologists have main-guard differences in diagnosis that can lead to misdiagnosis and poor treatment. The aim of this study was to explore whether CT-based outlining of WT foci can be automated using deep learning.MethodsWe included CT intravenous phase images of 105 patients with WT and double-blind outlining of lesions by two radiologists. Then, we trained an automatic segmentation model using nnUnet. The Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95) were used to assess the performance. Next, we optimized the automatic segmentation results based on the ratio of the three-dimensional diameter of the lesion to improve the performance of volumetric assessment.ResultsThe DSC and HD95 was 0.83 ± 0.22 and 10.50 ± 8.98 mm. The absolute difference and percentage difference in tumor size was 72.27 ± 134.84 cm3 and 21.08% ± 30.46%. After optimization according to our method, it decreased to 40.22 ± 96.06 cm3 and 10.16% ± 9.70%.ConclusionWe introduce a novel method that enhances the accuracy of predicting WT volume by integrating AI automated outlining and 3D tumor diameters. This approach surpasses the accuracy of using AI outcomes alone and has the potential to enhance the clinical evaluation of pediatric patients with WT. By intertwining AI outcomes with clinical data, this method becomes more interpretive and offers promising applications beyond Wilms tumor, extending to other pediatric diseases.

Impact of bias field correction on 0.35 T pelvic MR images: evaluation on generative adversarial network-based OARs' auto-segmentation and visual grading assessment.

Magnetic resonance imaging (MRI)-guided radiotherapy enables adaptive treatment plans based on daily anatomical changes and accurate organ visualization. However, the bias field artifact can compromise image quality, affecting diagnostic accuracy and quantitative analyses. This study aims to assess the impact of bias field correction on 0.35 T pelvis MRIs by evaluating clinical anatomy visualization and generative adversarial network (GAN) auto-segmentation performance. 3D simulation MRIs from 60 prostate cancer patients treated on MR-Linac (0.35 T) were collected and preprocessed with the N4ITK algorithm for bias field correction. A 3D GAN architecture was trained, validated, and tested on 40, 10, and 10 patients, respectively, to auto-segment the organs at risk (OARs) rectum and bladder. The GAN was trained and evaluated either with the original or the bias-corrected MRIs. The Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95th) were computed for the segmented volumes of each patient. The Wilcoxon signed-rank test assessed the statistical difference of the metrics within OARs, both with and without bias field correction. Five radiation oncologists blindly scored 22 randomly chosen patients in terms of overall image quality and visibility of boundaries (prostate, rectum, bladder, seminal vesicles) of the original and bias-corrected MRIs. Bennett's S score and Fleiss' kappa were used to assess the pairwise interrater agreement and the interrater agreement among all the observers, respectively. In the test set, the GAN trained and evaluated on original and bias-corrected MRIs showed DSC/HD95th of 0.92/5.63mm and 0.92/5.91mm for the bladder and 0.84/10.61mm and 0.83/9.71mm for the rectum. No statistical differences in the distribution of the evaluation metrics were found neither for the bladder (DSC: p = 0.07; HD95th: p = 0.35) nor for the rectum (DSC: p = 0.32; HD95th: p = 0.63). From the clinical visual grading assessment, the bias-corrected MRI resulted mostly in either no change or an improvement of the image quality and visualization of the organs' boundaries compared with the original MRI. The bias field correction did not improve the anatomy visualization from a clinical point of view and the OARs' auto-segmentation outputs generated by the GAN.

Transfer learning for auto-segmentation of 17 organs-at-risk in the head and neck: Bridging the gap between institutional and public datasets.

Auto-segmentation of organs-at-risk (OARs) in the head and neck (HN) on computed tomography (CT) images is a time-consuming component of the radiation therapy pipeline that suffers from inter-observer variability. Deep learning (DL) has shown state-of-the-art results in CT auto-segmentation, with larger and more diverse datasets showing better segmentation performance. Institutional CT auto-segmentation datasets have been small historically (n<50) due to the time required for manual curation of images and anatomical labels. Recently, large public CT auto-segmentation datasets (n>1000 aggregated) have become available through online repositories such as The Cancer Imaging Archive. Transfer learning is a technique applied when training samples are scarce, but a large dataset from a closely related domain is available. The purpose of this study was to investigate whether a large public dataset could be used in place of an institutional dataset (n>500), or to augment performance via transfer learning, when building HN OAR auto-segmentation models for institutional use. Auto-segmentation models were trained on a large public dataset (public models) and a smaller institutional dataset (institutional models). The public models were fine-tuned on the institutional dataset using transfer learning (transfer models). We assessed both public model generalizability and transfer model performance by comparison with institutional models. Additionally, the effect of institutional dataset size on both transfer and institutional models was investigated. All DL models used a high-resolution, two-stage architecture based on the popular 3D U-Net. Model performance was evaluated using five geometric measures: the dice similarity coefficient (DSC), surface DSC, 95th percentile Hausdorff distance, mean surface distance (MSD), and added path length. For a small subset of OARs (left/right optic nerve, spinal cord, left submandibular), the public models performed significantly better (p<0.05) than, or showed no significant difference to, the institutional models under most of the metrics examined. For the remaining OARs, the public models were inferior to the institutional models, although performance differences were small (DSC≤0.03, MSD<0.5mm) for seven OARs (brainstem, left/right lens, left/right parotid, mandible, right submandibular). The transfer models performed significantly better than the institutional models for seven OARs (brainstem, right lens, left/right optic nerve, left/right parotid, spinal cord) with a small margin of improvement (DSC≤0.02, MSD<0.4mm). When numbers of institutional training samples were limited, public and transfer models outperformed the institutional models for most OARs (brainstem, left/right lens, left/right optic nerve, left/right parotid, spinal cord, and left/right submandibular). Training auto-segmentation models with public data alone was suitable for a small number of OARs. Using only public data incurred a small performance deficit for most other OARs, when compared with institutional data alone, but may be preferable over time-consuming curation of a large institutional dataset. When a large institutional dataset was available, transfer learning with models pretrained on a large public dataset provided a modest performance improvement for several OARs. When numbers of institutional samples were limited, using the public dataset alone, or as a pretrained model, was beneficial for most OARs.

NnU-Net versus mesh growing algorithm as a tool for the robust and timely segmentation of neurosurgical 3D images in contrast-enhanced T1 MRI scans

PurposeThis study evaluates the nnU-Net for segmenting brain, skin, tumors, and ventricles in contrast-enhanced T1 (T1CE) images, benchmarking it against an established mesh growing algorithm (MGA).MethodsWe used 67 retrospectively collected annotated single-center T1CE brain scans for training models for brain, skin, tumor, and ventricle segmentation. An additional 32 scans from two centers were used test performance compared to that of the MGA. The performance was measured using the Dice-Sørensen coefficient (DSC), intersection over union (IoU), 95th percentile Hausdorff distance (HD95), and average symmetric surface distance (ASSD) metrics, with time to segment also compared.ResultsThe nnU-Net models significantly outperformed the MGA (p < 0.0125) with a median brain segmentation DSC of 0.971 [95CI: 0.945–0.979], skin: 0.997 [95CI: 0.984–0.999], tumor: 0.926 [95CI: 0.508–0.968], and ventricles: 0.910 [95CI: 0.812–0.968]. Compared to the MGA’s median DSC for brain: 0.936 [95CI: 0.890, 0.958], skin: 0.991 [95CI: 0.964, 0.996], tumor: 0.723 [95CI: 0.000–0.926], and ventricles: 0.856 [95CI: 0.216–0.916]. NnU-Net performance between centers did not significantly differ except for the skin segmentations Additionally, the nnU-Net models were faster (mean: 1139 s [95CI: 685.0–1616]) than the MGA (mean: 2851 s [95CI: 1482–6246]).ConclusionsThe nnU-Net is a fast, reliable tool for creating automatic deep learning-based segmentation pipelines, reducing the need for extensive manual tuning and iteration. The models are able to achieve this performance despite a modestly sized training set. The ability to create high-quality segmentations in a short timespan can prove invaluable in neurosurgical settings.

Auto-segmentation of pelvic organs at risk on 0.35T MRI using 2D and 3D Generative Adversarial Network models

PurposeManual recontouring of targets and Organs At Risk (OARs) is a time-consuming and operator-dependent task. We explored the potential of Generative Adversarial Networks (GAN) to auto-segment the rectum, bladder and femoral heads on 0.35T MRIs to accelerate the online MRI-guided-Radiotherapy (MRIgRT) workflow. Methods3D planning MRIs from 60 prostate cancer patients treated with 0.35T MR-Linac were collected. A 3D GAN architecture and its equivalent 2D version were trained, validated and tested on 40, 10 and 10 patients respectively. The volumetric Dice Similarity Coefficient (DSC) and 95th percentile Hausdorff Distance (HD95th) were computed against expert drawn ground-truth delineations. The networks were also validated on an independent external dataset of 16 patients. ResultsIn the internal test set, the 3D and 2D GANs showed DSC/HD95th of 0.83/9.72 mm and 0.81/10.65 mm for the rectum, 0.92/5.91 mm and 0.85/15.72 mm for the bladder, and 0.94/3.62 mm and 0.90/9.49 mm for the femoral heads. In the external test set, the performance was 0.74/31.13 mm and 0.72/25.07 mm for the rectum, 0.92/9.46 mm and 0.88/11.28 mm for the bladder, and 0.89/7.00 mm and 0.88/10.06 mm for the femoral heads. The 3D and 2D GANs required on average 1.44 s and 6.59 s respectively to generate the OARs’ volumetric segmentation for a single patient. ConclusionsThe proposed 3D GAN auto-segments pelvic OARs with high accuracy on 0.35T, in both the internal and the external test sets, outperforming its 2D equivalent in both segmentation robustness and volume generation time.

Multitask Weakly Supervised Generative Network for MR-US Registration.

Registering pre-operative modalities, such as magnetic resonance imaging or computed tomography, to ultrasound images is crucial for guiding clinicians during surgeries and biopsies. Recently, deep-learning approaches have been proposed to increase the speed and accuracy of this registration problem. However, all of these approaches need expensive supervision from the ultrasound domain. In this work, we propose a multitask generative framework that needs weak supervision only from the pre-operative imaging domain during training. To perform a deformable registration, the proposed framework translates a magnetic resonance image to the ultrasound domain while preserving the structural content. To demonstrate the efficacy of the proposed method, we tackle the registration problem of pre-operative 3D MR to transrectal ultrasonography images as necessary for targeted prostate biopsies. We use an in-house dataset of 600 patients, divided into 540 for training, 30 for validation, and the remaining for testing. An expert manually segmented the prostate in both modalities for validation and test sets to assess the performance of our framework. The proposed framework achieves a 3.58 mm target registration error on the expert-selected landmarks, 89.2% in the Dice score, and 1.81 mm 95th percentile Hausdorff distance on the prostate masks in the test set. Our experiments demonstrate that the proposed generative model successfully translates magnetic resonance images into the ultrasound domain. The translated image contains the structural content and fine details due to an ultrasound-specific two-path design of the generative model. The proposed framework enables training learning-based registration methods while only weak supervision from the pre-operative domain is available.

Multi-Center Fetal Brain Tissue Annotation (FeTA) Challenge 2022 Results.

Segmentation is a critical step in analyzing the developing human fetal brain. There have been vast improvements in automatic segmentation methods in the past several years, and the Fetal Brain Tissue Annotation (FeTA) Challenge 2021 helped to establish an excellent standard of fetal brain segmentation. However, FeTA 2021 was a single center study, limiting real-world clinical applicability and acceptance. The multi-center FeTA Challenge 2022 focused on advancing the generalizability of fetal brain segmentation algorithms for magnetic resonance imaging (MRI). In FeTA 2022, the training dataset contained images and corresponding manually annotated multi-class labels from two imaging centers, and the testing data contained images from these two centers as well as two additional unseen centers. The multi-center data included different MR scanners, imaging parameters, and fetal brain super-resolution algorithms applied. 16 teams participated and 17 algorithms were evaluated. Here, the challenge results are presented, focusing on the generalizability of the submissions. Both in- and out-of-domain, the white matter and ventricles were segmented with the highest accuracy (Top Dice scores: 0.89, 0.87 respectively), while the most challenging structure remains the grey matter (Top Dice score: 0.75) due to anatomical complexity. The top 5 average Dices scores ranged from 0.81-0.82, the top 5 average 95th percentile Hausdorff distance values ranged from 2.3-2.5mm, and the top 5 volumetric similarity scores ranged from 0.90-0.92. The FeTA Challenge 2022 was able to successfully evaluate and advance generalizability of multi-class fetal brain tissue segmentation algorithms for MRI and it continues to benchmark new algorithms.

Lesion segmentation on 18F-fluciclovine PET/CT images using deep learning.

A novel radiotracer, 18F-fluciclovine (anti-3-18F-FACBC), has been demonstrated to be associated with significantly improved survival when it is used in PET/CT imaging to guide postprostatectomy salvage radiotherapy for prostate cancer. We aimed to investigate the feasibility of using a deep learning method to automatically detect and segment lesions on 18F-fluciclovine PET/CT images. We retrospectively identified 84 patients who are enrolled in Arm B of the Emory Molecular Prostate Imaging for Radiotherapy Enhancement (EMPIRE-1) trial. All 84 patients had prostate adenocarcinoma and underwent prostatectomy and 18F-fluciclovine PET/CT imaging with lesions identified and delineated by physicians. Three different neural networks with increasing levels of complexity (U-net, Cascaded U-net, and a cascaded detection segmentation network) were trained and tested on the 84 patients with a fivefold cross-validation strategy and a hold-out test, using manual contours as the ground truth. We also investigated using both PET and CT or using PET only as input to the neural network. Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95), center-of-mass distance (CMD), and volume difference (VD) were used to quantify the quality of segmentation results against ground truth contours provided by physicians. All three deep learning methods were able to detect 144/155 lesions and 153/155 lesions successfully when PET+CT and PET only, respectively, served as input. Quantitative results demonstrated that the neural network with the best performance was able to segment lesions with an average DSC of 0.68 ± 0.15 and HD95 of 4 ± 2mm. The center of mass of the segmented contours deviated from physician contours by approximately 2mm on average, and the volume difference was less than 1 cc. The novel network proposed by us achieves the best performance compared to current networks. The addition of CT as input to the neural network contributed to more cases of failure (DSC = 0), and among those cases of DSC > 0, it was shown to produce no statistically significant difference with the use of only PET as input for our proposed method. Quantitative results demonstrated the feasibility of the deep learning methods in automatically segmenting lesions on 18F-fluciclovine PET/CT images. This indicates the great potential of 18F-fluciclovine PET/CT combined with deep learning for providing a second check in identifying lesions as well as saving time and effort for physicians in contouring.

Deep learning-based clinical-radiomics nomogram for preoperative prediction of lymph node metastasis in patients with rectal cancer: a two-center study.

Precise preoperative evaluation of lymph node metastasis (LNM) is crucial for ensuring effective treatment for rectal cancer (RC). This research aims to develop a clinical-radiomics nomogram based on deep learning techniques, preoperative magnetic resonance imaging (MRI) and clinical characteristics, enabling the accurate prediction of LNM in RC. Between January 2017 and May 2023, a total of 519 rectal cancer cases confirmed by pathological examination were retrospectively recruited from two tertiary hospitals. A total of 253 consecutive individuals were selected from Center I to create an automated MRI segmentation technique utilizing deep learning algorithms. The performance of the model was evaluated using the dice similarity coefficient (DSC), the 95th percentile Hausdorff distance (HD95), and the average surface distance (ASD). Subsequently, two external validation cohorts were established: one comprising 178 patients from center I (EVC1) and another consisting of 88 patients from center II (EVC2). The automatic segmentation provided radiomics features, which were then used to create a Radscore. A predictive nomogram integrating the Radscore and clinical parameters was constructed using multivariate logistic regression. Receiver operating characteristic (ROC) curve analysis and decision curve analysis (DCA) were employed to evaluate the discrimination capabilities of the Radscore, nomogram, and subjective evaluation model, respectively. The mean DSC, HD95 and ASD were 0.857 ± 0.041, 2.186 ± 0.956, and 0.562 ± 0.194 mm, respectively. The nomogram, which incorporates MR T-stage, CEA, CA19-9, and Radscore, exhibited a higher area under the ROC curve (AUC) compared to the Radscore and subjective evaluation in the training set (0.921 vs. 0.903 vs. 0.662). Similarly, in both external validation sets, the nomogram demonstrated a higher AUC than the Radscore and subjective evaluation (0.908 vs. 0.735 vs. 0.640, and 0.884 vs. 0.802 vs. 0.734). The application of the deep learning method enables efficient automatic segmentation. The clinical-radiomics nomogram, utilizing preoperative MRI and automatic segmentation, proves to be an accurate method for assessing LNM in RC. This approach has the potential to enhance clinical decision-making and improve patient care. Research registry, identifier 9158, https://www.researchregistry.com/browse-the-registry#home/registrationdetails/648e813efffa4e0028022796/.