Mean Surface Distance Research Articles

Deep learning for automated contouring of neurovascular structures on magnetic resonance imaging for prostate cancer patients.

Manual contouring of neurovascular structures on prostate magnetic resonance imaging (MRI) is labor-intensive and prone to considerable interrater disagreement. Our aim is to contour neurovascular structures automatically on prostate MRI by deep learning (DL) to improve workflow and interrater agreement. Segmentation of neurovascular structures was performed on pre-treatment 3.0T MRI data of 131 prostate cancer patients (training [n=105] and testing [n=26]). The neurovascular structures include the penile bulb (PB), corpora cavernosa (CCs), internal pudendal arteries (IPAs), and neurovascular bundles (NVBs). Two DL networks, nnU-Net and DeepMedic, were trained for auto-contouring on prostate MRI and evaluated using volumetric Dice similarity coefficient (DSC), mean surface distances (MSD), Hausdorff distances, and surface DSC. Three radiation oncologists evaluated the DL-generated contours and performed corrections when necessary. Interrater agreement was assessed and the time required for manual correction was recorded. nnU-Net achieved a median DSC of 0.92 (IQR: 0.90-0.93) for the PB, 0.90 (IQR: 0.86-0.92) for the CCs, 0.79 (IQR: 0.77-0.83) for the IPAs, and 0.77 (IQR: 0.72-0.81) for the NVBs, which outperformed DeepMedic for each structure (p<0.03). nnU-Net showed a median MSD of 0.24mm for the IPAs and 0.71mm for the NVBs. The median interrater DSC ranged from 0.93 to 1.00, with the majority of cases (68.9%) requiring manual correction times under two minutes. DL enables reliable auto-contouring of neurovascular structures on pre-treatment MRI data, easing the clinical workflow in neurovascular-sparing MR-guided radiotherapy.

Comparison of automated segmentation techniques for magnetic resonance images of the prostate

BackgroundContouring of anatomical regions is a crucial step in the medical workflow and is both time-consuming and prone to intra- and inter-observer variability. This study compares different strategies for automatic segmentation of the prostate in T2-weighted MRIs.MethodsThis study included 100 patients diagnosed with prostate adenocarcinoma who had undergone multi-parametric MRI and prostatectomy. From the T2-weighted MR images, ground truth segmentation masks were established by consensus from two expert radiologists. The prostate was then automatically contoured with six different methods: (1) a multi-atlas algorithm, (2) a proprietary algorithm in the Syngo.Via medical imaging software, and four deep learning models: (3) a V-net trained from scratch, (4) a pre-trained 2D U-net, (5) a GAN extension of the 2D U-net, and (6) a segmentation-adapted EfficientDet architecture. The resulting segmentations were compared and scored against the ground truth masks with one 70/30 and one 50/50 train/test data split. We also analyzed the association between segmentation performance and clinical variables.ResultsThe best performing method was the adapted EfficientDet (model 6), achieving a mean Dice coefficient of 0.914, a mean absolute volume difference of 5.9%, a mean surface distance (MSD) of 1.93 pixels, and a mean 95th percentile Hausdorff distance of 3.77 pixels. The deep learning models were less prone to serious errors (0.854 minimum Dice and 4.02 maximum MSD), and no significant relationship was found between segmentation performance and clinical variables.ConclusionsDeep learning-based segmentation techniques can consistently achieve Dice coefficients of 0.9 or above with as few as 50 training patients, regardless of architectural archetype. The atlas-based and Syngo.via methods found in commercial clinical software performed significantly worse (0.855-0.887 Dice).

Compensation cycle consistent generative adversarial networks (Comp-GAN) for synthetic CT generation from MR scans with truncated anatomy.

MR scans used in radiotherapy can be partially truncated due to the limited field of view (FOV), affecting dose calculation accuracy in MR-based radiation treatment planning. We proposed a novel Compensation-cycleGAN (Comp-cycleGAN) by modifying the cycle-consistent generative adversarial network (cycleGAN), to simultaneously create synthetic CT (sCT) images and compensate the missing anatomy from the truncated MR images. Computed tomography (CT) and T1 MR images with complete anatomy of 79 head-and-neck patients were used for this study. The original MR images were manually cropped 10-25mm off at the posterior head to simulate clinically truncated MR images. Fifteen patients were randomly chosen for testing and the rest of the patients were used for model training and validation. Both the truncated and original MR images were used in the Comp-cycleGAN training stage, which enables the model to compensate for the missing anatomy by learning the relationship between the truncation and known structures. After the model was trained, sCT images with complete anatomy can be generated by feeding only the truncated MR images into the model. In addition, the external body contours acquired from the CT images with full anatomy could be an optional input for the proposed method to leverage the additional information of the actual body shape for each test patient. The mean absolute error (MAE) of Hounsfield units (HU), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM) were calculated between sCT and real CT images to quantify the overall sCT performance. To further evaluate the shape accuracy, we generated the external body contours for sCT and original MR images with full anatomy. The Dice similarity coefficient (DSC) and mean surface distance (MSD) were calculated between the body contours of sCT and original MR images for the truncation region to assess the anatomy compensation accuracy. The average MAE, PSNR, and SSIM calculated over test patients were 93.1 HU/91.3 HU, 26.5dB/27.4dB, and 0.94/0.94 for the proposed Comp-cycleGAN models trained without/with body-contour information, respectively. These results were comparable with those obtained from the cycleGAN model which is trained and tested on full-anatomy MR images, indicating the high quality of the sCT generated from truncated MR images by the proposed method. Within the truncated region, the mean DSC and MSD were 0.85/0.89 and 1.3/0.7mm for the proposed Comp-cycleGAN models trained without/with body contour information, demonstrating good performance in compensating the truncated anatomy. We developed a novel Comp-cycleGAN model that can effectively create sCT with complete anatomy compensation from truncated MR images, which could potentially benefit the MRI-based treatment planning.

A Surgical Robotic System for Osteoporotic Hip Augmentation: System Development and Experimental Evaluation.

Minimally-invasive Osteoporotic Hip Augmentation (OHA) by injecting bone cement is a potential treatment option to reduce the risk of hip fracture. This treatment can significantly benefit from computer-assisted planning and execution system to optimize the pattern of cement injection. We present a novel robotic system for the execution of OHA that consists of a 6-DOF robotic arm and integrated drilling and injection component. The minimally-invasive procedure is performed by registering the robot and preoperative images to the surgical scene using multiview image-based 2D/3D registration with no external fiducial attached to the body. The performance of the system is evaluated through experimental sawbone studies as well as cadaveric experiments with intact soft tissues. In the cadaver experiments, distance errors of 3.28mm and 2.64mm for entry and target points and orientation error of 2.30° are calculated. Moreover, the mean surface distance error of 2.13mm with translational error of 4.47mm is reported between injected and planned cement profiles. The experimental results demonstrate the first application of the proposed Robot-Assisted combined Drilling and Injection System (RADIS), incorporating biomechanical planning and intraoperative fiducial-less 2D/3D registration on human cadavers with intact soft tissues.

Towards interactive deep-learning for tumour segmentation in head and neck cancer radiotherapy.

With deep-learning, gross tumour volume (GTV) auto-segmentation has substantially been improved, but still substantial manual corrections are needed. With interactive deep-learning (iDL), manual corrections can be used to update a deep-learning tool while delineating, minimising the input to achieve acceptable segmentations. We present an iDL tool for GTV segmentation that took annotated slices as input and simulated its performance on a head and neck cancer (HNC) dataset. Multimodal image data of 204 HNC patients with clinical tumour and lymph node GTV delineations were used. A baseline convolutional neural network (CNN) was trained (n=107 training, n=22 validation) and tested (n=24). Subsequently, user input was simulated on initial test set by replacing one or more of predicted slices with ground truth delineation, followed by re-training the CNN. The objective was to optimise re-training parameters and simulate slice selection scenarios while limiting annotations to maximally-five slices. The remaining 51 patients were used as an independent test set, where Dice similarity coefficient (DSC), mean surface distance (MSD), and 95% Hausdorff distance (HD95%) were assessed at baseline and after every update. Median segmentation accuracy at baseline was DSC=0.65, MSD=4.3mm, HD95% = 17.5mm. Updating CNN using three slices equally sampled from the craniocaudal axis of the GTV in the first round, followed by two rounds of annotating one extra slice, gave the best results. The accuracy improved to DSC=0.82, MSD=1.6mm, HD95% = 4.8mm. Every CNN update took 30s. The presented iDL tool achieved substantial segmentation improvement with only five annotated slices.

Brain segmentation in patients with perinatal arterial ischemic stroke

BackgroundPerinatal arterial ischemic stroke (PAIS) is associated with adverse neurological outcomes. Quantification of ischemic lesions and consequent brain development in newborn infants relies on labor-intensive manual assessment of brain tissues and ischemic lesions. Hence, we propose an automatic method utilizing convolutional neural networks (CNNs) to segment brain tissues and ischemic lesions in MRI scans of infants suffering from PAIS. Materials and MethodsThis single-center retrospective study included 115 patients with PAIS that underwent MRI after the stroke onset (baseline) and after three months (follow-up). Nine baseline and 12 follow-up MRI scans were manually annotated to provide reference segmentations (white matter, gray matter, basal ganglia and thalami, brainstem, ventricles, extra-ventricular cerebrospinal fluid, and cerebellum, and additionally on the baseline scans the ischemic lesions). Two CNNs were trained to perform automatic segmentation on the baseline and follow-up MRIs, respectively. Automatic segmentations were quantitatively evaluated using the Dice coefficient (DC) and the mean surface distance (MSD). Volumetric agreement between segmentations that were manually and automatically obtained was computed. Moreover, the scan quality and automatic segmentations were qualitatively evaluated in a larger set of MRIs without manual annotation by two experts. In addition, the scan quality was qualitatively evaluated in these scans to establish its impact on the automatic segmentation performance. ResultsAutomatic brain tissue segmentation led to a DC and MSD between 0.78–0.92 and 0.18–1.08 mm for baseline, and between 0.88–0.95 and 0.10–0.58 mm for follow-up scans, respectively. For the ischemic lesions at baseline the DC and MSD were between 0.72–0.86 and 1.23–2.18 mm, respectively. Volumetric measurements indicated limited oversegmentation of the extra-ventricular cerebrospinal fluid in both the follow-up and baseline scans, oversegmentation of the ischemic lesions in the left hemisphere, and undersegmentation of the ischemic lesions in the right hemisphere. In scans without imaging artifacts, brain tissue segmentation was graded as excellent in more than 85% and 91% of cases, respectively for the baseline and follow-up scans. For the ischemic lesions at baseline, this was in 61% of cases. ConclusionsAutomatic segmentation of brain tissue and ischemic lesions in MRI scans of patients with PAIS is feasible. The method may allow evaluation of the brain development and efficacy of treatment in large datasets.

Proof-of-Concept Study of a 3-D Ultrasound Scanner Used for Ankle Joint Assessment.

Joint arthropathies often require continuous monitoring of the joint condition, typically performed using magnetic resonance (MR) or ultrasound (US) imaging. US imaging is often the preferred screening or diagnostic tool as it is fast and inexpensive. However, conventional 2-D US has limited capability to compare imaging results between examinations because of its operator dependence and challenges related to repeat imaging in the same location and orientation. Comparison between several imaging sessions is crucial to assess the interval progression of joint conditions. We propose a novel 3-D US scanner for ankle joint assessment that can partially overcome these issues by enabling 3-D imaging. Here, we (i) present the design of the 3-D US ankle scanner system, (ii) validate the geometric reconstruction accuracy of the system, (iii) provide preliminary images of healthy volunteer ankles and (iv) compare 3-D US imaging results with MR imaging. The 3-D ankle scanner consists of a tub filled with water, a linear US probe attached to the wall of the tub and a motorized unit that rotates the US probe 360° around the center of the tub. As the probe rotates, a 3-D US image is formed of the ankle of the patient positioned in the middle of the tub. US probe height, angle and distance from the tub center can be adjusted. The reconstruction accuracy of the system was validated in each of the coordinate directions at different probe angles using two test phantoms. A phantom consisting of numerous Ø200-µm nylon threads with known spacing and a metal rod with machined grooves was used for validation in the horizontal and vertical directions, respectively. The volumetric reconstruction accuracy validation was performed by imaging an agar phantom with two embedded spheres of known volumes and comparing the segmented sphere volume and surface area with the expected. Three-dimensional US and MR images of both ankles of five healthy volunteers were acquired. Distal tibia and proximal talus were segmented in both imaging modalities and the surfaces of these segmentations were compared using the 95% Hausdorff and mean surface distances. The observed mean linear measurement error in all the coordinate directions and over several probe angles was 2.98%. The mean measured volumetric measurement error was 3.45%. The volunteer study revealed useful features for joint assessment present in the 3-D ankle scanner images, such as joint spacing, distal tibia and proximal talus. The mean 95% Hausdorff and mean surface distances between segmentations in 3-D US and MR images were 5.68 ± 0.83 and 2.01 ± 0.30 mm, respectively. In this proof-of-concept study, the 3-D US ankle scanner enabled visualization of the ankle joint features that are useful for joint assessment.

Quality assurance assessment of intra-acquisition diffusion-weighted and T2-weighted magnetic resonance imaging registration and contour propagation for head and neck cancer radiotherapy.

Adequate image registration of anatomical and functional magnetic resonance imaging (MRI) scans is necessary for MR-guided head and neck cancer (HNC) adaptive radiotherapy planning. Despite the quantitative capabilities of diffusion-weighted imaging (DWI) MRI for treatment plan adaptation, geometric distortion remains a considerable limitation. Therefore, we systematically investigated various deformable image registration (DIR) methods to co-register DWI and T2-weighted (T2W) images. We compared three commercial (ADMIRE, Velocity, Raystation) and three open-source (Elastix with default settings [Elastix Default], Elastix with parameter set 23 [Elastix 23], Demons) post-acquisition DIR methods applied to T2W and DWI MRI images acquired during the same imaging session in twenty immobilized HNC patients. In addition, we used the non-registered images (None) as a control comparator. Ground-truth segmentations of radiotherapy structures (tumour and organs at risk) were generated by a physician expert on both image sequences. For each registration approach, structures were propagated from T2W to DWI images. These propagated structures were then compared with ground-truth DWI structures using the Dice similarity coefficient and mean surface distance. 19 left submandibular glands, 18 right submandibular glands, 20left parotid glands, 20 right parotid glands, 20 spinal cords, and 12 tumours were delineated. Most DIR methods took <30s to execute per case, with the exception of Elastix 23 which took ∼458s to execute per case. ADMIRE and Elastix 23 demonstrated improved performance over None for all metrics and structures (Bonferroni-corrected p<0.05), while the other methods did not. Moreover, ADMIRE and Elastix 23 significantly improved performance in individual and pooled analysis compared to all other methods. The ADMIRE DIR method offers improved geometric performance with reasonable execution time so should be favoured for registering T2W and DWI images acquired during the same scan session in HNC patients. These results are important to ensure the appropriate selection of registration strategies for MR-guided radiotherapy.

Interpretable cardiac anatomy modeling using variational mesh autoencoders.

Cardiac anatomy and function vary considerably across the human population with important implications for clinical diagnosis and treatment planning. Consequently, many computer-based approaches have been developed to capture this variability for a wide range of applications, including explainable cardiac disease detection and prediction, dimensionality reduction, cardiac shape analysis, and the generation of virtual heart populations. In this work, we propose a variational mesh autoencoder (mesh VAE) as a novel geometric deep learning approach to model such population-wide variations in cardiac shapes. It embeds multi-scale graph convolutions and mesh pooling layers in a hierarchical VAE framework to enable direct processing of surface mesh representations of the cardiac anatomy in an efficient manner. The proposed mesh VAE achieves low reconstruction errors on a dataset of 3D cardiac meshes from over 1,000 patients with acute myocardial infarction, with mean surface distances between input and reconstructed meshes below the underlying image resolution. We also find that it outperforms a voxelgrid-based deep learning benchmark in terms of both mean surface distance and Hausdorff distance while requiring considerably less memory. Furthermore, we explore the quality and interpretability of the mesh VAE's latent space and showcase its ability to improve the prediction of major adverse cardiac events over a clinical benchmark. Finally, we investigate the method's ability to generate realistic virtual populations of cardiac anatomies and find good alignment between the synthesized and gold standard mesh populations in terms of multiple clinical metrics.

Abdomen CT multi-organ segmentation using token-based MLP-Mixer.

Manual contouring is very labor-intensive, time-consuming, and subject to intra- and inter-observer variability. An automated deep learning approach to fast and accurate contouring and segmentation is desirable during radiotherapy treatment planning. This work investigates an efficient deep-learning-based segmentation algorithm in abdomen computed tomography (CT) to facilitate radiation treatment planning. In this work, we propose a novel deep-learning model utilizing U-shaped multi-layer perceptron mixer (MLP-Mixer) and convolutional neural network (CNN) for multi-organ segmentation in abdomen CT images. The proposed model has a similar structure to V-net, while a proposed MLP-Convolutional block replaces each convolutional block. The MLP-Convolutional block consists of three components: an early convolutional block for local features extraction and feature resampling, a token-based MLP-Mixer layer for capturing global features with high efficiency, and a token projector for pixel-level detail recovery. We evaluate our proposed network using: (1) an institutional dataset with 60 patient cases and (2) a public dataset (BCTV) with 30 patient cases. The network performance was quantitatively evaluated in three domains: (1) volume similarity between the ground truth contours and the network predictions using the Dice score coefficient (DSC), sensitivity, and precision; (2) surface similarity using Hausdorff distance (HD), mean surface distance (MSD) and residual mean square distance (RMS); and (3) the computational complexity reported by the number of network parameters, training time, and inference time. The performance of the proposed network is compared with other state-of-the-art networks. In the institutional dataset, the proposed network achieved the following volume similarity measures when averaged over all organs: DSC = 0.912, sensitivity = 0.917, precision = 0.917, average surface similarities were HD = 11.95 mm, MSD = 1.90 mm, RMS = 3.86 mm. The proposed network achieved DSC = 0.786 and HD = 9.04 mm on the public dataset. The network also shows statistically significant improvement, which is evaluated by a two-tailed Wilcoxon Mann-Whitney U test, on right lung (MSD where the maximum p-value is 0.001), spinal cord (sensitivity, precision, HD, RMSD where p-value ranges from 0.001 to 0.039), and stomach (DSC where the maximum p-value is 0.01) over all other competing networks. On the public dataset, the network report statistically significant improvement, which is shown by the Wilcoxon Mann-Whitney test, on pancreas (HD where the maximum p-value is 0.006), left (HD where the maximum p-value is 0.022) and right adrenal glands (DSC where the maximum p-value is 0.026). In both datasets, the proposed method can generate contours in less than 5 s. Overall, the proposed MLP-Vnet demonstrates comparable or better performance than competing methods with much lower memory complexity and higher speed. The proposed MLP-Vnet demonstrates superior segmentation performance, in terms of accuracy and efficiency, relative to state-of-the-art methods. This reliable and efficient method demonstrates potential to streamline clinical workflows in abdominal radiotherapy, which may be especially important for online adaptive treatments.

Automatic segmentation of esophageal gross tumor volume in 18F-FDG PET/CT images via GloD-LoATUNet

Background and objectiveFor esophageal squamous cell carcinoma, radiotherapy is one of the primary treatments. During the planning before radiotherapy, the intractable task is to precisely delineate the esophageal gross tumor volume (GTV) on medical images. In current clinical practice, the manual delineation suffers from high intra- and inter-rater variability, while also exhausting the oncologists on a treadmill. There is an urgent demand for effective computer-aided automatic segmentation methods. To this end, we designed a novel deep network, dubbed as GloD-LoATUNet. MethodsGloD-LoATUNet follows the effective U-shape structure. On the contractile path, the global deformable dense attention transformer (GloDAT), local attention transformer (LoAT), and convolution blocks are integrated to model long-range dependencies and localized information. On the center bridge and the expanding path, convolution blocks are adopted to upsample the extracted representations for pixel-wise semantic prediction. Between the peer-to-peer counterparts, enhanced skip connections are built to compensate for the lost spatial information and dependencies. By exploiting complementary strengths of the GloDAT, LoAT, and convolution, GloD-LoATUNet has remarkable representation learning capabilities, performing well in the prediction of the small and variable esophageal GTV. ResultsThe proposed approach was validated in the clinical positron emission tomography/computed tomography (PET/CT) cohort. For 4 different data partitions, we report the Dice similarity coefficient (DSC), Hausdorff distance (HD), and Mean surface distance (MSD) as: 0.83±0.13, 4.88±9.16 mm, and 1.40±4.11 mm; 0.84±0.12, 6.89±12.04 mm, and 1.18±3.02 mm; 0.84±0.13, 3.89±7.64 mm, and 1.28±3.68 mm; 0.86±0.09, 3.71±4.79 mm, and 0.90±0.37 mm; respectively. The predicted contours present a desirable consistency with the ground truth. ConclusionsThe inspiring results confirm the accuracy and generalizability of the proposed model, demonstrating the potential for automatic segmentation of esophageal GTV in clinical practice.

Customizable landmark-based field aperture design for automated whole-brain radiotherapy treatment planning.

To develop and evaluate an automated whole-brain radiotherapy (WBRT) treatment planning pipeline with a deep learning-based auto-contouring and customizable landmark-based field aperture design. The pipeline consisted of the following steps: (1) Auto-contour normal structures on computed tomography scans and digitally reconstructed radiographs using deep learning techniques, (2) locate the landmark structures using the beam's-eye-view, (3) generate field apertures based on eight different landmark rules addressing different clinical purposes and physician preferences. Two parallel approaches for generating field apertures were developed for quality control. The performance of the generated field shapes and dose distributions were compared with the original clinical plans. The clinical acceptability of the plans was assessed by five radiation oncologists from four hospitals. The performance of the generated field apertures was evaluated by the Hausdorff distance (HD) and mean surface distance (MSD) from 182 patients' field apertures used in the clinic. The average HD and MSD for the generated field apertures were 16±7 and 7±3mm for the first approach, respectively, and 17±7 and 7±3mm, respectively, for the second approach. The differences regarding HD and MSD between the first and the second approaches were 1±2 and 1±3mm, respectively. A clinical review of the field aperture design, conducted using 30 patients, achieved a 100% acceptance rate for both the first and second approaches, and the plan review achieved a 100% acceptance rate for the first approach and a 93% acceptance rate for the second approach. The average acceptance rate for meeting lens dosimetric recommendations was 80% (left lens) and 77% (right lens) for the first approach, and 70% (both left and right lenses) for the second approach, compared with 50% (left lens) and 53% (right lens) for the clinical plans. This study provided an automated pipeline with two field aperture generation approaches to automatically generate WBRT treatment plans. Both quantitative and qualitative evaluations demonstrated that our novel pipeline was comparable with the original clinical plans.

DeepHeartCT: A fully automatic artificial intelligence hybrid framework based on convolutional neural network and multi-atlas segmentation for multi-structure cardiac computed tomography angiography image segmentation.

Cardiac computed tomography angiography (CTA) is an emerging imaging modality for assessing coronary artery as well as various cardiovascular structures. Recently, deep learning (DL) methods have been successfully applied to many applications of medical image analysis including cardiac CTA structure segmentation. However, DL requires a large amounts of data and high-quality labels for training which can be burdensome to obtain due to its labor-intensive nature. In this study, we aim to develop a fully automatic artificial intelligence (AI) system, named DeepHeartCT, for accurate and rapid cardiac CTA segmentation based on DL. The proposed system was trained using a large clinical dataset with computer-generated labels to segment various cardiovascular structures including left and right ventricles (LV, RV), left and right atria (LA, RA), and LV myocardium (LVM). This new system was trained directly using high-quality computer labels generated from our previously developed multi-atlas based AI system. In addition, a reverse ranking strategy was proposed to assess the segmentation quality in the absence of manual reference labels. This strategy allowed the new framework to assemble optimal computer-generated labels from a large dataset for effective training of a deep convolutional neural network (CNN). A large clinical cardiac CTA studies (n = 1,064) were used to train and validate our framework. The trained model was then tested on another independent dataset with manual labels (n = 60). The Dice score, Hausdorff distance and mean surface distance were used to quantify the segmentation accuracy. The proposed DeepHeartCT framework yields a high median Dice score of 0.90 [interquartile range (IQR), 0.90-0.91], a low median Hausdorff distance of 7 mm (IQR, 4-15 mm) and a low mean surface distance of 0.80 mm (IQR, 0.57-1.29 mm) across all segmented structures. An additional experiment was conducted to evaluate the proposed DL-based AI framework trained with a small vs. large dataset. The results show our framework also performed well when trained on a small optimal training dataset (n = 110) with a significantly reduced training time. These results demonstrated that the proposed DeepHeartCT framework provides accurate and rapid cardiac CTA segmentation that can be readily generalized for handling large-scale medical imaging applications.

CAN3D: Fast 3D medical image segmentation via compact context aggregation.

Direct automatic segmentation of objects in 3D medical imaging, such as magnetic resonance (MR) imaging, is challenging as it often involves accurately identifying multiple individual structures with complex geometries within a large volume under investigation. Most deep learning approaches address these challenges by enhancing their learning capability through a substantial increase in trainable parameters within their models. An increased model complexity will incur high computational costs and large memory requirements unsuitable for real-time implementation on standard clinical workstations, as clinical imaging systems typically have low-end computer hardware with limited memory and CPU resources only. This paper presents a compact convolutional neural network (CAN3D) designed specifically for clinical workstations and allows the segmentation of large 3D Magnetic Resonance (MR) images in real-time. The proposed CAN3D has a shallow memory footprint to reduce the number of model parameters and computer memory required for state-of-the-art performance and maintain data integrity by directly processing large full-size 3D image input volumes with no patches required. The proposed architecture significantly reduces computational costs, especially for inference using the CPU. We also develop a novel loss function with extra shape constraints to improve segmentation accuracy for imbalanced classes in 3D MR images. Compared to state-of-the-art approaches (U-Net3D, improved U-Net3D and V-Net), CAN3D reduced the number of parameters up to two orders of magnitude and achieved much faster inference, up to 5 times when predicting with a standard commercial CPU (instead of GPU). For the open-access OAI-ZIB knee MR dataset, in comparison with manual segmentation, CAN3D achieved Dice coefficient values of (mean = 0.87±0.02 and 0.85±0.04) with mean surface distance errors (mean = 0.36±0.32mm and 0.29±0.10mm) for imbalanced classes such as (femoral and tibial) cartilage volumes respectively when training volume-wise under only 12G video memory. Similarly, CAN3D demonstrated high accuracy and efficiency on a pelvis 3D MR imaging dataset for prostate cancer consisting of 211 examinations with expert manual semantic labels (bladder, body, bone, rectum, prostate) now released publicly for scientific use as part of this work.

Male pelvic multi-organ segmentation using token-based transformer Vnet

Objective. This work aims to develop an automated segmentation method for the prostate and its surrounding organs-at-risk in pelvic computed tomography to facilitate prostate radiation treatment planning. Approach. In this work, we propose a novel deep learning algorithm combining a U-shaped convolutional neural network (CNN) and vision transformer (VIT) for multi-organ (i.e. bladder, prostate, rectum, left and right femoral heads) segmentation in male pelvic CT images. The U-shaped model consists of three components: a CNN-based encoder for local feature extraction, a token-based VIT for capturing global dependencies from the CNN features, and a CNN-based decoder for predicting the segmentation outcome from the VIT’s output. The novelty of our network is a token-based multi-head self-attention mechanism used in the transformer, which encourages long-range dependencies and forwards informative high-resolution feature maps from the encoder to the decoder. In addition, a knowledge distillation strategy is deployed to further enhance the learning capability of the proposed network. Main results. We evaluated the network using: (1) a dataset collected from 94 patients with prostate cancer; (2) and a public dataset CT-ORG. A quantitative evaluation of the proposed network’s performance was performed on each organ based on (1) volume similarity between the segmented contours and ground truth using Dice score, segmentation sensitivity, and precision, (2) surface similarity evaluated by Hausdorff distance (HD), mean surface distance (MSD) and residual mean square distance (RMS), (3) and percentage volume difference (PVD). The performance was then compared against other state-of-art methods. Average volume similarity measures obtained by the network overall organs were Dice score = 0.91, sensitivity = 0.90, precision = 0.92, average surface similarities were HD = 3.78 mm, MSD = 1.24 mm, RMS = 2.03 mm; average percentage volume difference was PVD = 9.9% on the first dataset. The network also obtained Dice score = 0.93, sensitivity = 0.93, precision = 0.93, average surface similarities were HD = 5.82 mm, MSD = 1.16 mm, RMS = 1.24 mm; average percentage volume difference was PVD = 6.6% on the CT-ORG dataset. Significance. In summary, we propose a token-based transformer network with knowledge distillation for multi-organ segmentation using CT images. This method provides accurate and reliable segmentation results for each organ using CT imaging, facilitating the prostate radiation clinical workflow.

Automated lung tumor delineation on positron emission tomography/computed tomography via a hybrid regional network.

Multimodality positron emission tomography/computed tomography (PET/CT) imaging combines the anatomical information of CT with the functional information of PET. In the diagnosis and treatment of many cancers, such as non-small cell lung cancer (NSCLC), PET/CT imaging allows more accurate delineation of tumor or involved lymph nodes for radiation planning. In this paper, we propose a hybrid regional network method of automatically segmenting lung tumors from PET/CT images. The hybrid regional network architecture synthesizes the functional and anatomical information from the two image modalities, whereas the mask regional convolutional neural network (R-CNN) and scoring fine-tune the regional location and quality of the output segmentation. This model consists of five major subnetworks, that is, a dual feature representation network (DFRN), a regional proposal network (RPN), a specific tumor-wise R-CNN, a mask-Net, and a score head. Given a PET/CT image as inputs, the DFRN extracts feature maps from the PET and CT images. Then, the RPN and R-CNN work together to localize lung tumors and reduce the image size and feature map size by removing irrelevant regions. The mask-Net is used to segment tumor within a volume-of-interest (VOI) with a score head evaluating the segmentation performed by the mask-Net. Finally, the segmented tumor within the VOI was mapped back to the volumetric coordinate system based on the location information derived via the RPN and R-CNN. We trained, validated, and tested the proposed neural network using 100 PET/CT images of patients with NSCLC. A fivefold cross-validation study was performed. The segmentation was evaluated with two indicators: (1) multiple metrics, including the Dice similarity coefficient, Jacard, 95th percentile Hausdorff distance, mean surface distance (MSD), residual mean square distance, and center-of-mass distance; (2) Bland-Altman analysis and volumetric Pearson correlation analysis. In fivefold cross-validation, this method achieved Dice and MSD of 0.84±0.15 and 1.38±2.2mm, respectively. A new PET/CT can be segmented in 1s by this model. External validation on The Cancer Imaging Archive dataset (63 PET/CT images) indicates that the proposed model has superior performance compared to other methods. The proposed method shows great promise to automatically delineate NSCLC tumors on PET/CT images, thereby allowing for a more streamlined clinical workflow that is faster and reduces physician effort.

Ultrasound Image Guided and Mixed Reality-Based Surgical System With Real-Time Soft Tissue Deformation Computing for Robotic Cervical Pedicle Screw Placement.

Cervical pedicle screw (CPS) placement surgery remains technically demanding due to the complicated anatomy with neurovascular structures. State-of-the-art surgical navigation or robotic systems still suffer from the problem of hand-eye coordination and soft tissue deformation. In this study, we aim at tracking the intraoperative soft tissue deformation and constructing a virtual-physical fusion surgical scene, and integrating them into the robotic system for CPS placement surgery. Firstly, we propose a real-time deformation computation method based on the prior shape model and intraoperative partial information acquired from ultrasound images. According to the generated posterior shape, the structure representation of deformed target tissue gets updated continuously. Secondly, a hand tremble compensation method is proposed to improve the accuracy and robustness of the virtual-physical calibration procedure, and a mixed reality based surgical scene is further constructed for CPS placement surgery. Thirdly, we integrate the soft tissue deformation method and virtual-physical fusion method into our previously proposed surgical robotic system, and the surgical workflow for CPS placement surgery is introduced. We conducted phantom and animal experiments to evaluate the feasibility and accuracy of the proposed system. Our system yielded a mean surface distance error of 1.52 ± 0.43 mm for soft tissue deformation computing, and an average distance deviation of 1.04 ± 0.27 mm for CPS placement. Results demonstrate that our system involves tremendous clinical application potential. Our proposed system promotes the efficiency and safety of the CPS placement surgery.

Intracranial vessel wall segmentation with deep learning using a novel tiered loss function incorporating class inclusion.

To develop an automated vessel wall segmentation method on T1-weighted intracranial vessel wall magnetic resonance images, with a focus on modeling the inclusion relation between the inner and outer boundaries of the vesselwall. We propose a novel method that estimates the inner and outer vessel wall boundaries simultaneously, using a network with a single output channel resembling the level-set function height. The network is driven by a unique tiered loss that accounts for data fidelity of the lumen and vessel wall classes and a length regularization to encourage boundary smoothness. Implemented with a 2.5D UNet with a ResNet backbone, the proposed method achieved Dice similarity coefficients (DSC) in 2D of 0.925 ± 0.048, 0.786 ± 0.084, Hausdorff distance (HD) of 0.286 ± 0.436, 0.345 ± 0.419 mm, and mean surface distance (MSD) of 0.083 ± 0.037 and 0.103 ± 0.032 mm for the lumen and vessel wall, respectively, on a test set; compared favorably to a baseline UNet model that achieved DSC 0.924 ± 0.047, 0.794 ± 0.082, HD 0.298 ± 0.477, 0.394 ± 0.431 mm, and MSD 0.087 ± 0.056, 0.119 ± 0.059 mm. Our vessel wall segmentation method achieved substantial improvement in morphological integrity and accuracy compared to benchmark methods. The proposed method provides a systematic approach to model the inclusion morphology and incorporate it into an optimization infrastructure. It can be applied to any application where inclusion exists among a (sub)set of classes to be segmented. Improved feasibility in result morphology promises better support for clinical quantification anddecision.

Clinical evaluation of deep learning–based clinical target volume three-channel auto-segmentation algorithm for adaptive radiotherapy in cervical cancer

ObjectivesAccurate contouring of the clinical target volume (CTV) is a key element of radiotherapy in cervical cancer. We validated a novel deep learning (DL)-based auto-segmentation algorithm for CTVs in cervical cancer called the three-channel adaptive auto-segmentation network (TCAS).MethodsA total of 107 cases were collected and contoured by senior radiation oncologists (ROs). Each case consisted of the following: (1) contrast-enhanced CT scan for positioning, (2) the related CTV, (3) multiple plain CT scans during treatment and (4) the related CTV. After registration between (1) and (3) for the same patient, the aligned image and CTV were generated. Method 1 is rigid registration, method 2 is deformable registration, and the aligned CTV is seen as the result. Method 3 is rigid registration and TCAS, method 4 is deformable registration and TCAS, and the result is generated by a DL-based method.ResultsFrom the 107 cases, 15 pairs were selected as the test set. The dice similarity coefficient (DSC) of method 1 was 0.8155 ± 0.0368; the DSC of method 2 was 0.8277 ± 0.0315; the DSCs of method 3 and 4 were 0.8914 ± 0.0294 and 0.8921 ± 0.0231, respectively. The mean surface distance and Hausdorff distance of methods 3 and 4 were markedly better than those of method 1 and 2.ConclusionsThe TCAS achieved comparable accuracy to the manual delineation performed by senior ROs and was significantly better than direct registration.

Robustness and reproducibility of an artificial intelligence-assisted online segmentation and adaptive planning process for online adaptive radiation therapy.

Clinical implementation of online adaptive radiation therapy requires initial and ongoing performance assessment of the underlying auto‐segmentation and adaptive planning algorithms, although a straightforward and efficient process for this in phantom is lacking. The purpose of this work was to investigate robustness and repeatability of the artificial intelligence‐assisted online segmentation and adaptive planning process on the Varian Ethos adaptive platform, and to develop an end‐to‐end test strategy for online adaptive radiation therapy. Five synthetic deformations were generated and applied to a computed tomography image of an anthropomorphic pelvis phantom, and reference treatment plans were generated from each of the resulting deformed images. The undeformed phantom was repeatedly imaged, and the online adaptive process was performed including auto‐segmentation, review and manual correction of contours, and adaptive plan creation. One adaptive fractions in five different deformation scenarios were performed. The manually corrected contours had a high degree of consistency (> 93% Dice similarity coefficient and < 1.0 mm mean surface distance) across repeated fractions, with no significant variation across the synthetic deformation instance except for bowel (p = 0.026, one‐way ANOVA). Adaptive treatment plans also resulted in highly consistent dose–volume values for targets and organs at risk. A straightforward and efficient process was developed and used to quantify a set of organ specific contouring and dosimetric action levels to help establish uncertainty bounds for an end‐to‐end test on the Varian Ethos system.