3D U-Net Research Articles

Edit Temporal-Consistent Videos with Image Diffusion Model

Large-scale text-to-image (T2I) diffusion models have been extended for text-guided video editing, yielding impressive zero-shot video editing performance. Nonetheless, the generated videos usually show spatial irregularities and temporal inconsistencies as the temporal characteristics of videos have not been faithfully modeled. In this paper, we propose an elegant yet effective Temporal-Consistent Video Editing (TCVE) method to mitigate the temporal inconsistency challenge for robust text-guided video editing. In addition to the utilization of a pretrained T2I 2D Unet for spatial content manipulation, we establish a dedicated temporal Unet architecture to faithfully capture the temporal coherence of the input video sequences. Furthermore, to establish coherence and interrelation between the spatial-focused and temporal-focused components, a cohesive spatial-temporal modeling unit is formulated. This unit effectively interconnects the temporal Unet with the pretrained 2D Unet, thereby enhancing the temporal consistency of the generated videos while preserving the capacity for video content manipulation. Quantitative experimental results and visualization results demonstrate that TCVE achieves state-of-the-art performance in both video temporal consistency and video editing capability, surpassing existing benchmarks in the field. Codes are released at https://github.com/mdswyz/TCVE .

Enhancing brain tumor segmentation in MRI images: A hybrid approach using UNet, attention mechanisms, and transformers

Accurate brain tumor segmentation in MRI images is crucial for effective treatment planning and monitoring. Traditional methods often encounter challenges due to the complexity and variability of tumor shapes and textures. Consequently, there is a growing need for automated solutions to assist healthcare professionals in segmentation tasks, improving efficiency and reducing workload. This study introduces an innovative method for accurately segmenting brain tumors in MRI images by employing a refined 3D UNet model integrated with a Transformer. The goal is to leverage self-attention mechanisms to enhance segmentation capabilities. The proposed model combines Contextual Transformer (CoT) and Double Attention (DA) architectures. CoT is extended to a 3D format and integrated with the baseline model to exploit intricate contextual details in MRI images. DA blocks in skip connections aggregate and distribute long-range features, emphasizing inter-dependencies within an expanded spatial scope. Experimental results demonstrate superior segmentation performance compared to current state-of-the-art methods. With its ability to accurately segment and delineate tumors in 3D, our segmentation model promises to be a powerful tool for medical image processing and performance optimization, saving time for healthcare professionals and healthcare systems.

Rapid identification of time-frequency domain gravitational wave signals from binary black holes using deep learning* *Supported by the National SKA Program of China (2022SKA0110200, 2022SKA0110203), the National Natural Science Foundation of China (12473001, 11975072, 11875102, 11835009), and the National 111 Project (B16009)

Recent developments in deep learning techniques have provided alternative and complementary approaches to the traditional matched-filtering methods for identifying gravitational wave (GW) signals. The rapid and accurate identification of GW signals is crucial to the advancement of GW physics and multi-messenger astronomy, particularly considering the upcoming fourth and fifth observing runs of LIGO-Virgo-KAGRA. In this study, we used the 2D U-Net algorithm to identify time-frequency domain GW signals from stellar-mass binary black hole (BBH) mergers. We simulated BBH mergers with component masses ranging from 7 to 50 and accounted for the LIGO detector noise. We found that the GW events in the first and second observation runs could all be clearly and rapidly identified. For the third observing run, approximately 80% of the GW events could be identified. In contrast to traditional convolutional neural networks, the U-Net algorithm can output time-frequency domain signal images corresponding to probabilities, providing a more intuitive analysis. In conclusion, the U-Net algorithm can rapidly identify the time-frequency domain GW signals from BBH mergers.

Performance enhancement of deep learning based solutions for pharyngeal airway space segmentation on MRI scans

The automatic segmentation of the pharyngeal airway space has many potential medical use, one of which is to help facilitate the creation of the Tubingen Palatal Plate. Therefore, it is of great importance to understand which methods are suitable for this task. Here, neural network based solutions available in the literature are compared to find the best methods. Neural network models were chosen to encompass a diverse landscape. Some models were taken from the general semantic segmentation literature, while others were taken from the medical or pharyngeal airway space segmentation literature. Some models are convolutional neural networks, while others are transformer-based model or a mix of both convolutional and transformer-based model. These models include 2d/3d U-Net, Deeplabv3, YOLOv8, Swinv2 UNETR, SegFormer, and 3D MRU-Net. Furthermore, additional strategies to enhance performance were also considered. These strategies consisted of training two separate networks in multiple stages as well leveraging unlabeled data to pretrain the neural networks before fine-tuning them on the labeled data. It was found that out of all the models considered here, the 2d U-Net performed the best achieving an average dice score of 0.9180 ± 0.0111. Out of all the strategies to enhance performance, only two strategies improve the actual results but only by a small margin. Therefore, these strategies can be consider if a small increase in performance is desired from the 2d U-Net at the expense of computational resource.

A hierarchical deep learning approach for diagnosing impacted canine-induced root resorption via cone-beam computed tomography

ObjectivesCanine-induced root resorption (CIRR) is caused by impacted canines and CBCT images have shown to be more accurate in diagnosing CIRR than panoramic and periapical radiographs with the reported AUCs being 0.95, 0.49, and 0.57, respectively. The aim of this study was to use deep learning to automatically evaluate the diagnosis of CIRR in maxillary incisors using CBCT images.MethodsA total of 50 cone beam computed tomography (CBCT) images and 176 incisors were selected for the present study. The maxillary incisors were manually segmented and labeled from the CBCT images by two independent radiologists as either healthy or affected by root resorption induced by the impacted canines. We used five different strategies for training the model: (A) classification using 3D ResNet50 (Baseline), (B) classification of the segmented masks using the outcome of a 3D U-Net pretrained on the 3D MNIST, (C) training a 3D U-Net for the segmentation task and use its outputs for classification, (D) pretraining a 3D U-Net for the segmentation and transfer of the model, and (E) pretraining a 3D U-Net for the segmentation and fine-tuning the model with only the model encoder. The segmentation models were evaluated using the mean intersection over union (mIoU) and Dice coefficient (DSC). The classification models were evaluated in terms of classification accuracy, precision, recall, and F1 score.ResultsThe segmentation model achieved a mean intersection over union (mIoU) of 0.641 and a DSC of 0.901, indicating good performance in segmenting the tooth structures from the CBCT images. For the main classification task of detecting CIRR, Model C (classification of the segmented masks using 3D ResNet) and Model E (pretraining on segmentation followed by fine-tuning for classification) performed the best, both achieving 82% classification accuracy and 0.62 F1-scores on the test set. These results demonstrate the effectiveness of the proposed hierarchical, data-efficient deep learning approaches in improving the accuracy of automated CIRR diagnosis from limited CBCT data compared to the 3D ResNet baseline model.ConclusionThe proposed approaches are effective at improving the accuracy of classification tasks and are helpful when the diagnosis is based on the volume and boundaries of an object. While the study demonstrated promising results, future studies with larger sample size are required to validate the effectiveness of the proposed method in enhancing the medical image classification tasks.

Multi-Stage Feature Fusion of Multispectral and SAR Satellite Images for Seasonal Crop-Type Mapping at Regional Scale Using an Adapted 3D U-Net Model

Multi-Stage Feature Fusion of Multispectral and SAR Satellite Images for Seasonal Crop-Type Mapping at Regional Scale Using an Adapted 3D U-Net Model

An end-to-end deep learning pipeline to derive blood input with partial volume corrections for automated parametric brain PET mapping

Dynamic 2-[18F] fluoro-2-deoxy-D-glucose positron emission tomography (dFDG-PET) for human brain imaging has considerable clinical potential, yet its utilization remains limited. A key challenge in the quantitative analysis of dFDG-PET is characterizing a patient-specific blood input function, traditionally reliant on invasive arterial blood sampling. This research introduces a novel approach employing non-invasive deep learning model-based computations from the internal carotid arteries (ICA) with partial volume (PV) corrections, thereby eliminating the need for invasive arterial sampling. We present an end-to-end pipeline incorporating a 3D U-Net based ICA-net for ICA segmentation, alongside a Recurrent Neural Network (RNN) based MCIF-net for the derivation of a model-corrected blood input function (MCIF) with PV corrections. The developed 3D U-Net and RNN was trained and validated using a 5-fold cross-validation approach on 50 human brain FDG PET scans. The ICA-net achieved an average Dice score of 82.18% and an Intersection over Union of 68.54% across all tested scans. Furthermore, the MCIF-net exhibited a minimal root mean squared error of 0.0052. The application of this pipeline to ground truth data for dFDG-PET brain scans resulted in the precise localization of seizure onset regions, which contributed to a successful clinical outcome, with the patient achieving a seizure-free state after treatment. These results underscore the efficacy of the ICA-net and MCIF-net deep learning pipeline in learning the ICA structure’s distribution and automating MCIF computation with PV corrections. This advancement marks a significant leap in non-invasive neuroimaging.

Development of deep learning-based novel auto-segmentation for the prostatic urethra on planning CT images for prostate cancer radiotherapy.

Urinary toxicities are one of the serious complications of radiotherapy for prostate cancer, and dose-volume histogram of prostatic urethra has been associated with such toxicities in previous reports. Previous research has focused on estimating the prostatic urethra, which is difficult to delineate in CT images; however, these studies, which are limited in number, mainly focused on cases undergoing brachytherapy uses low-dose-rate sources and do not involve external beam radiation therapy (EBRT). In this study, we aimed to develop a deep learning-based method of determining the position of the prostatic urethra in patients eligible for EBRT. We used contour data from 430 patients with localized prostate cancer. In all cases, a urethral catheter was placed when planning CT to identify the prostatic urethra. We used 2D and 3D U-Net segmentation models. The input images included the bladder and prostate, while the output images focused on the prostatic urethra. The 2D model determined the prostate's position based on results from both coronal and sagittal directions. Evaluation metrics included the average distance between centerlines. The average centerline distances for the 2D and 3D models were 2.07 ± 0.87mm and 2.05 ± 0.92mm, respectively. Increasing the number of cases while maintaining equivalent accuracy as we did in this study suggests the potential for high generalization performance and the feasibility of using deep learning technology for estimating the position of the prostatic urethra.

First implementation of artificial intelligence empowered all-in-one radiotherapy workflow for nasopharyngeal carcinoma.

134 Background: To establish an artificial intelligence (AI)-empowered All-In-One (AIO) workflow for providing one-stop radiotherapy to patients with nasopharyngeal carcinoma (NPC), and evaluate its feasibility and performance. Methods: The NPC-specific AIO workflow operates on a CT-integrated linac and integrates AI contouring and AI planning modules. A 3D U-net was employed for contours generation and 3D dose distribution. Knowledge-based planning approach and a protocol-based approach were used for plan optimization. MRI datasets with contours of primary gross tumors, CT datasets with contours of targets and organs at risk, and radiotherapy treatment planning datasets were collected from 877, 139, and 243 patients, respectively, for the training, validation, and testing of AI contouring and planning models. The NPC-specific AIO workflow underwent real-world clinical testing with an end-to-end evaluation involving 120 patients with NPC. The technical characteristics, implementation procedures and performance of AIO workflow were described and evaluated. Results: From March 2022 to October 2023, 120 patients with newly-diagnosed, non-metastatic NPC underwent AIO workflow, and 117 patients completed it in a median time of 23.2 min (range: 16.3 to 45.8 min). Median translation motions were 0.2 mm (CT scan to planning approval) and 0.1 mm (during beam delivery). AI-generated contours required little revision for clinical target volume (CTV1) and organs at risk (OARs), minor revision for cervical lymph nodes (GTVn) and clinical target volume (CTV2), with median Dice similarity coefficients (DSCs) of 0.98 and 0.94, respectively; more revision for GTVp, with a median DSC of 0.84. Of the 117 AI-generated plannings, 108 (92.3%) passed after initial auto-optimization. Volumes receiving ≥ 100% of the prescribed dose for all planning target volumes was ≥ 97.8%. Mean dosimetric parameters met dose constraints for all OARs except the thyroid and submandibulars. One patient died from acute renal failure after 10 fractions of radiotherapy. At week 12 after radiotherapy completion, 115 patients achieved complete response;1 had stable disease. Grade 3-4 acute effects were reported by 54 of 115 patients (36%). Conclusions: We successfully established an AI-empowered AIO workflow for initial radiotherapy of NPC, confirming its feasibility and safety for clinical application.

Anatomy-constrained synthesis for spleen segmentation improvement in unpaired mouse micro-CT scans with 3D CycleGAN

Objective. Auto-segmentation in mouse micro-CT enhances the efficiency and consistency of preclinical experiments but often struggles with low-native-contrast and morphologically complex organs, such as the spleen, resulting in poor segmentation performance. While CT contrast agents can improve organ conspicuity, their use complicates experimental protocols and reduces feasibility. We developed a 3D Cycle Generative Adversarial Network (CycleGAN) incorporating anatomy-constrained U-Net models to leverage contrast-enhanced CT (CECT) insights to improve unenhanced native CT (NACT) segmentation. Approach. We employed a standard CycleGAN with an anatomical loss function to synthesize virtual CECT images from unpaired NACT scans at two different resolutions. Prior to training, two U-Nets were trained to automatically segment six major organs in NACT and CECT datasets, respectively. These pretrained 3D U-Nets were integrated during the CycleGAN training, segmenting synthetic images, and comparing them against ground truth annotations. The compound loss within the CycleGAN maintained anatomical fidelity. Full image processing was achieved for low-resolution datasets, while high-resolution datasets employed a patch-based method due to GPU memory constraints. Automated segmentation was applied to original NACT and synthetic CECT scans to evaluate CycleGAN performance using the Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD95p). Main results. High-resolution scans showed improved auto-segmentation, with an average DSC increase from 0.728 to 0.773 and a reduced HD95p from 1.19 mm to 0.94 mm. Low-resolution scans benefited more from synthetic contrast, showing a DSC increase from 0.586 to 0.682 and an HD95p reduction from 3.46 mm to 1.24 mm. Significance. Implementing CycleGAN to synthesize CECT scans substantially improved the visibility of the mouse spleen, leading to more precise auto-segmentation. This approach shows the potential in preclinical imaging studies where contrast agent use is impractical.

Improving 3D dose prediction for breast radiotherapy using novel glowing masks and gradient-weighted loss functions.

The quality of treatment plans for breast cancer can vary greatly. This variation could be reduced by using dose prediction to automate treatment planning. Our work investigates novel methods for training deep-learning models that are capable of producing high-quality dose predictions for breast cancer treatment planning. The goal of this work was to compare the performance impact of two novel techniques for deep learning dose prediction models for tangent field treatments for breast cancer. The first technique, a "glowing" mask algorithm, encodes the distance from a contour into each voxel in a mask. The second, a gradient-weighted mean squared error (MSE) loss function, emphasizes the error in high-dose gradient regions in the predicted image. Four 3D U-Net deep learning models were trained using the planning CT and contours of the heart, lung, and tumor bed as inputs. The dataset consisted of 305 treatment plans split into 213/46/46 training/validation/test sets using a 70/15/15% split. We compared the impact of novel "glowing" anatomical mask inputs and a novel gradient-weighted MSE loss function to their standard counterparts, binary anatomical masks, and MSE loss, using an ablation study methodology. To assess performance, we examined the mean error and mean absolute error (ME/MAE) in dose across all within-body voxels, the error in mean dose to heart, ipsilateral lung, and tumor bed, dice similarity coefficient (DSC) across isodose volumes defined by 0%-100% prescribed dose thresholds, and gamma analysis (3%/3mm). The combination of novel glowing masks and gradient weighted loss function yielded the best-performing model in this study. This model resulted in a mean ME of 0.40%, MAE of 2.70%, an error in mean dose to heart and lung of -0.10 and 0.01Gy, and an error in mean dose to the tumor bed of -0.01%. The median DSC at 50/95/100% isodose levels were 0.91/0.87/0.82. The mean 3D gamma pass rate (3%/3mm) was 93%. This study found the combination of novel anatomical mask inputs and loss function for dose prediction resulted in superior performance to their standard counterparts. These results have important implications for the field of radiotherapy dose prediction, as the methods used here can be easily incorporated into many other dose prediction models for other treatment sites. Additionally, this dose prediction model for breast radiotherapy has sufficient performance to be used in an automated planning pipeline for tangent field radiotherapy and has the major benefit of not requiring a PTV for accurate dose prediction.

Enhanced 3D dose prediction for hypofractionated SRS (gamma knife radiosurgery) in brain tumor using cascaded-deep-supervised convolutional neural network.

Gamma Knife radiosurgery (GKRS) is a well-established technique in radiation therapy (RT) for treating small-size brain tumors. It administers highly concentrated doses during each treatment fraction, with even minor dose errors posing a significant risk of causing severe damage to healthy tissues. It underscores the critical need for precise and meticulous precision in GKRS. However, the planning process for GKRS is complex and time-consuming, heavily reliant on the expertise of medical physicists. Incorporating deep learning approaches for GKRS dose prediction can reduce this dependency, improve planning efficiency and homogeneity, streamline clinical workflows, and reduce patient lagging times. Despite this, precise Gamma Knife plan dose distribution prediction using existing models remains a significant challenge. The complexity stems from the intricate nature of dose distributions, subtle contrasts in CT scans, and the interdependence of dosimetric metrics. To overcome these challenges, we have developed a "Cascaded-Deep-Supervised" Convolutional Neural Network (CDS-CNN) that employs a hybrid-weighted optimization scheme. Our innovative method incorporates multi-level deep supervision and a strategic sequential multi-network training approach. It enables the extraction of intra-slice and inter-slice features, leading to more realistic dose predictions with additional contextual information. CDS-CNN was trained and evaluated using data from 105 brain cancer patients who underwent GKRS treatment, with 85 cases used for training and 20 for testing. Quantitative assessments and statistical analyses demonstrated high consistency between the predicted dose distributions and the reference doses from the treatment planning system (TPS). The 3D overall gamma passing rates (GPRs) reached 97.15% ± 1.36% (3mm/3%, 10% threshold), surpassing the previous best performance by 2.53% using the 3D Dense U-Net model. When evaluated against more stringent criteria (2mm/3%, 10% threshold, and 1mm/3%, 10% threshold), the overall GPRs still achieved 96.53% ± 1.08% and 95.03% ± 1.18%. Furthermore, the average target coverage (TC) was 98.33% ± 1.16%, dose selectivity (DS) was 0.57 ± 0.10, gradient index (GI) was 2.69 ± 0.30, and homogeneity index (HI) was 1.79 ± 0.09. Compared to the 3D Dense U-Net, CDS-CNN predictions demonstrated a 3.5% improvement in TC, and CDS-CNN's dose prediction yielded better outcomes than the 3D Dense U-Net across all evaluation criteria. The experimental results demonstrated that the proposed CDS-CNN model outperformed other models in predicting GKRS dose distributions, with predictions closely matching the TPS doses.

Charged Gold Nanoparticles for Target Identification-Alignment and Automatic Segmentation of CT Image-Guided Adaptive Radiotherapy in Small Hepatocellular Carcinoma.

Because of the challenges posed by anatomical uncertainties and the low resolution of plain computed tomography (CT) scans, implementing adaptive radiotherapy (ART) for small hepatocellular carcinoma (sHCC) using artificial intelligence (AI) faces obstacles in tumor identification-alignment and automatic segmentation. The current study aims to improve sHCC imaging for ART using a gold nanoparticle (Au NP)-based CT contrast agent to enhance AI-driven automated image processing. The synthesized charged Au NPs demonstrated notable in vitro aggregation, low cytotoxicity, and minimal organ toxicity. Over time, an in situ sHCC mouse model was established for in vivo CT imaging at multiple time points. The enhanced CT images processed using 3D U-Net and 3D Trans U-Net AI models demonstrated high geometric and dosimetric accuracy. Therefore, charged Au NPs enable accurate and automatic sHCC segmentation in CT images using classical AI models, potentially addressing the technical challenges related to tumor identification, alignment, and automatic segmentation in CT-guided online ART.

Improving lung nodule segmentation in thoracic CT scans through the ensemble of 3D U-Net models.

The current study explores the application of 3D U-Net architectures combined with Inception and ResNet modules for precise lung nodule detection through deep learning-based segmentation technique. This investigation is motivated by the objective of developing a Computer-Aided Diagnosis (CAD) system for effective diagnosis and prognostication of lung nodules in clinical settings. The proposed method trained four different 3D U-Net models on the retrospective dataset obtained from AIIMS Delhi. To augment the training dataset, affine transformations and intensity transforms were utilized. Preprocessing steps included CT scan voxel resampling, intensity normalization, and lung parenchyma segmentation. Model optimization utilized a hybrid loss function that combined Dice Loss and Focal Loss. The model performance of all four 3D U-Nets was evaluated patient-wise using dice coefficient and Jaccard coefficient, then averaged to obtain the average volumetric dice coefficient (DSCavg) and average Jaccard coefficient (IoUavg) on a test dataset comprising 53 CT scans. Additionally, an ensemble approach (Model-V) was utilized featuring 3D U-Net (Model-I), ResNet (Model-II), and Inception (Model-III) 3D U-Net architectures, combined with two distinct patch sizes for further investigation. The ensemble of models obtained the highest DSCavg of 0.84 ± 0.05 and IoUavg of 0.74 ± 0.06 on the test dataset, compared against individual models. It mitigated false positives, overestimations, and underestimations observed in individual U-Net models. Moreover, the ensemble of models reduced average false positives per scan in the test dataset (1.57 nodules/scan) compared to individual models (2.69-3.39 nodules/scan). The suggested ensemble approach presents a strong and effective strategy for automatically detecting and delineating lung nodules, potentially aiding CAD systems in clinical settings. This approach could assist radiologists in laborious and meticulous lung nodule detection tasks in CT scans, improving lung cancer diagnosis and treatment planning.

EfficientQ: An efficient and accurate post-training neural network quantization method for medical image segmentation

Model quantization is a promising technique that can simultaneously compress and accelerate a deep neural network by limiting its computation bit-width, which plays a crucial role in the fast-growing AI industry. Despite model quantization’s success in producing well-performing low-bit models, the quantization process itself can still be expensive, which may involve a long fine-tuning stage on a large, well-annotated training set. To make the quantization process more efficient in terms of both time and data requirements, this paper proposes a fast and accurate post-training quantization method, namely EfficientQ. We develop this new method with a layer-wise optimization strategy and leverage the powerful alternating direction method of multipliers (ADMM) algorithm to ensure fast convergence. Furthermore, a weight regularization scheme is incorporated to provide more guidance for the optimization of the discrete weights, and a self-adaptive attention mechanism is proposed to combat the class imbalance problem. Extensive comparison and ablation experiments are conducted on two publicly available medical image segmentation datasets, i.e., LiTS and BraTS2020, and the results demonstrate the superiority of the proposed method over various existing post-training quantization methods in terms of both accuracy and optimization speed. Remarkably, with EfficientQ, the quantization of a practical 3D UNet only requires less than 5 min on a single GPU and one data sample. The source code is available at https://github.com/rongzhao-zhang/EfficientQ.

Dual Attention-Based 3D U-Net Liver Segmentation Algorithm on CT Images.

The liver is a vital organ in the human body, and CT images can intuitively display its morphology. Physicians rely on liver CT images to observe its anatomical structure and areas of pathology, providing evidence for clinical diagnosis and treatment planning. To assist physicians in making accurate judgments, artificial intelligence techniques are adopted. Addressing the limitations of existing methods in liver CT image segmentation, such as weak contextual analysis and semantic information loss, we propose a novel Dual Attention-Based 3D U-Net liver segmentation algorithm on CT images. The innovations of our approach are summarized as follows: (1) We improve the 3D U-Net network by introducing residual connections to better capture multi-scale information and alleviate semantic information loss. (2) We propose the DA-Block encoder structure to enhance feature extraction capability. (3) We introduce the CBAM module into skip connections to optimize feature transmission in the encoder, reducing semantic gaps and achieving accurate liver segmentation. To validate the effectiveness of the algorithm, experiments were conducted on the LiTS dataset. The results showed that the Dice coefficient and HD95 index for liver images were 92.56% and 28.09 mm, respectively, representing an improvement of 0.84% and a reduction of 2.45 mm compared to 3D Res-UNet.

3D U-Net Neural Network Architecture-Assisted LDCT to Acquire Vertebral Morphology Parameters: A Vertebral Morphology Comprehensive Analysis in a Chinese Population.

To evaluate the feasibility of acquiring vertebral height from chest low-dose computed tomography (LDCT) images using an artificial intelligence (AI) system based on 3D U-Net vertebral segmentation technology and the correlation and features of vertebral morphology with sex and age of the Chinese population. Patients who underwent chest LDCT between September 2020 and April 2023 were enrolled. The Altman and Pearson's correlation analyses were used to compare the correlation and consistency between the AI software and manual measurement of vertebral height. The anterior height (Ha), middle height (Hm), posterior height (Hp), and vertebral height ratios (VHRs) (Ha/Hp and Hm/Hp) were measured from T1 to L2 using an AI system. The VHR is the ratio of Ha to Hp or the ratio of Hm to Hp of the vertebrae, which can reflect the shape of the anterior wedge and biconcave vertebrae. Changes in these parameters, particularly the VHR, were analysed at different vertebral levels in different age and sex groups. The results of the AI methods were highly consistent and correlated with manual measurements. The Pearson's correlation coefficients were 0.855, 0.919, and 0.846, respectively. The trend of VHRs showed troughs at T7 and T11 and a peak at T9; however, Hm/Hp showed slight fluctuations. Regarding the VHR, significant sex differences were found at L1 and L2 in all age bands. This innovative study focuses on vertebral morphology for opportunistic analysis in the mainland Chinese population and the distribution tendency of vertebral morphology with ageing using a chest LDCT aided by an AI system based on 3D U-Net vertebral segmentation technology. The AI system demonstrates the potential to automatically perform opportunistic vertebral morphology analyses using LDCT scans obtained during lung cancer screening. We advocate the use of age-, sex-, and vertebral level-specific criteria for the morphometric evaluation of vertebral osteoporotic fractures for a more accurate diagnosis of vertebral fractures and spinal pathologies.

Automated segmentation of MRI white matter hyperintensities in 8,421 patients with acute ischemic stroke.

To date, only a few small studies have attempted deep learning-based automatic segmentation of white matter hyperintensity (WMH) lesions in patients with cerebral infarction, which is complicated because stroke-related lesions can obscure WMH borders. We developed and validated deep learning algorithms to segment WMH lesions accurately in patients with cerebral infarction, using multisite datasets involving 8,421 patients with acute ischemic stroke. We included 8,421 stroke patients from 9 centers in Korea. 2D UNet and SE-Unet models were trained using 2,408 FLAIR MRI from 3 hospitals and validated using 6,013 FLAIR MRIs from 6 hospitals. WMH segmentation performance was assessed by calculating DSC, correlation coefficient, and concordance correlation coefficient compared to a human-segmented gold standard. In addition, we obtained an uncertainty index that represents overall ambiguity in the voxel classification for WMH segmentation in each patient based on the Kullback-Leibler divergence. In the training dataset, the mean age was 67.4±13.0 years and 60.4% were men. The mean (95% CI) DSCs for Unet in internal testing and external validation were respectively 0.659 (0.649-0.669) and 0.710 (0.707-0.714), which were slightly lower than the reliability between humans (DSC=0.744; 95% CI=0.738-0.751; P=.031). Compared with the Unet, the SE-Unet demonstrated better performance, achieving a mean DSC of 0.675 (0.666-0.685; P<.001) in the internal testing and 0.722 (0.719-0.726; P<.001) in the external validation; moreover, it achieved high DSC values (ranging from 0.672 to 0.744) across multiple validation datasets. We observed a significant correlation between WMH volumes that were segmented automatically and manually for the Unet (r=0.917, P<.0001) and even stronger for the SE-Unet (r=0.933, P<.0001). The SE-Unet also attained a high concordance correlation coefficient (ranging from 0.841 to 0.956) in external test datasets. In addition, the uncertainty indices in the majority of patients (86%) in the external datasets were below 0.35, with an average DSC of 0.744 in these patients. We developed and validated deep learning algorithms to segment WMH in patients with acute cerebral infarction using the largest-ever MRI datasets. In addition, we showed that the uncertainty index can be used to identify cases where automatic WMH segmentation is less accurate and requires human review. WMH = white matter hyperintensity; CNN = convolutional neural networks; SE = squeeze-and-excitation; KL = Kullback-Leibler; ReLU = rectified linear unit; LKW = last known well; mRS = modified Rankin Scale; NIHSS = National Institute of Health Stroke Scale; LAA = large artery atherosclerosis; SVO = small vessel occlusion; CE = cardioembolism.

Convolutional neural network for automated tooth segmentation on intraoral scans

BackgroundTooth segmentation on intraoral scanned (IOS) data is a prerequisite for clinical applications in digital workflows. Current state-of-the-art methods lack the robustness to handle variability in dental conditions. This study aims to propose and evaluate the performance of a convolutional neural network (CNN) model for automatic tooth segmentation on IOS images.MethodsA dataset of 761 IOS images (380 upper jaws, 381 lower jaws) was acquired using an intraoral scanner. The inclusion criteria included a full set of permanent teeth, teeth with orthodontic brackets, and partially edentulous dentition. A multi-step 3D U-Net pipeline was designed for automated tooth segmentation on IOS images. The model’s performance was assessed in terms of time and accuracy. Additionally, the model was deployed on an online cloud-based platform, where a separate subsample of 18 IOS images was used to test the clinical applicability of the model by comparing three modes of segmentation: automated artificial intelligence-driven (A-AI), refined (R-AI), and semi-automatic (SA) segmentation.ResultsThe average time for automated segmentation was 31.7 ± 8.1 s per jaw. The CNN model achieved an Intersection over Union (IoU) score of 91%, with the full set of teeth achieving the highest performance and the partially edentulous group scoring the lowest. In terms of clinical applicability, SA took an average of 860.4 s per case, whereas R-AI showed a 2.6-fold decrease in time (328.5 s). Furthermore, R-AI offered higher performance and reliability compared to SA, regardless of the dentition group.ConclusionsThe 3D U-Net pipeline was accurate, efficient, and consistent for automatic tooth segmentation on IOS images. The online cloud-based platform could serve as a viable alternative for IOS segmentation.

Head and neck tumor segmentation from [18F]F-FDG PET/CT images based on 3D diffusion model

Objective. Head and neck (H&N) cancers are among the most prevalent types of cancer worldwide, and [18F]F-FDG PET/CT is widely used for H&N cancer management. Recently, the diffusion model has demonstrated remarkable performance in various image-generation tasks. In this work, we proposed a 3D diffusion model to accurately perform H&N tumor segmentation from 3D PET and CT volumes. Approach. The 3D diffusion model was developed considering the 3D nature of PET and CT images acquired. During the reverse process, the model utilized a 3D UNet structure and took the concatenation of 3D PET, CT, and Gaussian noise volumes as the network input to generate the tumor mask. Experiments based on the HECKTOR challenge dataset were conducted to evaluate the effectiveness of the proposed diffusion model. Several state-of-the-art techniques based on U-Net and Transformer structures were adopted as the reference methods. Benefits of employing both PET and CT as the network input, as well as further extending the diffusion model from 2D to 3D, were investigated based on various quantitative metrics and qualitative results. Main results. Results showed that the proposed 3D diffusion model could generate more accurate segmentation results compared with other methods (mean Dice of 0.739 compared to less than 0.726 for other methods). Compared to the diffusion model in 2D form, the proposed 3D model yielded superior results (mean Dice of 0.739 compared to 0.669). Our experiments also highlighted the advantage of utilizing dual-modality PET and CT data over only single-modality data for H&N tumor segmentation (with mean Dice less than 0.570). Significance. This work demonstrated the effectiveness of the proposed 3D diffusion model in generating more accurate H&N tumor segmentation masks compared to the other reference methods.