U-Net Model Research Articles

Integrating Deep Learning-Based Dose Distribution Prediction with Bayesian Networks for Decision Support in Radiotherapy for Upper Gastrointestinal cancer.

Selecting the better techniques to harbor optimal motion management, either a stereotactic linear accelerator delivery using TrueBeam (TBX) or Magnetic Resonance (MR)-guided gated delivery using MRIdian (MRG), is time-consuming and costly. To address this challenge, we aimed to develop a decision-supporting algorithm based on a combination of deep learning-generated dose distributions and clinical data. We retrospectively analyzed 65 patients with liver or pancreatic cancer who underwent both TBX and MRG simulations and planning process. We trained three-dimensional U-Net deep learning models to predict dose distributions and generated dose volume histograms (DVHs) for each system. We integrated predicted DVH metrics into a Bayesian network (BN) model incorporating clinical data. The MRG prediction model outperformed the TBX model, demonstrating statistically significant superiorities in predicting normalized dose to the PTV and liver. We developed a final BN prediction model integrating the predictive DVH metrics with patient factors like age, PTV size, and tumor location. This BN model an area under the receiver operating characteristic curve index of 83.56%. The decision tree derived from the BN model showed that the tumor location (abutting vs. apart of PTV to hollow viscus organs) was the most important factor to determine TBX or MRG. We demonstrated a decision-supporting algorithm for selecting optimal RT plans in upper gastrointestinal cancers, incorporating both deep learning-based dose prediction and BN-based treatment selection. This approach might streamline the decision-making process, saving resources and improving treatment outcomes for patients undergoing RT.

Leveraging Pretrained Transformers for Efficient Segmentation and Lesion Detection in Cone-Beam Computed Tomography Scans

IntroductionCone-beam computed tomography (CBCT) is widely used to detect jaw lesions, although CBCT interpretation is time-consuming and challenging. Artificial intelligence for CBCT segmentation may improve lesion detection accuracy. However, consistent automated lesion detection remains difficult, especially with limited training data. This study aimed to assess the applicability of pretrained transformer-based architectures for semantic segmentation of CBCT volumes when applied to periapical lesion detection. MethodsCBCT volumes (n = 138) were collected and annotated by expert clinicians using 5 labels – "lesion," "restorative material," "bone," "tooth structure," and "background." U-Net (convolutional neural network-based) and Swin-UNETR (transformer-based) models, pretrained (Swin-UNETR-PRETRAIN), and from scratch (Swin-UNETR-SCRATCH), were trained with subsets of the annotated CBCTs. These models were then evaluated for semantic segmentation performance using the Sørensen–Dice coefficient (DICE), lesion detection performance using sensitivity and specificity, and training sample size requirements by comparing models trained with 20, 40, 60, or 103 samples. ResultsTrained with 103 samples, Swin-UNETR-PRETRAIN achieved a DICE of 0.8512 for "lesion," 0.8282 for "restorative materials," 0.9178 for "bone," 0.9029 for "tooth structure," and 0.9901 for "background." “Lesion” DICE was statistically similar between Swin-UNETR-PRETRAIN trained with 103 and 60 images (P > .05), with the latter achieving 1.00 sensitivity and 0.94 specificity in lesion detection. With small training sets, Swin-UNETR-PRETRAIN outperformed Swin-UNETR-SCRATCH in DICE over all labels (P < .001 [n = 20], P < .001 [n = 40]), and U-Net in lesion detection specificity (P = .006 [n = 20], P = .031 [n = 40]). ConclusionsTransformer-based Swin-UNETR architectures allowed for excellent semantic segmentation and periapical lesion detection. Pretrained, it may provide an alternative with smaller training datasets compared to classic U-Net architectures.

Abstract We016: Generative Artificial Intelligence for hPSC-derived Cardiac Organoid Florescence Generation

Background: Human pluripotent stem cell (hPSC)-derived cardiac organoids (COs) are extensively applied in modeling heart development and disease. COs are daily monitored by bright-field microscopic imaging for tracking organoid differentiation, morphology, and size, however, it does not provide important information on cell distribution and network in COs. Then, fluorescence microscopic imaging is required with extra steps of experiments when multiple hPSC lines are required. Recently, generative artificial intelligence (AI) has been increasingly applied for image colorization, which extends the possibility of colorizing COs with fluorescence from phase/contrast images. While the phase/contrast image is routinely applied and conveniently used in every biomedical lab daily, is it possible to generate accurate and useful fluorescence information or image colorization from the phase/contrast images of cardiac organoids under the same differentiation protocol is crucial for a broader characterization and analysis of hPSC-derived cardiac organoids and further applications? Hypothesis: We hypothesize that generative AI will generate accurate fluorescence images from phase/contrast images of COs. Approach: 1300 images from COs (differentiated from one reporter hPSC line) were used as the training dataset for the Pix2Pix Conditional Generative Adversarial Networks (GANs) model ( Fig.1a ) with the Convolution Block Attention Module (CBAM) in the U-Net generator ( Fig.1b ). Three models were designed: U-Net (Model 1), U-Net generator with CBAM (Model 2), and U-Net with CBAM and Generator Iteration (Model 3). Three different metrics were included: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Weighted Patch Histogram to evaluate the prediction outcomes. Results: The predicted fluorescence images from phase/contrast images of different COs are very similar to the ground truths ( Fig.1c ). The similarities evaluated by all three metrics ( Fig.1d ) have the scores of PSNR over 30, SSIM over 0.92, and a Weighted Patch Histogram score over 0.75, which are the most accurate and similar to the ground truth. Conclusions: A generative AI model was established to colorize phase images of COs using cGANs and CBAM, which capture intricate fluorescence details to support the biomedical research of COs with enhanced interpretability and analysis of cardiac organoid morphology and structure in biomedical research and applications.

Improving diffuse optical tomography imaging quality using APU-Net: an attention-based physical U-Net model.

Traditional diffuse optical tomography (DOT) reconstructions are hampered by image artifacts arising from factors such as DOT sources being closer to shallow lesions, poor optode-tissue coupling, tissue heterogeneity, and large high-contrast lesions lacking information in deeper regions (known as shadowing effect). Addressing these challenges is crucial for improving the quality of DOT images and obtaining robust lesion diagnosis. We address the limitations of current DOT imaging reconstruction by introducing an attention-based U-Net (APU-Net) model to enhance the image quality of DOT reconstruction, ultimately improving lesion diagnostic accuracy. We designed an APU-Net model incorporating a contextual transformer attention module to enhance DOT reconstruction. The model was trained on simulation and phantom data, focusing on challenges such as artifact-induced distortions and lesion-shadowing effects. The model was then evaluated by the clinical data. Transitioning from simulation and phantom data to clinical patients' data, our APU-Net model effectively reduced artifacts with an average artifact contrast decrease of 26.83% and improved image quality. In addition, statistical analyses revealed significant contrast improvements in depth profile with an average contrast increase of 20.28% and 45.31% for the second and third target layers, respectively. These results highlighted the efficacy of our approach in breast cancer diagnosis. The APU-Net model improves the image quality of DOT reconstruction by reducing DOT image artifacts and improving the target depth profile.

Automatic segmentation of the maxillary sinus on cone beam computed tomographic images with U-Net deep learning model

BackgroundMedical imaging segmentation is the use of image processing techniques to expand specific structures or areas in medical images. This technique is used to separate and display different textures or shapes in an image. The aim of this study is to develop a deep learning-based method to perform maxillary sinus segmentation using cone beam computed tomography (CBCT) images. The proposed segmentation method aims to provide better image guidance to surgeons and specialists by determining the boundaries of the maxillary sinus cavities. In this way, more accurate diagnoses can be made and surgical interventions can be performed more successfully.MethodsIn the study, axial CBCT images of 100 patients (200 maxillary sinuses) were used. These images were marked to identify the maxillary sinus walls. The marked regions are masked for use in the maxillary sinus segmentation model. U-Net, one of the deep learning methods, was used for segmentation. The training process was carried out for 10 epochs and 100 iterations per epoch. The epoch and iteration numbers in which the model showed maximum success were determined using the early stopping method.ResultsAfter the segmentation operations performed with the U-Net model trained using CBCT images, both visual and numerical results were obtained. In order to measure the performance of the U-Net model, IoU (Intersection over Union) and F1 Score metrics were used. As a result of the tests of the model, the IoU value was found to be 0.9275 and the F1 Score value was 0.9784.ConclusionThe U-Net model has shown high success in maxillary sinus segmentation. In this way, fast and highly accurate evaluations are possible, saving time by reducing the workload of clinicians and eliminating subjective errors.

Deep Learning Based Automatic Left Ventricle Segmentation from the Transgastric Short-Axis View on Transesophageal Echocardiography: A Feasibility Study.

Segmenting the left ventricle from the transgastric short-axis views (TSVs) on transesophageal echocardiography (TEE) is the cornerstone for cardiovascular assessment during perioperative management. Even for seasoned professionals, the procedure remains time-consuming and experience-dependent. The current study aims to evaluate the feasibility of deep learning for automatic segmentation by assessing the validity of different U-Net algorithms. A large dataset containing 1388 TSV acquisitions was retrospectively collected from 451 patients (32% women, average age 53.42 years) who underwent perioperative TEE between July 2015 and October 2023. With image preprocessing and data augmentation, 3336 images were included in the training set, 138 images in the validation set, and 138 images in the test set. Four deep neural networks (U-Net, Attention U-Net, UNet++, and UNeXt) were employed for left ventricle segmentation and compared in terms of the Jaccard similarity coefficient (JSC) and Dice similarity coefficient (DSC) on the test set, as well as the number of network parameters, training time, and inference time. The Attention U-Net and U-Net++ models performed better in terms of JSC (the highest average JSC: 86.02%) and DSC (the highest average DSC: 92.00%), the UNeXt model had the smallest network parameters (1.47 million), and the U-Net model had the least training time (6428.65 s) and inference time for a single image (101.75 ms). The Attention U-Net model outperformed the other three models in challenging cases, including the impaired boundary of left ventricle and the artifact of the papillary muscle. This pioneering exploration demonstrated the feasibility of deep learning for the segmentation of the left ventricle from TSV on TEE, which will facilitate an accelerated and objective alternative of cardiovascular assessment for perioperative management.

Enhanced 3D dose prediction for hypofractionated SRS (gamma knife radiosurgery) in brain tumor using cascaded-deep-supervised convolutional neural network.

Gamma Knife radiosurgery (GKRS) is a well-established technique in radiation therapy (RT) for treating small-size brain tumors. It administers highly concentrated doses during each treatment fraction, with even minor dose errors posing a significant risk of causing severe damage to healthy tissues. It underscores the critical need for precise and meticulous precision in GKRS. However, the planning process for GKRS is complex and time-consuming, heavily reliant on the expertise of medical physicists. Incorporating deep learning approaches for GKRS dose prediction can reduce this dependency, improve planning efficiency and homogeneity, streamline clinical workflows, and reduce patient lagging times. Despite this, precise Gamma Knife plan dose distribution prediction using existing models remains a significant challenge. The complexity stems from the intricate nature of dose distributions, subtle contrasts in CT scans, and the interdependence of dosimetric metrics. To overcome these challenges, we have developed a "Cascaded-Deep-Supervised" Convolutional Neural Network (CDS-CNN) that employs a hybrid-weighted optimization scheme. Our innovative method incorporates multi-level deep supervision and a strategic sequential multi-network training approach. It enables the extraction of intra-slice and inter-slice features, leading to more realistic dose predictions with additional contextual information. CDS-CNN was trained and evaluated using data from 105 brain cancer patients who underwent GKRS treatment, with 85 cases used for training and 20 for testing. Quantitative assessments and statistical analyses demonstrated high consistency between the predicted dose distributions and the reference doses from the treatment planning system (TPS). The 3D overall gamma passing rates (GPRs) reached 97.15% ± 1.36% (3mm/3%, 10% threshold), surpassing the previous best performance by 2.53% using the 3D Dense U-Net model. When evaluated against more stringent criteria (2mm/3%, 10% threshold, and 1mm/3%, 10% threshold), the overall GPRs still achieved 96.53% ± 1.08% and 95.03% ± 1.18%. Furthermore, the average target coverage (TC) was 98.33% ± 1.16%, dose selectivity (DS) was 0.57 ± 0.10, gradient index (GI) was 2.69 ± 0.30, and homogeneity index (HI) was 1.79 ± 0.09. Compared to the 3D Dense U-Net, CDS-CNN predictions demonstrated a 3.5% improvement in TC, and CDS-CNN's dose prediction yielded better outcomes than the 3D Dense U-Net across all evaluation criteria. The experimental results demonstrated that the proposed CDS-CNN model outperformed other models in predicting GKRS dose distributions, with predictions closely matching the TPS doses.

Real-time detection and geometric analysis algorithm for concrete cracks based on the improved U-net model

Real-time detection and geometric analysis algorithm for concrete cracks based on the improved U-net model

Simultaneous soot parameters fields predictions accuracy improvements in laminar sooting flames from soot radiation measurements — a multi-task learning approach

Machine learning assisted optical diagnostics could reduce conventional extensive experimental data collecting and post-processing costs and time in academic and industrial combustion measurements. In this paper, a novel Trident-Net (T-Net) architecture is designed and assisted for retrieval of high-fidelity soot temperature, volume fraction (SVF), and diameter fields simultaneously from soot radiation measurements in laminar sooting flames. Uniquely, the T-net is subtly fabricated in one branch of the encoder and three branches of decoders, which enables three adjustable cost functions and the corresponding decoder branch for soot three respective parameter fields. Contrasted with previous Back-propagation (BP) and U-net models, the T-Net is more efficient and achieves a higher entire score in terms of individual decoder manipulation. In addition, owing to the generalization performance improvement of multi-task learning, the T-Net model demonstrates decent prediction performance under few-shot learning. Thus, this proposed T-Net model showcases the great opportunities for real-time, in situ, monitoring of practical combustion pollutant emission, by embedded in a response strategy of the detection devices.

A Method for Extracting Joints on Mountain Tunnel Faces Based on Mask R-CNN Image Segmentation Algorithm

The accurate distribution of joints on the tunnel face is crucial for assessing the stability and safety of surrounding rock during tunnel construction. This paper introduces the Mask R-CNN image segmentation algorithm, a state-of-the-art deep learning model, to achieve efficient and accurate identification and extraction of joints on tunnel face images. First, digital images of tunnel faces were captured and stitched, resulting in 286 complete images suitable for analysis. Then, the joints on the tunnel face were extracted using traditional image processing algorithms, the commonly used U-net image segmentation model, and the Mask R-CNN image segmentation model introduced in this paper to address the lack of recognition accuracy. Finally, the extraction results obtained by the three methods were compared. The comparison results show that the joint extraction method based on the Mask R-CNN image segmentation deep learning model introduced in this paper achieved the best joint extraction effect with a Dice similarity coefficient of 87.48%, outperforming traditional methods and the U-net model, which scored 60.59% and 75.36%, respectively, realizing accurate and efficient acquisition of tunnel face rock joints. These findings suggest that the Mask R-CNN model can be effectively implemented in real-time monitoring systems for tunnel construction projects.

Accelerating photoacoustic microscopy by reconstructing undersampled images using diffusion models

Photoacoustic Microscopy (PAM) integrates optical and acoustic imaging, offering enhanced penetration depth for detecting optical-absorbing components in tissues. Nonetheless, challenges arise in scanning large areas with high spatial resolution. With speed limitations imposed by laser pulse repetition rates, the potential role of computational methods is highlighted in accelerating PAM imaging. We propose a novel and highly adaptable algorithm named DiffPam that utilizes diffusion models to speed up the photoacoustic imaging process. We leveraged a diffusion model trained exclusively on natural images, comparing its performance with an in-domain trained U-Net model using a dataset focused on PAM images of mice brain microvasculature. Our findings indicate that DiffPam performs similarly to a dedicated U-Net model without needing a large dataset. We demonstrate that scanning can be accelerated fivefold with limited information loss. We achieved a 24.70%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$24.70\\%$$\\end{document} increase in peak signal-to-noise ratio and a 27.54%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$27.54\\%$$\\end{document} increase in structural similarity index compared to the baseline bilinear interpolation method. The study also introduces the efficacy of shortened diffusion processes for reducing computing time without compromising accuracy. DiffPam stands out from existing methods as it does not require supervised training or detailed parameter optimization typically needed for other unsupervised methods. This study underscores the significance of DiffPam as a practical algorithm for reconstructing undersampled PAM images, particularly for researchers with limited artificial intelligence expertise and computational resources.

Improving lung nodule segmentation in thoracic CT scans through the ensemble of 3D U-Net models.

The current study explores the application of 3D U-Net architectures combined with Inception and ResNet modules for precise lung nodule detection through deep learning-based segmentation technique. This investigation is motivated by the objective of developing a Computer-Aided Diagnosis (CAD) system for effective diagnosis and prognostication of lung nodules in clinical settings. The proposed method trained four different 3D U-Net models on the retrospective dataset obtained from AIIMS Delhi. To augment the training dataset, affine transformations and intensity transforms were utilized. Preprocessing steps included CT scan voxel resampling, intensity normalization, and lung parenchyma segmentation. Model optimization utilized a hybrid loss function that combined Dice Loss and Focal Loss. The model performance of all four 3D U-Nets was evaluated patient-wise using dice coefficient and Jaccard coefficient, then averaged to obtain the average volumetric dice coefficient (DSCavg) and average Jaccard coefficient (IoUavg) on a test dataset comprising 53 CT scans. Additionally, an ensemble approach (Model-V) was utilized featuring 3D U-Net (Model-I), ResNet (Model-II), and Inception (Model-III) 3D U-Net architectures, combined with two distinct patch sizes for further investigation. The ensemble of models obtained the highest DSCavg of 0.84 ± 0.05 and IoUavg of 0.74 ± 0.06 on the test dataset, compared against individual models. It mitigated false positives, overestimations, and underestimations observed in individual U-Net models. Moreover, the ensemble of models reduced average false positives per scan in the test dataset (1.57 nodules/scan) compared to individual models (2.69-3.39 nodules/scan). The suggested ensemble approach presents a strong and effective strategy for automatically detecting and delineating lung nodules, potentially aiding CAD systems in clinical settings. This approach could assist radiologists in laborious and meticulous lung nodule detection tasks in CT scans, improving lung cancer diagnosis and treatment planning.

A new deep learning-based approach for concrete crack identification and damage assessment

Concrete building structures are prone to cracking as they are subjected to environmental temperatures, freeze-thaw cycles, and other operational environmental factors. Failure to detect cracks in the key building structure at the early stage can result in serious accidents and associated economic losses. A new method using the SE-U-Net model based on a conditional generative adversarial network (CGAN) has been developed to identify small cracks in concrete structures in this paper. This proposed method was a pixel-level U-Net model based on a generative network, that was integrated the original convolutional layer with an attention mechanism, and an SE module in the jump connection section was added to improve the identifiability of the model. The discriminative network compared the generated images with real images using the PatchGAN model. Through the adversarial training of generator and discriminator, the performance of generator in crack image segmentation task is improved, and the trained generation network is used to segment cracks. In damage assessments, the crack skeleton was represented by the individual pixel width and recognized using the binary morphological crack skeleton method, in which the final length, area, and average width of the crack could be determined through the geometric correction index. The results showed that compared with other methods, the proposed method could better identify subtle pixel-level cracks, and the identification accuracy is 98.48%. These methods are of great significance for the identification of cracks and the damage assessment of concrete structures in practice.

FringeNet: A cyclic U-Net model with continuity imposed hybrid cyclic loss for demodulation of isochromatics in digital photoelasticity

This study introduces FringeNet, an innovative deep learning-based cyclic model to enhance the fringe order demodulation from single isochromatic images. A Continuity-Imposed Hybrid Cyclic Loss (CHCL) function, which combines Mean Squared Error (MSE), Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and a continuity loss is proposed for optimizing multiple objectives, while enforcing the fringe order continuity in the isochromatic data. The proposed FringeNet is trained using available open-source ischromatic image dataset and the effectiveness of the model is validated using realistic experimental isochromatics. The FringeNet demonstrates a 92.4% improvement in MSE over existing methods, indicating substantial gains in predictive accuracy and model robustness. Additionally, a 16.19% enhancement in PSNR is observed, highlighting superior fidelity compared to existing approaches. The cyclic model employed in FringeNet represents a significant advancement in enhancing the deep learning-based predictive modeling used in the field of digital photoelasticity.

TTFDNet: Precise Depth Estimation from Single-Frame Fringe Patterns.

This work presents TTFDNet, a transformer-based and transfer learning network for end-to-end depth estimation from single-frame fringe patterns in fringe projection profilometry. TTFDNet features a precise contour and coarse depth (PCCD) pre-processor, a global multi-dimensional fusion (GMDF) module and a progressive depth extractor (PDE). It utilizes transfer learning through fringe structure consistency evaluation (FSCE) to leverage the transformer's benefits even on a small dataset. Tested on 208 scenes, the model achieved a mean absolute error (MAE) of 0.00372 mm, outperforming Unet (0.03458 mm) models, PDE (0.01063 mm) and PCTNet (0.00518 mm). It demonstrated precise measurement capabilities with deviations of ~90 μm for a 25.4 mm radius ball and ~6 μm for a 20 mm thick metal part. Additionally, TTFDNet showed excellent generalization and robustness in dynamic reconstruction and varied imaging conditions, making it appropriate for practical applications in manufacturing, automation and computer vision.

Semantic segmentation method for micro-cracks in silicon nitride ceramic bearing balls based on coupling of edge channel enhancement and weighted gated attention mechanism in EMU-Net+

In addressing the issues of blurred microcrack edge gradients and mutual interference among various types of microcracks in the semantic segmentation of microcracks in silicon nitride bearing balls. Based on coupling edge channel enhancement and a weighted gated attention mechanism within the EMU-Net + framework method is proposed. The Unet + function matrix is constructed with long-short connection structures, incorporating feature fusion across different layers to obtain a larger receptive field. The edge channel enhancement and weighted gated attention mechanism are established, feeding back features segmented at different scales to the higher level, enhancing the contrast between features and the background, achieving finer segmentation of microcrack mask images. Compared to the baseline Unet model, segmentation metrics including Miou, Mpa, Precision, and Recall on the microcrack dataset are improved by 6.73 %, 3.1 %, 5 %, and 3.41 %, respectively. The average intersection and merger ratios of three different microcrack types all exceed 80 %.

Open Access Data and Deep Learning for Cardiac Device Identification on Standard DICOM and Smartphone-based Chest Radiographs.

Purpose To develop and evaluate a publicly available deep learning model for segmenting and classifying cardiac implantable electronic devices (CIEDs) on Digital Imaging and Communications in Medicine (DICOM) and smartphone-based chest radiographs. Materials and Methods This institutional review board-approved retrospective study included patients with implantable pacemakers, cardioverter defibrillators, cardiac resynchronization therapy devices, and cardiac monitors who underwent chest radiography between January 2012 and January 2022. A U-Net model with a ResNet-50 backbone was created to classify CIEDs on DICOM and smartphone images. Using 2321 chest radiographs in 897 patients (median age, 76 years [range, 18-96 years]; 625 male, 272 female), CIEDs were categorized into four manufacturers, 27 models, and one "other" category. Five smartphones were used to acquire 11 072 images. Performance was reported using the Dice coefficient on the validation set for segmentation or balanced accuracy on the test set for manufacturer and model classification, respectively. Results The segmentation tool achieved a mean Dice coefficient of 0.936 (IQR: 0.890-0.958). The model had an accuracy of 94.36% (95% CI: 90.93%, 96.84%; 251 of 266) for CIED manufacturer classification and 84.21% (95% CI: 79.31%, 88.30%; 224 of 266) for CIED model classification. Conclusion The proposed deep learning model, trained on both traditional DICOM and smartphone images, showed high accuracy for segmentation and classification of CIEDs on chest radiographs. Keywords: Conventional Radiography, Segmentation Supplemental material is available for this article. © RSNA, 2024 See also the commentary by Júdice de Mattos Farina and Celi in this issue.

Thoracic computed tomography (CT) image-based identification and severity classification of COVID-19 cases using vision transformer (ViT)

In this research, we developed a two-stage deep learning (DL) model using Vision Transformer (ViT) to detect COVID-19 and assess its severity from thoracic CT images. In the first stage, we utilized a pre-trained ViT model (ViT_B/32) and a custom CNN model to classify CT images as COVID-19 or non-COVID-19. The ViT model achieved superior performance with a fivefold cross-validated accuracy of 99.7%, compared to the custom CNN’s 98%. In the second stage, we employed a ViT-based U-Net model (Vision Transformer for Biomedical Image Segmentation, VITBIS) to segment lung and infection regions in COVID-19 positive CT images, determining the infection severity. This model uses transformers with attention mechanisms in both the encoder and decoder. The lung segmentation network achieved an Intersection Over Union (IOU) of 95.8% and a sensitivity of 99.67%, while the lesion segmentation network attained an IOU of 94% and a sensitivity of 98.3%.

Automated segmentation of MRI white matter hyperintensities in 8,421 patients with acute ischemic stroke.

To date, only a few small studies have attempted deep learning-based automatic segmentation of white matter hyperintensity (WMH) lesions in patients with cerebral infarction, which is complicated because stroke-related lesions can obscure WMH borders. We developed and validated deep learning algorithms to segment WMH lesions accurately in patients with cerebral infarction, using multisite datasets involving 8,421 patients with acute ischemic stroke. We included 8,421 stroke patients from 9 centers in Korea. 2D UNet and SE-Unet models were trained using 2,408 FLAIR MRI from 3 hospitals and validated using 6,013 FLAIR MRIs from 6 hospitals. WMH segmentation performance was assessed by calculating DSC, correlation coefficient, and concordance correlation coefficient compared to a human-segmented gold standard. In addition, we obtained an uncertainty index that represents overall ambiguity in the voxel classification for WMH segmentation in each patient based on the Kullback-Leibler divergence. In the training dataset, the mean age was 67.4±13.0 years and 60.4% were men. The mean (95% CI) DSCs for Unet in internal testing and external validation were respectively 0.659 (0.649-0.669) and 0.710 (0.707-0.714), which were slightly lower than the reliability between humans (DSC=0.744; 95% CI=0.738-0.751; P=.031). Compared with the Unet, the SE-Unet demonstrated better performance, achieving a mean DSC of 0.675 (0.666-0.685; P<.001) in the internal testing and 0.722 (0.719-0.726; P<.001) in the external validation; moreover, it achieved high DSC values (ranging from 0.672 to 0.744) across multiple validation datasets. We observed a significant correlation between WMH volumes that were segmented automatically and manually for the Unet (r=0.917, P<.0001) and even stronger for the SE-Unet (r=0.933, P<.0001). The SE-Unet also attained a high concordance correlation coefficient (ranging from 0.841 to 0.956) in external test datasets. In addition, the uncertainty indices in the majority of patients (86%) in the external datasets were below 0.35, with an average DSC of 0.744 in these patients. We developed and validated deep learning algorithms to segment WMH in patients with acute cerebral infarction using the largest-ever MRI datasets. In addition, we showed that the uncertainty index can be used to identify cases where automatic WMH segmentation is less accurate and requires human review. WMH = white matter hyperintensity; CNN = convolutional neural networks; SE = squeeze-and-excitation; KL = Kullback-Leibler; ReLU = rectified linear unit; LKW = last known well; mRS = modified Rankin Scale; NIHSS = National Institute of Health Stroke Scale; LAA = large artery atherosclerosis; SVO = small vessel occlusion; CE = cardioembolism.

Unet retinal vessel image segmentation algorithm based on coordinate attention mechanism and Min-ASPP module optimisation

In this paper, the Unet image segmentation algorithm is optimised and improved by introducing the coordinate attention mechanism and integrating the Min-ASPP module, and compared with the traditional Unet model, targeting the segmentation of retinal blood vessel images, which is a key problem in the field of medical image processing. The experimental results show that the optimised Unet algorithm combining the coordinate attention mechanism and Min-ASPP module has a smoother performance in the loss curve, faster convergence speed, and lower final converged loss value, which has an obvious advantage compared with the traditional Unet algorithm. Meanwhile, the optimised Unet algorithm shows higher accuracy and stability in the prediction results, providing ophthalmologists with more reliable information about the retinal vascular structure. Therefore, the research results of this paper show that the Unet image segmentation algorithm optimised by combining the coordinate attention mechanism and Min-ASPP module has better application prospects and effects in the retinal blood vessel image segmentation task, which provides a useful exploration and practice for improving the level of medical image processing technology.