Hausdorff Distance Research Articles

Spatial-aware contrastive learning for cross-domain medical image registration.

With the rapid advancement of medical imaging technologies, precise image analysis and diagnosis play a crucial role in enhancing treatment outcomes and patient care. Computed tomography (CT) and magnetic resonance imaging (MRI), as pivotal technologies in medical imaging, exhibit unique advantages in bone imaging and soft tissue contrast, respectively. However, cross-domain medical image registration confronts significant challenges due to the substantial differences in contrast, texture, and noise levels between different imaging modalities. The purpose of this study is to address the major challenges encountered in the field of cross-domain medical image registration by proposing a spatial-aware contrastive learning approach that effectively integrates shared information from CT and MRI images. Our objective is to optimize the feature space representation by employing advanced reconstruction and contrastive loss functions, overcoming the limitations of traditional registration methods when dealing with different imaging modalities. Through this approach, we aim to enhance the model's ability to learn structural similarities across domain images, improve registration accuracy, and provide more precise imaging analysis tools for clinical diagnosis and treatmentplanning. With prior knowledge that different domains of images (CT and MRI) share same content-style information, we extract equivalent feature spaces from both images, enabling accurate cross-domain point matching. We employ a structure resembling that of an autoencoder, augmented with designed reconstruction and contrastive losses to fulfill our objectives. We also propose region mask to solve the conflict between spatial correlation and distinctiveness, to obtain a better representation space. Our research results demonstrate the significant superiority of the proposed spatial-aware contrastive learning approach in the domain of cross-domain medical image registration. Quantitatively, our method achieved an average Dice similarity coefficient (DSC) of 85.68%, target registration error (TRE) of 1.92 mm, and mean Hausdorff distance (MHD) of 1.26 mm, surpassing current state-of-the-art methods. Additionally, the registration processing time was significantly reduced to 2.67 s on a GPU, highlighting the efficiency of our approach. The experimental outcomes not only validate the effectiveness of our method in improving the accuracy of cross-domain image registration but also prove its adaptability across different medical image analysis scenarios, offering robust support for enhancing diagnostic precision and patient treatmentoutcomes. The spatial-aware contrastive learning approach proposed in this paper introduces a new perspective and solution to the domain of cross-domain medical image registration. By effectively optimizing the feature space representation through carefully designed reconstruction and contrastive loss functions, our method significantly improves the accuracy and stability of registration between CT and MRI images. The experimental results demonstrate the clear advantages of our approach in enhancing the accuracy of cross-domain image registration, offering significant application value in promoting precise diagnosis and personalized treatment planning. In the future, we look forward to further exploring the application of this method in a broader range of medical imaging datasets and its potential integration with other advanced technologies, contributing more innovations to the field of medical image analysis andprocessing.

A modified dynamic relaxation form-finding method for symmetrical tensegrity structures with group theory and fuzzy clustering

For tensegrity structures, there are many groups of cables and bars with symmetrical structures or similar internal forces. In this paper, an application of a fuzzy clustering algorithm in the context of form-finding processes for symmetric tensegrity structures is proposed. This algorithm aims to automate grouping, optimize form-finding strategies, and expedite convergence. Point sets are generated through the segmentation of structural components, and Hausdorff distance is used to extract spatial features. Following this, fuzzy clustering automatically groups components with geometric symmetry. The resultant clustering matrix facilitates the refinement of form-finding processes, thus reducing the computational load associated with solving the equilibrium matrix for internal forces within tensegrity structures. By clustering components with analogous internal forces, computational efficiency is enhanced. Additionally, this methodology refines numerical form-finding outcomes based on symmetrical attributes, improving form-finding precision.

Enhanced multistage deep learning for diagnosing anterior disc displacement in the temporomandibular joint using MRI.

This study aimed to propose a new method for the automatic diagnosis of anterior disc displacement of the temporomandibular joint (TMJ) using MRI and deep learning. By using a multistage approach, the factors affecting the final result can be easily identified and improved. This study introduces a multistage automatic diagnostic technique using deep learning. This process involves segmenting the target from MR images, extracting distance parameters, and classifying the diagnosis into 3 classes. MRI exams of 368 TMJs from 204 patients were evaluated for anterior disc displacement. In the first stage, 5 algorithms were used for the semantic segmentation of the disc and condyle. In the second stage, 54 distance parameters were extracted from the segments. In the third stage, a rule-based decision model was developed to link the parameters with the expert diagnosis results. In the first stage, DeepLabV3+ showed the best result (95% Hausdorff distance, Dice coefficient, and sensitivity of 6.47 ± 7.22, 0.84 ± 0.07, and 0.84 ± 0.09, respectively). This study used the original MRI exams as input without preprocessing and showed high segmentation performance compared with that of previous studies. In the third stage, the combination of SegNet and a random forest model yielded an accuracy of 0.89 ± 0.06. An algorithm was developed to automatically diagnose TMJ-anterior disc displacement using MRI. Through a multistage approach, this algorithm facilitated the improvement of results and demonstrated high accuracy from more complex inputs. Furthermore, existing radiological knowledge was applied and validated.

Multi-modal fusion and feature enhancement U-Net coupling with stem cell niches proximity estimation for voxel-wise GBM recurrence prediction.

Objective.We aim to develop a Multi-modal Fusion and Feature Enhancement U-Net (MFFE U-Net) coupling with stem cell niche proximity estimation to improve voxel-wise Glioblastoma (GBM) recurrence prediction.Approach.57 patients with pre- and post-surgery magnetic resonance (MR) scans were retrospectively solicited from 4 databases. Post-surgery MR scans included two months before the clinical diagnosis of recurrence and the day of the radiologicaly confirmed recurrence. The recurrences were manually annotated on the T1ce. The high-risk recurrence region was first determined. Then, a sparse multi-modal feature fusion U-Net was developed. The 50 patients from 3 databases were divided into 70% training, 10% validation, and 20% testing. 7 patients from the 4th institution were used as external testing with transfer learning. Model performance was evaluated by recall, precision, F1-score, and Hausdorff Distance at the 95% percentile (HD95). The proposed MFFE U-Net was compared to the support vector machine (SVM) model and two state-of-the-art neural networks. An ablation study was performed.Main results.The MFFE U-Net achieved a precision of 0.79 ± 0.08, a recall of 0.85 ± 0.11, and an F1-score of 0.82 ± 0.09. Statistically significant improvement was observed when comparing MFFE U-Net with proximity estimation couple SVM (SVMPE), mU-Net, and Deeplabv3. The HD95 was 2.75 ± 0.44 mm and 3.91 ± 0.83 mm for the 10 patients used in the model construction and 7 patients used for external testing, respectively. The ablation test showed that all five MR sequences contributed to the performance of the final model, with T1ce contributing the most. Convergence analysis, time efficiency analysis, and visualization of the intermediate results further discovered the characteristics of the proposed method.Significance. We present an advanced MFFE learning framework, MFFE U-Net, for effective voxel-wise GBM recurrence prediction. MFFE U-Net performs significantly better than the state-of-the-art networks and can potentially guide early RT intervention of the disease recurrence.

Domain adaptation using AdaBN and AdaIN for high-resolution IVD mesh reconstruction from clinical MRI.

Deep learning has firmly established its dominance in medical imaging applications. However, careful consideration must be exercised when transitioning a trained source model to adapt to an entirely distinct environment that deviates significantly from the training set. The majority of the efforts to mitigate this issue have predominantly focused on classification and segmentation tasks. In this work, we perform a domain adaptation of a trained source model to reconstruct high-resolution intervertebral disc meshes from low-resolution MRI. To address the outlined challenges, we use MRI2Mesh as the shape reconstruction network. It incorporates three major modules: image encoder, mesh deformation, and cross-level feature fusion. This feature fusion module is used to encapsulate local and global disc features. We evaluate two major domain adaptation techniques: adaptive batch normalization (AdaBN) and adaptive instance normalization (AdaIN) for the task of shape reconstruction. Experiments conducted on distinct datasets, including data from different populations, machines, and test sites demonstrate the effectiveness of MRI2Mesh for domain adaptation. MRI2Mesh achieved up to a 14% decrease in Hausdorff distance (HD) and a 19% decrease in the point-to-surface (P2S) metric for both AdaBN and AdaIN experiments, indicating improved performance. MRI2Mesh has demonstrated consistent superiority to the state-of-the-art Voxel2Mesh network across a diverse range of datasets, populations, and scanning protocols, highlighting its versatility. Additionally, AdaBN has emerged as a robust method compared to the AdaIN technique. Further experiments show that MRI2Mesh, when combined with AdaBN, holds immense promise for enhancing the precision of anatomical shape reconstruction in domain adaptation.

Development and clinical validation of a deep learning-based knee CT image segmentation method for robotic-assisted total knee arthroplasty.

This study aimed to develop a novel deep convolutional neural network called Dual-path Double Attention Transformer (DDA-Transformer) designed to achieve precise and fast knee joint CT image segmentation and to validate it in robotic-assisted total knee arthroplasty (TKA). The femoral, tibial, patellar, and fibular segmentation performance and speed were evaluated and the accuracy of component sizing, bone resection and alignment of the robotic-assisted TKA system constructed using this deep learning network was clinically validated. Overall, DDA-Transformer outperformed six other networks in terms of the Dice coefficient, intersection over union, average surface distance, and Hausdorff distance. DDA-Transformer exhibited significantly faster segmentation speeds than nnUnet, TransUnet and 3D-Unet (p<0.01). Furthermore, the robotic-assisted TKA system outperforms the manual group in surgical accuracy. DDA-Transformer exhibited significantly improved accuracy and robustness in knee joint segmentation, and this convenient and stable knee joint CT image segmentation network significantly improved the accuracy of the TKA procedure.

FreqSNet: a multiaxial integration of frequency and spatial domains for medical image segmentation

Objective. In recent years, convolutional neural networks, which typically focus on extracting spatial domain features, have shown limitations in learning global contextual information. However, frequency domain can offer a global perspective that spatial domain methods often struggle to capture. To address this limitation, we propose FreqSNet, which leverages both frequency and spatial features for medical image segmentation. Approach. To begin, we propose a frequency-space representation aggregation block (FSRAB) to replace conventional convolutions. FSRAB contains three frequency domain branches to capture global frequency information along different axial combinations, while a convolutional branch is designed to interact information across channels in local spatial features. Secondly, the multiplex expansion attention block extracts long-range dependency information using dilated convolutional blocks, while suppressing irrelevant information via attention mechanisms. Finally, the introduced Feature Integration Block enhances feature representation by integrating semantic features that fuse spatial and channel positional information. Main results. We validated our method on 5 public datasets, including BUSI, CVC-ClinicDB, CVC-ColonDB, ISIC-2018, and Luna16. On these datasets, our method achieved Intersection over Union (IoU) scores of 75.46%, 87.81%, 79.08%, 84.04%, and 96.99%, and Hausdorff distance values of 22.22 mm, 13.20 mm, 13.08 mm, 13.51 mm, and 5.22 mm, respectively. Compared to other state-of-the-art methods, our FreqSNet achieves better segmentation results. Significance. Our method can effectively combine frequency domain information with spatial domain features, enhancing the segmentation performance and generalization capability in medical image segmentation tasks.

Random walks in the high-dimensional limit II: The crinkled subordinator

A crinkled subordinator is an ℓ2-valued random process which can be thought of as a version of the usual one-dimensional subordinator with each out of countably many jumps being in a direction orthogonal to the directions of all other jumps. We show that the path of a d-dimensional random walk with n independent identically distributed steps with heavy-tailed distribution of the radial components and asymptotically orthogonal angular components converges in distribution in the Hausdorff distance up to isometry and also in the Gromov–Hausdorff sense, if viewed as a random metric space, to the closed range of a crinkled subordinator, as d,n→∞.

Geometric validation of a pediatric upper airways model made using a mainstream desktop 3D printer

BackgroundThree-dimensional (3D) printing has become increasingly affordable. Several research projects used 3D printing to create in vitro upper airways model. However, studies using a mainstream desktop 3D printer never performed geometric validation of their model. The aim of this study was to perform geometric validation of a pediatric upper airways model printed with a mainstream desktop 3D printer. MethodsHead computerized tomography (CT) scan of a 10-month-old female underwent segmentation between airways and surrounding anatomical structures. Airways segmentation allowed their measurement for further comparison with printed model. Head segmentation enabled the creation of a 3D printable volume file. To proceed to the geometric validation of the head model, the latter underwent a CT scan. Similar segmentation work was performed on the printed model for comparison. Overlap proportion between the original infant volume and the printed model as well as an average Hausdorff distance were calculated after manual alignment between the original and printed model. ResultsVolumes were 12.31 cm3 and 12.32 cm3 for the pediatric and the printed model upper airways, respectively (0.08% difference). Dice coefficient of original and printed model was 0.92. The average Hausdorff distance was 0.21 mm. ConclusionDesktop mainstream 3D printers can generate pediatric upper airway model with a high dimensional accuracy, as evidenced by our comprehensive geometrical validation.

Sequences of operator algebras converging to odd spheres in the quantum Gromov–Hausdorff distance

Sequences of operator algebras converging to odd spheres in the quantum Gromov–Hausdorff distance

An improved 3D-UNet-based brain hippocampus segmentation model based on MR images

ObjectiveAccurate delineation of the hippocampal region via magnetic resonance imaging (MRI) is crucial for the prevention and early diagnosis of neurosystemic diseases. Determining how to accurately and quickly delineate the hippocampus from MRI results has become a serious issue. In this study, a pixel-level semantic segmentation method using 3D-UNet is proposed to realize the automatic segmentation of the brain hippocampus from MRI results. Methods: Two hundred three-dimensional T1-weighted (3D-T1) nongadolinium contrast-enhanced magnetic resonance (MR) images were acquired at Hangzhou Cancer Hospital from June 2020 to December 2022. These samples were divided into two groups, containing 175 and 25 samples. In the first group, 145 cases were used to train the hippocampus segmentation model, and the remaining 30 cases were used to fine-tune the hyperparameters of the model. Images for twenty-five patients in the second group were used as the test set to evaluate the performance of the model. The training set of images was processed via rotation, scaling, grey value augmentation and transformation with a smooth dense deformation field for both image data and ground truth labels. A filling technique was introduced into the segmentation network to establish the hippocampus segmentation model. In addition, the performance of models established with the original network, such as VNet, SegResNet, UNetR and 3D-UNet, was compared with that of models constructed by combining the filling technique with the original segmentation network. Results: The results showed that the performance of the segmentation model improved after the filling technique was introduced. Specifically, when the filling technique was introduced into VNet, SegResNet, 3D-UNet and UNetR, the segmentation performance of the models trained with an input image size of 48 × 48 × 48 improved. Among them, the 3D-UNet-based model with the filling technique achieved the best performance, with a Dice score (Dice score) of 0.7989 ± 0.0398 and a mean intersection over union (mIoU) of 0.6669 ± 0.0540, which were greater than those of the original 3D-UNet-based model. In addition, the oversegmentation ratio (OSR), average surface distance (ASD) and Hausdorff distance (HD) were 0.0666 ± 0.0351, 0.5733 ± 0.1018 and 5.1235 ± 1.4397, respectively, which were better than those of the other models. In addition, when the size of the input image was set to 48 × 48 × 48, 64 × 64 × 64 and 96 × 96 × 96, the model performance gradually improved, and the Dice scores of the proposed model reached 0.7989 ± 0.0398, 0.8371 ± 0.0254 and 0.8674 ± 0.0257, respectively. In addition, the mIoUs reached 0.6669 ± 0.0540, 0.7207 ± 0.0370 and 0.7668 ± 0.0392, respectively. Conclusion: The proposed hippocampus segmentation model constructed by introducing the filling technique into a segmentation network performed better than models built solely on the original network and can improve the efficiency of diagnostic analysis.

Localized fine-tuning and clinical evaluation of deep-learning based auto-segmentation (DLAS) model for clinical target volume (CTV) and organs-at-risk (OAR) in rectal cancer radiotherapy

Background and purposeVarious deep learning auto-segmentation (DLAS) models have been proposed, some of which have been commercialized. However, the issue of performance degradation is notable when pretrained models are deployed in the clinic. This study aims to enhance precision of a popular commercial DLAS product in rectal cancer radiotherapy by localized fine-tuning, addressing challenges in practicality and generalizability in real-world clinical settings.Materials and methodsA total of 120 Stage II/III mid-low rectal cancer patients were retrospectively enrolled and divided into three datasets: training (n = 60), external validation (ExVal, n = 30), and generalizability evaluation (GenEva, n = 30) datasets respectively. The patients in the training and ExVal dataset were acquired on the same CT simulator, while those in GenEva were on a different CT simulator. The commercial DLAS software was first localized fine-tuned (LFT) for clinical target volume (CTV) and organs-at-risk (OAR) using the training data, and then validated on ExVal and GenEva respectively. Performance evaluation involved comparing the LFT model with the vendor-provided pretrained model (VPM) against ground truth contours, using metrics like Dice similarity coefficient (DSC), 95th Hausdorff distance (95HD), sensitivity and specificity.ResultsLFT significantly improved CTV delineation accuracy (p < 0.05) with LFT outperforming VPM in target volume, DSC, 95HD and specificity. Both models exhibited adequate accuracy for bladder and femoral heads, and LFT demonstrated significant enhancement in segmenting the more complex small intestine. We did not identify performance degradation when LFT and VPM models were applied in the GenEva dataset.ConclusionsThe necessity and potential benefits of LFT DLAS towards institution-specific model adaption is underscored. The commercial DLAS software exhibits superior accuracy once localized fine-tuned, and is highly robust to imaging equipment changes.

Quantitative susceptibility mapping based basal ganglia segmentation via AGSeg: leveraging active gradient guiding mechanism in deep learning.

With better visual contrast and the ability for magnetic susceptibility quantification analysis, quantitative susceptibility mapping (QSM) has emerged as an important magnetic resonance imaging (MRI) method for basal ganglia studies. Precise segmentation of basal ganglia is a prerequisite for quantification analysis of tissue magnetic susceptibility, which is crucial for subsequent disease diagnosis and surgical planning. The conventional method of localizing and segmenting basal ganglia heavily relies on layer-by-layer manual annotation by experts, resulting in a tedious amount of workload. Although several morphology registration and deep learning based methods have been developed to automate segmentation, the voxels around the nuclei boundary remain a challenge to distinguish due to insufficient tissue contrast. This paper proposes AGSeg, an active gradient guidance-based susceptibility and magnitude information complete (MIC) network for real-time and accurate basal ganglia segmentation. Various datasets, including clinical scans and data from healthy volunteers, were collected across multiple centers with different magnetic field strengths (3T/5T/7T), with a total of 210 three-dimensional (3D) susceptibility measurements. Manual segmentations following fixed rules for anatomical borders annotated by experts were used as ground truth labels. The proposed network took QSM maps and Magnitude images as two individual inputs, of which the features are selectively enhanced in the proposed magnitude information complete (MIC) module. AGSeg utilized a dual-branch architecture, with Seg-branch aiming to generate a proper segmentation map and Grad-branch to reconstruct the gradient map of regions of interest (ROIs). With the support of the newly designed active gradient module (AGM) and gradient guiding module (GGM), the Grad-branch provided attention guidance for the Seg-branch, facilitating it to focus on the boundary of target nuclei. Ablation studies were conducted to assess the functionality of the proposed modules. Significant performance decrement was observed after ablating relative modules. AGSeg was evaluated against several existing methods on both healthy and clinical data, achieving an average Dice similarity coefficient (DSC) =0.874 and average 95% Hausdorff distance (HD95) =2.009. Comparison experiments indicated that our model had superior performance on basal ganglia segmentation and better generalization ability over existing methods. The AGSeg outperformed all implemented comparison deep learning algorithms with average DSC enhancement ranging from 0.036 to 0.074. The current work integrates a deep learning-based method into automated basal ganglia segmentation. The high processing speed and segmentation robustness of AGSeg contribute to the feasibility of future surgery planning and intraoperative navigation. Experiments show that leveraging active gradient guidance mechanisms and magnitude information completion can facilitate the segmentation process. Moreover, this approach also offers a portable solution for other multi-modality medical image segmentation tasks.

Retrospective Comparison of Geometrical Accuracy among Atlas-based Auto-segmentation, Deep Learning Auto-segmentation, and Deformable Image Registration in the Treatment Replanning for Adaptive Radiotherapy of Head-and-Neck Cancer

Aims: This study aimed to evaluate the geometrical accuracy of atlas-based auto-segmentation (ABAS), deformable image registration (DIR), and deep learning auto-segmentation (DLAS) in adaptive radiotherapy (ART) for head-and-neck cancer (HNC). Subjects and Methods: Seventeen patients who underwent replanning for ART were retrospectively studied, and delineated contours on their replanning computed tomography (CT2) images were delineated. For DIR, the planning CT image (CT1) of the evaluated patients was utilized. In contrast, ABAS was performed using an atlas dataset comprising 30 patients who were not part of the evaluated group. DLAS was trained with 143 patients from different patients from the evaluated patients. The ABAS model was improved, and a modified ABAS (mABAS) was created by adding the evaluated patients’ own CT1 to the atlas datasets of ABAS (number of patients of the atlas dataset, 31). The geometrical accuracy of DIR, DLAS, ABAS, and mABAS was evaluated. Results: The Dice similarity coefficient in DIR was the highest, at &gt;0.8 at all organs at risk. The mABAS was delineated slightly more accurately than the standard ABAS. There was no significant difference between ABAS and DLAS in delineation accuracy. DIR had the lowest Hausdorff distance (HD) value (within 10 mm). The HD values in ABAS, mABAS, and DLAS were within 16 mm. Conclusions: DIR delineation is the most geometrically accurate ART for HNC.

Left Atrial Appendage Analysis from Echocardiographic Images: Relevance in Left Atrial Appendage Occlusion

Abstract Atrial fibrillation, a prevalent cardiac arrhythmia, disrupts the normal path of electrical signals within the heart. To address this issue, the left atrial appendage closure is often considered. This involves using a closure instrument named the Watchman to obstruct the left atrial appendage (LAA) ostium. The wide variety of LAA shapes, sizes, and forms in individuals makes selecting the correct Watchman size crucial. This study first isolates the LAA from echocardiographic images and then recommends the appropriate Watchman size. The 3-D echocardiographic images used in this study are from 32 male and female patients who underwent successful LAA closure at Kings College Hospital London over two years. For each patient, there are 208 cardiac echo slices in the imaging mode. This study presents an image processing-based model to separate LAA regions and extract relevant features. These features are then inputted into a classification platform in which a neural network determines the suitable Watchman size. The average and mode Hausdorff distances, calculated between the proposed method and the traditional manual calculation for the 501 analyzed LAA images, are approximately 0.2467 and 0.0587, respectively. Because of limited data, a 10-fold cross-validation was used to assess the classifier, yielding an accuracy of 74.07 %. The proposed model effectively isolated the LAA from the corresponding slices of the 3-D echocardiographic images. However, the classifier’s accuracy is not ideal because of insufficient data, which could be improved by expanding the database. This research’s outcomes could aid physicians in selecting the proper Watchman size.

Dose robustness of deep learning models for anatomic segmentation of computed tomography images.

The trend towards lower radiation doses and advances in computed tomography (CT) reconstruction may impair the operation of pretrained segmentation models, giving rise to the problem of estimating the dose robustness of existing segmentation models. Previous studies addressing the issue suffer either from a lack of registered low- and full-dose CT images or from simplified simulations. We employed raw data from full-dose acquisitions to simulate low-dose CT scans, avoiding the need to rescan a patient. The accuracy of the simulation is validated using a real CT scan of a phantom. We consider down to 20% reduction of radiation dose, for which we measure deviations of several pretrained segmentation models from the full-dose prediction. In addition, compatibility with existing denoising methods is considered. The results reveal the surprising robustness of the TotalSegmentator approach, showing minimal differences at the pixel level even without denoising. Less robust models show good compatibility with the denoising methods, which help to improve robustness in almost all cases. With denoising based on a convolutional neural network (CNN), the median Dice between low- and full-dose data does not fall below 0.9 (12 for the Hausdorff distance) for all but one model. We observe volatile results for labels with effective radii less than 19mm and improved results for contrasted CT acquisitions. The proposed approach facilitates clinically relevant analysis of dose robustness for human organ segmentation models. The results outline the robustness properties of a diverse set of models. Further studies are needed to identify the robustness of approaches for lesion segmentation and to rank the factors contributing to dose robustness.

Extended nnU-Net for Brain Metastasis Detection and Segmentation in Contrast-Enhanced Magnetic Resonance Imaging With a Large Multi-Institutional Data Set

PurposeTo investigate an extended self-adapting nnU-Net framework for detecting and segmenting brain metastases (BM) on MRI. ApproachSix different nnU-Net systems with adaptive data sampling, adaptive Dice loss (ADL) or different patch/batch sizes were trained and tested for detecting and segmenting intraparenchymal BM with a size ≥ 2 mm on 3D post-Gd T1-weighted MRI volumes using 2092 patients from seven institutions (1712, 195 and 185 patients for training, validation and testing, respectively). Gross tumor volumes (GTVs) of BM delineated by physicians for stereotactic radiosurgery (SRS) were collected retrospectively and curated at each institute. Additional centralized data curation was carried out to create GTVs of uncontoured BM by two radiologists to improve accuracy of ground truth. The training dataset was augmented with synthetic BMs of 1025 MRI volumes using a 3D generative pipeline. BM detection was evaluated by lesion-level sensitivity and false-positive (FP) rate. BM segmentation was assessed by lesion-level Dice similarity coefficient (DSC), 95-percentile Hausdorff distance (HD95) and average HD. The performances were assessed across different BM sizes. Additional testing was performed using a second dataset of 206 patients. ResultsOf the six nnU-Net systems, the nnU-Net with ADL achieved the best detection and segmentation performance on the first testing dataset. At an FP rate of 0.65±1.17, overall sensitivity was 0.904 for all sizes of BM, 0.966 for BM ≥ 0.1 cm3 and 0.824 for BM < 0.1 cm3. Mean values of DSC, HD95 and average HD of all detected BMs were 0.758, 1.45 mm and 0.23 mm, respectively. Performances on the second testing dataset achieved sensitivity of 0.907 at an FP rate of 0.57±0.85 for all BM sizes, and average HD of 0.33 mm for all detected BM. ConclusionsOur proposed extension of the self-configuring nnU-Net framework substantially improved small BM detection sensitivity while maintaining a controlled FP rate. Clinical utility of the extended nnU-Net model for assisting early BM detection and SRS planning will be investigated.

Estimating Surgical Urethral Length on Intraoperative Robot-Assisted Prostatectomy Images Using Artificial Intelligence Anatomy Recognition.

Objective: To construct a convolutional neural network (CNN) model that can recognize and delineate anatomic structures on intraoperative video frames of robot-assisted radical prostatectomy (RARP) and to use these annotations to predict the surgical urethral length (SUL). Background: Urethral dissection during RARP impacts patient urinary incontinence (UI) outcomes, and requires extensive training. Large differences exist between incontinence outcomes of different urologists and hospitals. Also, surgeon experience and education are critical toward optimal outcomes. Therefore, new approaches are warranted. SUL is associated with UI. Artificial intelligence (AI) surgical image segmentation using a CNN could automate SUL estimation and contribute toward future AI-assisted RARP and surgeon guidance. Methods: Eighty-eight intraoperative RARP videos between June 2009 and September 2014 were collected from a single center. Two hundred sixty-four frames were annotated according to prostate, urethra, ligated plexus, and catheter. Thirty annotated images from different RARP videos were used as a test data set. The dice (similarity) coefficient (DSC) and 95th percentile Hausdorff distance (Hd95) were used to determine model performance. SUL was calculated using the catheter as a reference. Results: The DSC of the best performing model were 0.735 and 0.755 for the catheter and urethra classes, respectively, with a Hd95 of 29.27 and 72.62, respectively. The model performed moderately on the ligated plexus and prostate. The predicted SUL showed a mean difference of 0.64 to 1.86 mm difference vs human annotators, but with significant deviation (standard deviation = 3.28-3.56). Conclusion: This study shows that an AI image segmentation model can predict vital structures during RARP urethral dissection with moderate to fair accuracy. SUL estimation derived from it showed large deviations and outliers when compared with human annotators, but with a small mean difference (<2 mm). This is a promising development for further research on AI-assisted RARP.

CMAF-Net: a cross-modal attention fusion-based deep neural network for incomplete multi-modal brain tumor segmentation.

The information between multimodal magnetic resonance imaging (MRI) is complementary. Combining multiple modalities for brain tumor image segmentation can improve segmentation accuracy, which has great significance for disease diagnosis and treatment. However, different degrees of missing modality data often occur in clinical practice, which may lead to serious performance degradation or even failure of brain tumor segmentation methods relying on full-modality sequences to complete the segmentation task. To solve the above problems, this study aimed to design a new deep learning network for incomplete multimodal brain tumor segmentation. We propose a novel cross-modal attention fusion-based deep neural network (CMAF-Net) for incomplete multimodal brain tumor segmentation, which is based on a three-dimensional (3D) U-Net architecture with encoding and decoding structure, a 3D Swin block, and a cross-modal attention fusion (CMAF) block. A convolutional encoder is initially used to extract the specific features from different modalities, and an effective 3D Swin block is constructed to model the long-range dependencies to obtain richer information for brain tumor segmentation. Then, a cross-attention based CMAF module is proposed that can deal with different missing modality situations by fusing features between different modalities to learn the shared representations of the tumor regions. Finally, the fused latent representation is decoded to obtain the final segmentation result. Additionally, channel attention module (CAM) and spatial attention module (SAM) are incorporated into the network to further improve the robustness of the model; the CAM to help focus on important feature channels, and the SAM to learn the importance of different spatial regions. Evaluation experiments on the widely-used BraTS 2018 and BraTS 2020 datasets demonstrated the effectiveness of the proposed CMAF-Net which achieved average Dice scores of 87.9%, 81.8%, and 64.3%, as well as Hausdorff distances of 4.21, 5.35, and 4.02 for whole tumor, tumor core, and enhancing tumor on the BraTS 2020 dataset, respectively, outperforming several state-of-the-art segmentation methods in missing modalities situations. The experimental results show that the proposed CMAF-Net can achieve accurate brain tumor segmentation in the case of missing modalities with promising application potential.

SymTC: A symbiotic Transformer-CNN net for instance segmentation of lumbar spine MRI

Intervertebral disc disease, a prevalent ailment, frequently leads to intermittent or persistent low back pain, and diagnosing and assessing of this disease rely on accurate measurement of vertebral bone and intervertebral disc geometries from lumbar MR images. Deep neural network (DNN) models may assist clinicians with more efficient image segmentation of individual instances (discs and vertebrae) of the lumbar spine in an automated way, which is termed as instance image segmentation. In this work, we proposed SymTC, an innovative lumbar spine MR image segmentation model that combines the strengths of Transformer and Convolutional Neural Network (CNN). Specifically, we designed a parallel dual-path architecture to merge CNN layers and Transformer layers, and we integrated a novel position embedding into the self-attention module of Transformer, enhancing the utilization of positional information for more accurate segmentation. To further improve model performance, we introduced a new data synthesis technique to create synthetic yet realistic MR image dataset, named SSMSpine, which is made publicly available. We evaluated our SymTC and the other 16 representative image segmentation models on our private in-house dataset and public SSMSpine dataset, using two metrics, Dice Similarity Coefficient and the 95th percentile Hausdorff Distance. The results indicate that SymTC surpasses the other 16 methods, achieving the highest dice score of 96.169 % for segmenting vertebral bones and intervertebral discs on the SSMSpine dataset. The SymTC code and SSMSpine dataset are publicly available at https://github.com/jiasongchen/SymTC.