Brain Tumor Segmentation Challenge Research Articles

Training and Comparison of nnU-Net and DeepMedic Methods for Autosegmentation of Pediatric Brain Tumors.

Tumor segmentation is essential in surgical and treatment planning and response assessment and monitoring in pediatric brain tumors, the leading cause of cancer-related death among children. However, manual segmentation is time-consuming and has high interoperator variability, underscoring the need for more efficient methods. After training, we compared 2 deep-learning-based 3D segmentation models, DeepMedic and nnU-Net, with pediatric-specific multi-institutional brain tumor data based on multiparametric MR images. Multiparametric preoperative MR imaging scans of 339 pediatric patients (n = 293 internal and n = 46 external cohorts) with a variety of tumor subtypes were preprocessed and manually segmented into 4 tumor subregions, ie, enhancing tumor, nonenhancing tumor, cystic components, and peritumoral edema. After training, performances of the 2 models on internal and external test sets were evaluated with reference to ground truth manual segmentations. Additionally, concordance was assessed by comparing the volume of the subregions as a percentage of the whole tumor between model predictions and ground truth segmentations using the Pearson or Spearman correlation coefficients and the Bland-Altman method. The mean Dice score for nnU-Net internal test set was 0.9 (SD, 0.07) (median, 0.94) for whole tumor; 0.77 (SD, 0.29) for enhancing tumor; 0.66 (SD, 0.32) for nonenhancing tumor; 0.71 (SD, 0.33) for cystic components, and 0.71 (SD, 0.40) for peritumoral edema, respectively. For DeepMedic, the mean Dice scores were 0.82 (SD, 0.16) for whole tumor; 0.66 (SD, 0.32) for enhancing tumor; 0.48 (SD, 0.27) for nonenhancing tumor; 0.48 (SD, 0.36) for cystic components, and 0.19 (SD, 0.33) for peritumoral edema, respectively. Dice scores were significantly higher for nnU-Net (P ≤ .01). Correlation coefficients for tumor subregion percentage volumes were higher (0.98 versus 0.91 for enhancing tumor, 0.97 versus 0.75 for nonenhancing tumor, 0.98 versus 0.80 for cystic components, 0.95 versus 0.33 for peritumoral edema in the internal test set). Bland-Altman plots were better for nnU-Net compared with DeepMedic. External validation of the trained nnU-Net model on the multi-institutional Brain Tumor Segmentation Challenge in Pediatrics (BraTS-PEDs) 2023 data set revealed high generalization capability in the segmentation of whole tumor, tumor core (a combination of enhancing tumor, nonenhancing tumor, and cystic components), and enhancing tumor with mean Dice scores of 0.87 (SD, 0.13) (median, 0.91), 0.83 (SD, 0.18) (median, 0.89), and 0.48 (SD, 0.38) (median, 0.58), respectively. The pediatric-specific data-trained nnU-Net model is superior to DeepMedic for whole tumor and subregion segmentation of pediatric brain tumors.

End-to-End Multi-task Learning Architecture for Brain Tumor Analysis with Uncertainty Estimation in MRI Images.

Brain tumors are a threat to life for every other human being, be it adults or children. Gliomas are one of the deadliest brain tumors with an extremely difficult diagnosis. The reason is their complex and heterogenous structure which gives rise to subjective as well as objective errors. Their manual segmentation is a laborious task due to their complex structure and irregular appearance. To cater to all these issues, a lot of research has been done and is going on to develop AI-based solutions that can help doctors and radiologists in the effective diagnosis of gliomas with the least subjective and objective errors, but an end-to-end system is still missing. An all-in-one framework has been proposed in this research. The developed end-to-end multi-task learning (MTL) architecture with a feature attention module can classify, segment, and predict the overall survival of gliomas by leveraging task relationships between similar tasks. Uncertainty estimation has also been incorporated into the framework to enhance the confidence level of healthcare practitioners. Extensive experimentation was performed by using combinations of MRI sequences. Brain tumor segmentation (BraTS) challenge datasets of 2019 and 2020 were used for experimental purposes. Results of the best modelwith four sequences show 95.1% accuracy for classification, 86.3% dicescore for segmentation, and a mean absolute error (MAE) of 456.59 for survival predictionon the test data. It is evident from the results that deep learning-based MTL models have the potential to automate the whole brain tumor analysis process and give efficient results with least inference time without human intervention. Uncertainty quantification confirms the idea that more data can improve the generalization ability and in turn can produce more accurate results with less uncertainty. The proposed model has the potential to be utilized in a clinical setup for the initial screening of glioma patients.

[Fully Automatic Glioma Segmentation Algorithm of Magnetic Resonance Imaging Based on 3D-UNet With More Global Contextual Feature Extraction: An Improvement on Insufficient Extraction of Global Features].

The fully automatic segmentation of glioma and its subregions is fundamental for computer-aided clinical diagnosis of tumors. In the segmentation process of brain magnetic resonance imaging (MRI), convolutional neural networks with small convolutional kernels can only capture local features and are ineffective at integrating global features, which narrows the receptive field and leads to insufficient segmentation accuracy. This study aims to use dilated convolution to address the problem of inadequate global feature extraction in 3D-UNet. 1) Algorithm construction: A 3D-UNet model with three pathways for more global contextual feature extraction, or 3DGE-UNet, was proposed in the paper. By using publicly available datasets from the Brain Tumor Segmentation Challenge (BraTS) of 2019 (335 patient cases), a global contextual feature extraction (GE) module was designed. This module was integrated at the first, second, and third skip connections of the 3D UNet network. The module was utilized to fully extract global features at different scales from the images. The global features thus extracted were then overlaid with the upsampled feature maps to expand the model's receptive field and achieve deep fusion of features at different scales, thereby facilitating end-to-end automatic segmentation of brain tumors. 2) Algorithm validation: The image data were sourced from the BraTs 2019 dataset, which included the preoperative MRI images of 335 patients across four modalities (T1, T1ce, T2, and FLAIR) and a tumor image with annotations made by physicians. The dataset was divided into the training, the validation, and the testing sets at an 8∶1∶1 ratio. Physician-labelled tumor images were used as the gold standard. Then, the algorithm's segmentation performance on the whole tumor (WT), tumor core (TC), and enhancing tumor (ET) was evaluated in the test set using the Dice coefficient (for overall effectiveness evaluation), sensitivity (detection rate of lesion areas), and 95% Hausdorff distance (segmentation accuracy of tumor boundaries). The performance was tested using both the 3D-UNet model without the GE module and the 3DGE-UNet model with the GE module to internally validate the effectiveness of the GE module setup. Additionally, the performance indicators were evaluated using the 3DGE-UNet model, ResUNet, UNet++, nnUNet, and UNETR, and the convergence of these five algorithm models was compared to externally validate the effectiveness of the 3DGE-UNet model. 1) In internal validation, the enhanced 3DGE-UNet model achieved Dice mean values of 91.47%, 87.14%, and 83.35% for segmenting the WT, TC, and ET regions in the test set, respectively, producing the optimal values for comprehensive evaluation. These scores were superior to the corresponding scores of the traditional 3D-UNet model, which were 89.79%, 85.13%, and 80.90%, indicating a significant improvement in segmentation accuracy across all three regions (P<0.05). Compared with the 3D-UNet model, the 3DGE-UNet model demonstrated higher sensitivity for ET (86.46% vs. 80.77%) (P<0.05) , demonstrating better performance in the detection of all the lesion areas. When dealing with lesion areas, the 3DGE-UNet model tended to correctly identify and capture the positive areas in a more comprehensive way, thereby effectively reducing the likelihood of missed diagnoses. The 3DGE-UNet model also exhibited exceptional performance in segmenting the edges of WT, producing a mean 95% Hausdorff distance superior to that of the 3D-UNet model (8.17 mm vs. 13.61 mm, P<0.05). However, its performance for TC (8.73 mm vs. 7.47 mm) and ET (6.21 mm vs. 5.45 mm) was similar to that of the 3D-UNet model. 2) In the external validation, the other four algorithms outperformed the 3DGE-UNet model only in the mean Dice for TC (87.25%), the mean sensitivity for WT (94.59%), the mean sensitivity for TC (86.98%), and the mean 95% Hausdorff distance for ET (5.37 mm). Nonetheless, these differences were not statistically significant (P>0.05). The 3DGE-UNet model demonstrated rapid convergence during the training phase, outpacing the other external models. The 3DGE-UNet model can effectively extract and fuse feature information on different scales, improving the accuracy of brain tumor segmentation.

Synthesizing Contrast-Enhanced MR Images from Noncontrast MR Images Using Deep Learning.

Recent developments in deep learning methods offer a potential solution to the need for alternative imaging methods due to concerns about the toxicity of gadolinium-based contrast agents. The purpose of the study was to synthesize virtual gadolinium contrast-enhanced T1-weighted MR images from noncontrast multiparametric MR images in patients with primary brain tumors by using deep learning. We trained and validated a deep learning network by using MR images from 335 subjects in the Brain Tumor Segmentation Challenge 2019 training data set. A held out set of 125 subjects from the Brain Tumor Segmentation Challenge 2019 validation data set was used to test the generalization of the model. A residual inception DenseNet network, called T1c-ET, was developed and trained to simultaneously synthesize virtual contrast-enhanced T1-weighted (vT1c) images and segment the enhancing portions of the tumor. Three expert neuroradiologists independently scored the synthesized vT1c images by using a 3-point Likert scale, evaluating image quality and contrast enhancement against ground truth T1c images (1 = poor, 2 = good, 3 = excellent). The synthesized vT1c images achieved structural similarity index, peak signal-to-noise ratio, and normalized mean square error scores of 0.91, 64.35, and 0.03, respectively. There was moderate interobserver agreement between the 3 raters, regarding the algorithm's performance in predicting contrast enhancement, with a Fleiss kappa value of 0.61. Our model was able to accurately predict contrast enhancement in 88.8% of the cases (scores of 2 to 3 on the 3-point scale). We developed a novel deep learning architecture to synthesize virtual postcontrast enhancement by using only conventional noncontrast brain MR images. Our results demonstrate the potential of deep learning methods to reduce the need for gadolinium contrast in the evaluation of primary brain tumors.

High-resolution MRI synthesis using a data-driven framework with denoising diffusion probabilistic modeling

Objective. High-resolution magnetic resonance imaging (MRI) can enhance lesion diagnosis, prognosis, and delineation. However, gradient power and hardware limitations prohibit recording thin slices or sub-1 mm resolution. Furthermore, long scan time is not clinically acceptable. Conventional high-resolution images generated using statistical or analytical methods include the limitation of capturing complex, high-dimensional image data with intricate patterns and structures. This study aims to harness cutting-edge diffusion probabilistic deep learning techniques to create a framework for generating high-resolution MRI from low-resolution counterparts, improving the uncertainty of denoising diffusion probabilistic models (DDPM). Approach. DDPM includes two processes. The forward process employs a Markov chain to systematically introduce Gaussian noise to low-resolution MRI images. In the reverse process, a U-Net model is trained to denoise the forward process images and produce high-resolution images conditioned on the features of their low-resolution counterparts. The proposed framework was demonstrated using T2-weighted MRI images from institutional prostate patients and brain patients collected in the Brain Tumor Segmentation Challenge 2020 (BraTS2020). Main results. For the prostate dataset, the bicubic interpolation model (Bicubic), conditional generative-adversarial network (CGAN), and our proposed DDPM framework improved the noise quality measure from low-resolution images by 4.4%, 5.7%, and 12.8%, respectively. Our method enhanced the signal-to-noise ratios by 11.7%, surpassing Bicubic (9.8%) and CGAN (8.1%). In the BraTS2020 dataset, the proposed framework and Bicubic enhanced peak signal-to-noise ratio from resolution-degraded images by 9.1% and 5.8%. The multi-scale structural similarity indexes were 0.970 ± 0.019, 0.968 ± 0.022, and 0.967 ± 0.023 for the proposed method, CGAN, and Bicubic, respectively. Significance. This study explores a deep learning-based diffusion probabilistic framework for improving MR image resolution. Such a framework can be used to improve clinical workflow by obtaining high-resolution images without penalty of the long scan time. Future investigation will likely focus on prospectively testing the efficacy of this framework with different clinical indications.

LightNet: a novel lightweight convolutional network for brain tumor segmentation in healthcare.

Diagnosis, treatment planning, surveillance, and the monitoring of clinical trials for brain diseases all benefit greatly from neuroimaging-based tumor segmentation. Recently, Convolutional Neural Networks (CNNs) have demonstrated promising results in enhancing the efficiency of image-based brain tumor segmentation. Most current work on CNNs, however, is devoted to creating increasingly complicated convolution modules to improve performance, which in turn raises the computing cost of the model. This work proposes a simple and effective feed-forward CNN, LightNet (Light Network). Based on multi-path and multi-level, it replaces traditional convolutional methods with light operations, which reduces network parameters and redundant feature maps. In the up-sampling stage, a light channel attention module is added to achieve richer multi-scale and spatial semantic feature information extraction of brain tumor. The performance of the network is evaluated in the Multimodal Brain Tumor Segmentation Challenge (BraTS 2015) dataset, and results are presented here alongside other high-performing CNNs. Results show comparable accuracy with other methods but with increased efficiency, segmentation performance, and reduced redundancy and computational complexity. The result is a high-performing network with a balance between efficiency and accuracy, allowing, for example, better energy performance on mobile devices.

MMGan: a multimodal MR brain tumor image segmentation method.

Computer-aided diagnosis has emerged as a rapidly evolving field, garnering increased attention in recent years. At the forefront of this field is the segmentation of lesions in medical images, which is a critical preliminary stage in subsequent treatment procedures. Among the most challenging tasks in medical image analysis is the accurate and automated segmentation of brain tumors in various modalities of brain tumor MRI. In this article, we present a novel end-to-end network architecture called MMGan, which combines the advantages of residual learning and generative adversarial neural networks inspired by classical generative adversarial networks. The segmenter in the MMGan network, which has a U-Net architecture, is constructed using a deep residual network instead of the conventional convolutional neural network. The dataset used for this study is the BRATS dataset from the Brain Tumor Segmentation Challenge at the Medical Image Computing and Computer Assisted Intervention Society. Our proposed method has been extensively tested, and the results indicate that this MMGan framework is more efficient and stable for segmentation tasks. On BRATS 2019, the segmentation algorithm improved accuracy and sensitivity in whole tumor, tumor core, and enhanced tumor segmentation. Particularly noteworthy is the higher dice score of 0.86 achieved by our proposed method in tumor core segmentation, surpassing those of stateof-the-art models. This study improves the accuracy and sensitivity of the tumor segmentation task, which we believe is significant for medical image analysis. And it should be further improved by replacing different loss functions such as cross-entropy loss function and other methods.

Artificial Intelligence-Based Methods for Integrating Local and Global Features for Brain Cancer Imaging: Scoping Review.

Transformer-based models are gaining popularity in medical imaging and cancer imaging applications. Many recent studies have demonstrated the use of transformer-based models for brain cancer imaging applications such as diagnosis and tumor segmentation. This study aims to review how different vision transformers (ViTs) contributed to advancing brain cancer diagnosis and tumor segmentation using brain image data. This study examines the different architectures developed for enhancing the task of brain tumor segmentation. Furthermore, it explores how the ViT-based models augmented the performance of convolutional neural networks for brain cancer imaging. This review performed the study search and study selection following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. The search comprised 4 popular scientific databases: PubMed, Scopus, IEEE Xplore, and Google Scholar. The search terms were formulated to cover the interventions (ie, ViTs) and the target application (ie, brain cancer imaging). The title and abstract for study selection were performed by 2 reviewers independently and validated by a third reviewer. Data extraction was performed by 2 reviewers and validated by a third reviewer. Finally, the data were synthesized using a narrative approach. Of the 736 retrieved studies, 22 (3%) were included in this review. These studies were published in 2021 and 2022. The most commonly addressed task in these studies was tumor segmentation using ViTs. No study reported early detection of brain cancer. Among the different ViT architectures, Shifted Window transformer-based architectures have recently become the most popular choice of the research community. Among the included architectures, UNet transformer and TransUNet had the highest number of parameters and thus needed a cluster of as many as 8 graphics processing units for model training. The brain tumor segmentation challenge data set was the most popular data set used in the included studies. ViT was used in different combinations with convolutional neural networks to capture both the global and local context of the input brain imaging data. It can be argued that the computational complexity of transformer architectures is a bottleneck in advancing the field and enabling clinical transformations. This review provides the current state of knowledge on the topic, and the findings of this review will be helpful for researchers in the field of medical artificial intelligence and its applications in brain cancer.

Multi-class glioma segmentation on real-world data with missing MRI sequences: comparison of three deep learning algorithms

This study tests the generalisability of three Brain Tumor Segmentation (BraTS) challenge models using a multi-center dataset of varying image quality and incomplete MRI datasets. In this retrospective study, DeepMedic, no-new-Unet (nn-Unet), and NVIDIA-net (nv-Net) were trained and tested using manual segmentations from preoperative MRI of glioblastoma (GBM) and low-grade gliomas (LGG) from the BraTS 2021 dataset (1251 in total), in addition to 275 GBM and 205 LGG acquired clinically across 12 hospitals worldwide. Data was split into 80% training, 5% validation, and 15% internal test data. An additional external test-set of 158 GBM and 69 LGG was used to assess generalisability to other hospitals’ data. All models’ median Dice similarity coefficient (DSC) for both test sets were within, or higher than, previously reported human inter-rater agreement (range of 0.74–0.85). For both test sets, nn-Unet achieved the highest DSC (internal = 0.86, external = 0.93) and the lowest Hausdorff distances (10.07, 13.87 mm, respectively) for all tumor classes (p < 0.001). By applying Sparsified training, missing MRI sequences did not statistically affect the performance. nn-Unet achieves accurate segmentations in clinical settings even in the presence of incomplete MRI datasets. This facilitates future clinical adoption of automated glioma segmentation, which could help inform treatment planning and glioma monitoring.

Multiencoder‐based federated intelligent deep learning model for brain tumor segmentation

AbstractGlioma, a primary tumor derived from brain glial cells, is around 45% of all intracranial tumors. Magnetic resonance imaging's (MRI's) precise glioma segmentation is crucial for clinical purposes. This article presents a novel automatic brain tumor segmentation approach based on a multi‐encoder‐based federated intelligent deep learning framework. The suggested method uses a U‐shaped network design that multiplies the single contraction path into several paths to explore semantic information modalities deeply. The basic convolutional layer uses an Inception module and dilated convolutions to extract multi‐scale features from the images using artificial intelligent. To emphasize segmentation‐related information while ignoring redundant channel dimension information and improving the accuracy of network segmentation, lightweight channel attention efficient channel attention (ECA) modules are inserted into the bottleneck layer and decoder. The collection of data for the 2018 Brain Tumor Segmentation Challenge (BraTS 2018) is used to test the effectiveness of the suggested structure, and the findings indicate that the growth core, for the entire tumor and the augmented tumor regions, respectively, the average Dice coefficients are 0.880, 0.784, and 0.757. These findings support the proposed algorithm's ability to accurately and successfully segregate multimodal MRI brain tumors.

Overall Survival Prediction in Stereotactic Radiosurgery Patients with Glioblastoma Via a Deep-Learning Approach

Accurate and automated early survival prediction is critical for glioblastoma (GBM) patients as their poor prognosis requires timely treatment decision-making. We have developed a deep learning (DL)-based GBM overall survival (OS) prediction model based on a multi-institutional public dataset using only pre-operative basic structural multi-parametric magnetic resonance images (MRIs). The purpose of this study is to evaluate this DL-based OS prediction model with an institutional stereotactic radiosurgery (SRS) clinical trial dataset. The task of this study is to classify GBM patients into 3 OS classes: long-survivors (>15 months), mid-survivors (between 10 and 15 months) and short-survivors (< 10 months). The proposed OS prediction model is an ensemble of a ResNet-based classifier and a K-NN classifier. The ResNet-based classifier is trained in a Siamese fashion to explore inter-class differences. During testing, training sample features are implemented with a K-NN classifier to ensemble with the ResNet-based classifier. A public dataset from Medical Image Computing and Computer Assisted Intervention (MICCAI) Brain Tumor Segmentation (BraTS) challenge 2020 (235 patients) were used for model establishing and initial validation. Then the validated model was evaluated on 19 GBM patients from an institutional SRS clinical trial. Each data entry consists of pre-operative basic structural multi-parametric MRIs and survival days, as well as patient ages for BraTS data and basic clinical characteristics for institutional data. GBM sub-regions, including contrast-enhancing tumor, peri-tumoral edema, and necrotic/non-enhancing tumor core, were segmented in the multi-parametric MRIs by an in-house DL model for both datasets. The OS prediction model was trained on 90% of the segmented BraTS data and validated on the rest 10%, then further evaluated on the institutional data. The model performance was assessed by prediction accuracy (ACC) and the area under the curve (AUC). For this 3-class OS classification task, our DL-based prediction model achieved an ACC of 65.22% and an AUC of 0.81 on the BraTS dataset compared with the top-ranked result from the BraTS challenge 2020 (Rank 1st: ACC 61.7%), and an ACC of 52.63% and an AUC of 0.69 on the institutional dataset. Further analysis of the institutional dataset found that the predicted OS class had a statistically significant correlation with treatment volume (p = 0.012) and age (p = 0.006), which matches the analysis that the patients' ground truth OS class is statistical significantly correlated with treatment volume (p = 0.045). Our DL-based OS prediction model for GBM using basic structural multi-parametric pre-operative MRIs has demonstrated promising performance in both public and institutional dataset with minimal manual processing requirements. This OS prediction model can be potentially applied to assist timely clinical decision-making.

An active learning approach to train a deep learning algorithm for tumor segmentation from brain MR images

PurposeThis study focuses on assessing the performance of active learning techniques to train a brain MRI glioma segmentation model.MethodsThe publicly available training dataset provided for the 2021 RSNA-ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge was used in this study, consisting of 1251 multi-institutional, multi-parametric MR images. Post-contrast T1, T2, and T2 FLAIR images as well as ground truth manual segmentation were used as input for the model. The data were split into a training set of 1151 cases and testing set of 100 cases, with the testing set remaining constant throughout. Deep convolutional neural network segmentation models were trained using the NiftyNet platform. To test the viability of active learning in training a segmentation model, an initial reference model was trained using all 1151 training cases followed by two additional models using only 575 cases and 100 cases. The resulting predicted segmentations of these two additional models on the remaining training cases were then addended to the training dataset for additional training.ResultsIt was demonstrated that an active learning approach for manual segmentation can lead to comparable model performance for segmentation of brain gliomas (0.906 reference Dice score vs 0.868 active learning Dice score) while only requiring manual annotation for 28.6% of the data.ConclusionThe active learning approach when applied to model training can drastically reduce the time and labor spent on preparation of ground truth training data.Critical relevance statementActive learning concepts were applied to a deep learning-assisted segmentation of brain gliomas from MR images to assess their viability in reducing the required amount of manually annotated ground truth data in model training.Key points• This study focuses on assessing the performance of active learning techniques to train a brain MRI glioma segmentation model.• The active learning approach for manual segmentation can lead to comparable model performance for segmentation of brain gliomas.• Active learning when applied to model training can drastically reduce the time and labor spent on preparation of ground truth training data.Graphical

Brain Tumor Segmentation from MRI Images Using Handcrafted Convolutional Neural Network.

Brain tumor segmentation from magnetic resonance imaging (MRI) scans is critical for the diagnosis, treatment planning, and monitoring of therapeutic outcomes. Thus, this research introduces a novel hybrid approach that combines handcrafted features with convolutional neural networks (CNNs) to enhance the performance of brain tumor segmentation. In this study, handcrafted features were extracted from MRI scans that included intensity-based, texture-based, and shape-based features. In parallel, a unique CNN architecture was developed and trained to detect the features from the data automatically. The proposed hybrid method was combined with the handcrafted features and the features identified by CNN in different pathways to a new CNN. In this study, the Brain Tumor Segmentation (BraTS) challenge dataset was used to measure the performance using a variety of assessment measures, for instance, segmentation accuracy, dice score, sensitivity, and specificity. The achieved results showed that our proposed approach outperformed the traditional handcrafted feature-based and individual CNN-based methods used for brain tumor segmentation. In addition, the incorporation of handcrafted features enhanced the performance of CNN, yielding a more robust and generalizable solution. This research has significant potential for real-world clinical applications where precise and efficient brain tumor segmentation is essential. Future research directions include investigating alternative feature fusion techniques and incorporating additional imaging modalities to further improve the proposed method's performance.

Combination of multi-modal MRI radiomics and liquid biopsy technique for preoperatively non-invasive diagnosis of glioma based on deep learning: protocol for a double-center, ambispective, diagnostical observational study.

2021 World Health Organization (WHO) Central Nervous System (CNS) tumor classification increasingly emphasizes the important role of molecular markers in glioma diagnoses. Preoperatively non-invasive "integrated diagnosis" will bring great benefits to the treatment and prognosis of these patients with special tumor locations that cannot receive craniotomy or needle biopsy. Magnetic resonance imaging (MRI) radiomics and liquid biopsy (LB) have great potential for non-invasive diagnosis of molecular markers and grading since they are both easy to perform. This study aims to build a novel multi-task deep learning (DL) radiomic model to achieve preoperative non-invasive "integrated diagnosis" of glioma based on the 2021 WHO-CNS classification and explore whether the DL model with LB parameters can improve the performance of glioma diagnosis. This is a double-center, ambispective, diagnostical observational study. One public database named the 2019 Brain Tumor Segmentation challenge dataset (BraTS) and two original datasets, including the Second Affiliated Hospital of Nanchang University, and Renmin Hospital of Wuhan University, will be used to develop the multi-task DL radiomic model. As one of the LB techniques, circulating tumor cell (CTC) parameters will be additionally applied in the DL radiomic model for assisting the "integrated diagnosis" of glioma. The segmentation model will be evaluated with the Dice index, and the performance of the DL model for WHO grading and all molecular subtype will be evaluated with the indicators of accuracy, precision, and recall. Simply relying on radiomics features to find the correlation with the molecular subtypes of gliomas can no longer meet the need for "precisely integrated prediction." CTC features are a promising biomarker that may provide new directions in the exploration of "precision integrated prediction" based on the radiomics, and this is the first original study that combination of radiomics and LB technology for glioma diagnosis. We firmly believe that this innovative work will surely lay a good foundation for the "precisely integrated prediction" of glioma and point out further directions for future research. This study was registered on ClinicalTrails.gov on 09/10/2022 with Identifier NCT05536024.

Deep Machine Learning Algorithms in Glioblastoma (GBM) MRI evaluation (PL4.005)

<h3>Objective:</h3> To test the reliability of automatic segmentation models on post-operative MRIs in glioblastoma evaluation. <h3>Background:</h3> The extent of tumor resection in glioblastoma is an independent prognostic factor associated with clinical outcomes. Because radiological assessment to identify residual tumoral components can be time-consuming and subject to inter-rater variability, the implementation of artificial intelligence (AI) may improve radiological assessment. Hereby, we evaluate performance of different automatic brain tumor segmentation algorithms trained on pre-operative MRI datasets, on post-operative MRI scans of patients with glioblastoma. <h3>Design/Methods:</h3> We evaluated the performance of ten different deep-learning models designed for automated brain tumor segmentation. Models were trained using a publicly available dataset from the Brain Tumor Segmentation (BRaTS) Challenge 2020 and was tested in our institutional MRI scans including pre-operative and post-operative MRI examinations. Semiautomatic segmentation was performed in the institutional scans and was used as the ground truth. Dice Similarity Coefficient (DSC) was used to compare each model’s performance against the ground truth. <h3>Results:</h3> A total of 329 MR examinations were used for training purposes and a separate 100 scans were used to test the deep-learning algorithms. Overall, models demonstrated better performance on pre-operative scans. Among post-operative MRI studies, Knowledge Distillation algorithm exhibited the best performance with a DSC score of 0.836 for whole tumor segmentation. When evaluating specific regions within the tumor, Knowledge Distillation demonstrated a DSC of 0.82 for the FLAIR hyperintense region, 3D Dilated Multi-Fiber Network achieved a DSC of 0.88 in the contrast-enhancing tumoral region, and 3D U-Net exhibited the highest DCS for the non-enhancing core (0.84). <h3>Conclusions:</h3> Deep machine learning has the potential to quantify enhancing and FLAIR volumes in brain MRI from glioblastoma patients with high accuracy. Current automatic segmentation models can perform automatic segmentation on post-operative MRI scans with an 88% accuracy. Further investigations are needed to improve this promising AI technology. <b>Disclosure:</b> An immediate family member of Dr. Zhu has stock in Abbott lab. The institution of Dr. Zhu has received research support from NRG NIH. The institution of Dr. Zhu has received research support from Novocure, Inc. The institution of Dr. Zhu has received research support from Five Prime pharmaceutical. The institution of Dr. Zhu has received research support from Denovo, inc. The institution of Dr. Zhu has received research support from Boston Biomedical Sumitomo Dainippon Pharma Global Oncology. The institution of Dr. Zhu has received research support from CNS pharmaceutical. Mr. HSIEH has nothing to disclose. Ms. Kabir has nothing to disclose. Dr. Nunez has nothing to disclose. Carlos Rodriguez Quintero has nothing to disclose. Dr. Cai has nothing to disclose. Dr. Hsu has nothing to disclose. Dr. Arevalo has nothing to disclose. Kangyi Zhao has nothing to disclose. Jackie Jiaqi Zhang has nothing to disclose. Roy Riascos-Castaneda has nothing to disclose. The institution of Dr. Jiang has received research support from NIH. The institution of Dr. Jiang has received research support from CPRIT. The institution of Dr. Jiang has received research support from NSF. shayan shams has nothing to disclose.

Abstract LB067: A confident and operator-independent deep segmentation model to measure residual tumor volume in the follow-up MRIs for glioblastoma

Abstract Introduction: Magnetic resonance imaging (MRI) is the most common tool to examine glioblastoma. Preoperative MRI can be used to initial diagnosis, and surgery planning, while follow-up MRIs can be used to evaluate treatment responses, identify recurrency, and detect side effects. The follow-up MRIs are usually taken after the first-line therapy, such as maximal safe resection. In the past, radiologists manually segment the tumor regions from normal brain tissue on follow-up MRIs, which is time-consuming, error-prone, and challenging. Several deep-learning (DL) models have been developed utilizing preoperative images, but their performance has yet to be evaluated on follow-up MRIs. In this research, we built the largest follow-up MRI cohort (311 patients) to assess these DL models and their generalizability and performance on independent preoperative and follow-up images. We also made the first follow-up-based deep learning models for this specific task. Methods: All evaluation deep learning models (10 models) were trained by the Brain Tumor Segmentation challenge 2020 (BraTS’20) and evaluated by fifty pairs of preoperative and follow-up scans from our institution. The segmentation form our institution is evaluated by board certified radiologist. MRIs in the BraTS’20 dataset were all preoperative scans. After the evaluation, we randomly assigned 264 patients’ scans from our institution to the training dataset and 47 patients’ scans to the testing dataset. We compared three types of models in our follow-up deep learning model, including 1) UNet-3D, 2) UNet-3D+transfer-learning, and 3) UNet-3D+transfer-learning+baysian-learning. The benchmark for all models was the Dice similarity coefficient (DSC). DSC can measure the spatial overlap between model prediction and ground truth. The value of DSC is between zero to one. Zero means no overlap and one indicates complete overlap. Results: Our study demonstrates that the BraTS'20 trained models' performance decreased by 13.05% in independent preoperative MRI scans and 19.04% in follow-up MRI scans. The most significant mismatch regions were FLAIR hyperintense regions (3.68% drop in independent preoperative scans and 10.61% drop in independent follow-up scans) and Non-enhancing core (5.20% drop in independent preoperative scans and 11.99% drop in independent follow-up scans). Our best model can achieve the best DSC among three tumor regions compared to all evaluation models (FLAIR hyperintense regions: DSC 0.77 V.S. DSC 0.58; Enhancing tumor region: DSC 0.87 V.S. DSC 0.68; Non-enhancing tumor region: DSC 0.92 V.S. DSC 0.55). Conclusion: Maximal safe resection induced brain structure change, decreasing the performance of the preoperative-based DL model. Implementing a follow-up MRI-based segmentation model is essential to make accurate and generalizable results to address structural changes after maximal safe resection. Our follow-up DL model demonstrates the DSC score can be recovered. We commit to further developing the tool to assist radiologists in handling follow-up MRIs for glioblastoma patients. Citation Format: Kang Lin Hsieh, Tanjida Kabir, Luis Nunez-Rubiano, Yu-Chun Hsu, Yu Cai, Juan Rodriguez Quintero, Octavio Arevalo, Kangyi Zhao, Jackie Zhang, Jiguang Zhu Zhu, Roy Riascos, Xioaqian Jiang, Shayan Shams. A confident and operator-independent deep segmentation model to measure residual tumor volume in the follow-up MRIs for glioblastoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 2 (Clinical Trials and Late-Breaking Research); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(8_Suppl):Abstract nr LB067.

Batch size: go big or go home? Counterintuitive improvement in medical autoencoders with smaller batch size.

Batch size is a key hyperparameter in training deep learning models. Conventional wisdom suggests larger batches produce improved model performance. Here we present evidence to the contrary, particularly when using autoencoders to derive meaningful latent spaces from data with spatially global similarities and local differences, such as electronic health records (EHR) and medical imaging. We investigate batch size effects in both EHR data from the Baltimore Longitudinal Study of Aging and medical imaging data from the multimodal brain tumor segmentation (BraTS) challenge. We train fully connected and convolutional autoencoders to compress the EHR and imaging input spaces, respectively, into 32-dimensional latent spaces via reconstruction losses for various batch sizes between 1 and 100. Under the same hyperparameter configurations, smaller batches improve loss performance for both datasets. Additionally, latent spaces derived by autoencoders with smaller batches capture more biologically meaningful information. Qualitatively, we visualize 2-dimensional projections of the latent spaces and find that with smaller batches the EHR network better separates the sex of the individuals, and the imaging network better captures the right-left laterality of tumors. Quantitatively, the analogous sex classification and laterality regressions using the latent spaces demonstrate statistically significant improvements in performance at smaller batch sizes. Finally, we find improved individual variation locally in visualizations of representative data reconstructions at lower batch sizes. Taken together, these results suggest that smaller batch sizes should be considered when designing autoencoders to extract meaningful latent spaces among EHR and medical imaging data driven by global similarities and local variation.

Clinical capability of modern brain tumor segmentation models.

State-of-the-art automated segmentation methods achieve exceptionally high performance on the Brain Tumor Segmentation (BraTS) challenge, a dataset of uniformly processed and standardized magnetic resonance generated images (MRIs) of gliomas. However, a reasonable concern is that these models may not fare well on clinical MRIs that do not belong to the specially curated BraTS dataset. Research using the previous generation of deep learning models indicates significant performance loss on cross-institutional predictions. Here, we evaluate the cross-institutional applicability and generalzsability of state-of-the-art deep learning models on new clinicaldata. We train a state-of-the-art 3D U-Net model on the conventional BraTS dataset comprising low- and high-grade gliomas. We then evaluate the performance of this model for automatic tumor segmentation of brain tumors on in-house clinical data. This dataset contains MRIs of different tumor types, resolutions, and standardization than those found in the BraTS dataset. Ground truth segmentations to validate the automated segmentation for in-house clinical data were obtained from expert radiationoncologists. We report average Dice scores of 0.764, 0.648, and 0.61 for the whole tumor, tumor core, and enhancing tumor, respectively, in the clinical MRIs. These means are higher than numbers reported previously on same institution and cross-institution datasets of different origin using different methods. There is no statistically significant difference when comparing the dice scores to the inter-annotation variability between two expert clinical radiation oncologists. Although performance on the clinical data is lower than on the BraTS data, these numbers indicate that models trained on the BraTS dataset have impressive segmentation performance on previously unseen images obtained at a separate clinical institution. These images differ in the imaging resolutions, standardization pipelines, and tumor types from the BraTSdata. State-of-the-art deep learning models demonstrate promising performance on cross-institutional predictions. They considerably improve on previous models and can transfer knowledge to new types of brain tumors without additionalmodeling.

Attention-based multimodal glioma segmentation with multi-attention layers for small-intensity dissimilarity

The segmentation of glioma by computer vision is one of the hot topics in medical image analysis, which further helps doctors to make a better treatment plan for glioma. At present, convolutional neural networks (CNN) with multi-kernels are the mainstream method to identify glioma regions. However, the segmentation is strongly affected if the intensity dissimilarity between adjacent glioma regions is small. To solve this challenge, we propose an attention-based multimodal glioma segmentation with multi-attention layers for small-intensity dissimilarity to focus on the small-intensity dissimilarity glioma regions. Firstly, to reduce background interferences, this paper proposes data enhancement in glioma-centered regions. In addition, a random multi-dimensional data view is generated in the glioma regions to reduce overfitting. Secondly, we embed a 3D U-Net to proposed attention layers, which focus on the intensity dissimilarity between adjacent glioma regions, and adaptively mine the glioma-related features, solving the problem that the existing algorithms are insensitive to the small-intensity dissimilarity between adjacent glioma regions. In particular, each attention layer can adaptively highlight valuable glioma features and suppress unrelated ones. Finally, experiment results on the multimodal brain tumor segmentation challenge (BraTS) 2020 dataset validate the effectiveness of the proposed method, where the Dice Similarity Coefficients (DSC) are improved on the segmentation of whole tumor (WT), tumor core (TC), and enhanced tumor (ET) regions, reaching higher results at 0.7803, 0.8831 and 0.8172 for WT, TC and ET regions respectively. Also, we make a test on the public dataset BraTS2019, reaching the results at 0.7675, 0.8925, and 0.8110 for WT, TC, and ET regions, respectively.

Region-related focal loss for 3D brain tumor MRI segmentation.

In the brain tumor magnetic resonance image (MRI) segmentation, although the 3D convolution networks (CNNs) has achieved state-of-the-art results, the class and hard-voxel imbalances in the 3D images have not been well addressed. Voxel independent losses are dependent on the setting of class weights for the class imbalance issue, and are hard to assign each class equally. Region-related losses cannot correctly focus on hard voxels dynamically and not be robust to misclassification of small structures. Meanwhile, repeatedly training on the additional hard samples augmented by existing methods would bring more class imbalance, overfitting and incorrect knowledge learning to themodel. A novel region-related loss with balanced dynamic weighting while alleviating the sensitivity to small structures is necessary. In addition, we need to increase the diversity of hard samples in the training to improve the performance ofmodel. The proposed Region-related Focal Loss (RFL) reshapes standard Dice Loss (DL) by up-weighting the loss assigned to hard-classified voxels. Compared to DL, RFL adaptively modulate its gradient with an invariant focalized point that voxels with lower-confidence than it would achieve a larger gradient, and higher-confidence voxels would get a smaller gradient. Meanwhile, RFL can adjust the parameters to set where and how much the network is focused. In addition, an Intra-classly Transformed Augmentation network (ITA-NET) is proposed to increase the diversity of hard samples, in which the 3D registration network and intra-class transfer layer are used to transform the shape and intensity respectively. A selective hard sample mining(SHSM) strategy is used to train the ITA-NET for avoiding excessive class imbalance. Source code (in Tensorflow) is available at: https://github.com/lb-whu/RFL_ITA. The experiments are carried out on public data set: Brain Tumor Segmentation Challenge 2020 (BratS2020). Experiments with BraTS2020 online validation set show that proposed methods achieve an average Dice scores of 0.905, 0.821, and 0.806 for whole tumor (WT), tumor core (TC) and enhancing tumor (ET), respectively. Compared with DL (baseline), the proposed RFL significantly improves the Dice scores by an average of 1%, and for the small region ET it can even increase by 3%. And the proposed method combined with ITA-NET improves the Dice scores of ET and TC by 5% and 3%respectively. The proposed RFL can converge with a invariant focalized point in the training of segmentation network, thus effectively alleviating the hard-voxel imbalance in brain tumor MRI segmentation. The negative region term of RFL can effectively reduce the sensitivity of the segmentation model to the misclassification of small structures. The proposed ITA-NET can increase the diversity of hard samples by transforming their shape and transfer their intra-class intensity, thereby effectively improving the robustness of the segmentation network to hardsamples.