Network For Tumor Segmentation Research Articles

Edge-guided multi-scale adaptive feature fusion network for liver tumor segmentation.

Automated segmentation of liver tumors on CT scans is essential for aiding diagnosis and assessing treatment. Computer-aided diagnosis can reduce the costs and errors associated with manual processes and ensure the provision of accurate and reliable clinical assessments. However, liver tumors in CT images vary significantly in size and have fuzzy boundaries, making it difficult for existing methods to achieve accurate segmentation. Therefore, this paper proposes MAEG-Net, a multi-scale adaptive feature fusion liver tumor segmentation network based on edge guidance. Specifically, we design a multi-scale adaptive feature fusion module that effectively incorporates multi-scale information to better guide the segmentation of tumors of different sizes. Additionally, to address the problem of blurred tumor boundaries in images, we introduce an edge-aware guidance module to improve the model's feature learning ability under these conditions. Evaluation results on the liver tumor dataset (LiTS2017) show that our method achieves a Dice coefficient of 71.84% and a VOE of 38.64%, demonstrating the best performance for liver tumor segmentation in CT images.

FTSNet: Fundus Tumor Segmentation Network on Multiple Scales Guided by Classification Results and Prompts.

The segmentation of fundus tumors is critical for ophthalmic diagnosis and treatment, yet it presents unique challenges due to the variability in lesion size and shape. Our study introduces Fundus Tumor Segmentation Network (FTSNet), a novel segmentation network designed to address these challenges by leveraging classification results and prompt learning. Our key innovation is the multiscale feature extractor and the dynamic prompt head. Multiscale feature extractors are proficient in eliciting a spectrum of feature information from the original image across disparate scales. This proficiency is fundamental for deciphering the subtle details and patterns embedded in the image at multiple levels of granularity. Meanwhile, a dynamic prompt head is engineered to engender bespoke segmentation heads for each image, customizing the segmentation process to align with the distinctive attributes of the image under consideration. We also present the Fundus Tumor Segmentation (FTS) dataset, comprising 254 pairs of fundus images with tumor lesions and reference segmentations. Experiments demonstrate FTSNet's superior performance over existing methods, achieving a mean Intersection over Union (mIoU) of 0.8254 and mean Dice (mDice) of 0.9042. The results highlight the potential of our approach in advancing the accuracy and efficiency of fundus tumor segmentation.

Leveraging segmentation-guided spatial feature embedding for overall survival prediction in glioblastoma with multimodal magnetic resonance imaging

Background and objectivePatients with glioblastoma have a five-year relative survival rate of less than 5 %. Thus, accurately predicting the overall survival (OS) of patients with glioblastoma is crucial for effective treatment planning. MethodsTo fully leverage the imaging characteristics of glioblastomas, we propose a segmentation-guided regression method for predicting OS of patients with brain tumors using multimodal magnetic resonance imaging. Specifically, a brain tumor segmentation network was first pre-trained without leveraging survival information. Subsequently, the survival regression network was jointly trained with the guidance of brain tumor segmentation, focusing on tumor voxels and suppressing irrelevant backgrounds. ResultsOur proposed framework, based on the well-known backbone of UNETR++, achieved a Dice score of 0.7910, Spearman correlation of 0.4112, and Harrell's concordance index of 0.6488. The model consistently showed promising results compared with baseline methods on two different datasets (BraTS and UCSF-PDGM). Furthermore, ablation studies on our training configurations demonstrated that both the pre-training segmentation network and contrastive loss significantly improved all metrics for OS prediction. ConclusionsIn this study, we propose a joint learning framework based on a pre-trained segmentation backbone for OS prediction by leveraging a brain tumor segmentation map. By utilizing a spatial feature map, our model can operate using a sliding-window approach, which can be adopted by varying the matrix sizes and resolutions of the input images.

Identifying radiogenomic associations of breast cancer based on DCE-MRI by using Siamese Neural Network with manufacturer bias normalization.

The immunohistochemical test (IHC) for Human Epidermal Growth Factor Receptor 2 (HER2) and hormone receptors (HR) provides prognostic information and guides treatment for patients with invasive breast cancer. The objective of this paper is to establish a non-invasive system for identifying HER2 and HR in breast cancer using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). In light of the absence of high-performance algorithms and external validation in previously published methods, this study utilizes 3D deep features and radiomics features to represent the information of the Region of Interest (ROI). A Siamese Neural Network was employed as the classifier, with 3D deep features and radiomics features serving as the network input. To neutralize manufacturer bias, a batch effect normalization method, ComBat, was introduced. To enhance the reliability of the study, two datasets, Predict Your Therapeutic Response with Imaging and moLecular Analysis (I-SPY 1) and I-SPY 2, were incorporated. I-SPY 2 was utilized for model training and validation, while I-SPY 1 was exclusively employed for external validation. Additionally, a breast tumor segmentation network was trained to improve radiomic feature extraction. The results indicate that our approach achieved an average Area Under the Curve (AUC) of 0.632, with a Standard Error of the Mean (SEM) of 0.042 for HER2 prediction in the I-SPY 2 dataset. For HR prediction, our method attained an AUC of 0.635 (SEM 0.041), surpassing other published methods in the AUC metric. Moreover, the proposed method yielded competitive results in other metrics. In external validation using the I-SPY 1 dataset, our approach achieved an AUC of 0.567 (SEM 0.032) for HR prediction and 0.563 (SEM 0.033) for HER2 prediction. This study proposes a non-invasive system for identifying HER2 and HR in breast cancer. Although the results do not conclusively demonstrate superiority in both tasks, they indicate that the proposed method achieved good performance and is a competitive classifier compared to other reference methods. Ablation studies demonstrate that both radiomics features and deep features for the Siamese Neural Network are beneficial for the model. The introduced manufacturer bias normalization method has been shown to enhance the method's performance. Furthermore, the external validation of the method enhances the reliability of this research. Source code, pre-trained segmentation network, Radiomics and deep features, data for statistical analysis, and Supporting Information of this article are online at: https://github.com/FORRESTHUACHEN/Siamese_Neural_Network_based_Brest_cancer_Radiogenomic.

SRTRP-Net: A multi-task learning network for segmentation and prediction of stereotactic radiosurgery treatment response in brain metastases

Before the Stereotactic Radiosurgery (SRS) treatment, it is of great clinical significance to avoid secondary genetic damage and guide the personalized treatment plans for patients with brain metastases (BM) by predicting the response to SRS treatment of brain metastatic lesions. Thus, we developed a multi-task learning model termed SRTRP-Net to provide prior knowledge of BM ROI and predict the SRS treatment response of the lesion. In dual-encoder tumor segmentation Network (DTS-Net), two parallel encoders encode the original and mirrored multi-modal MRI images. The differences in the dual-encoder features between foreground and background are enhanced by the symmetrical visual difference block (SVDB). In the bottom layer of the encoder, a transformer is used to extract local contextual features in the spatial and depth dimensions of low-resolution images. Then, the decoder of DTS-Net provides the prior knowledge for predicting the response to SRS treatment by performing BM segmentation. SRS response prediction network (SRP-Net) directly utilizes shared multi-modal MRI features weighted by the signed distance map (SDM) of the masks. The bidirectional multi-dimensional feature fusion module (BMDF) fuses the shared features and the clinical text information features to obtain comprehensive tumor information for characterizing tumors and predicting SRS treatment response. Experiments based on internal and external clinical datasets have shown that SRTRP-Net achieves comparable or better results. We believe that SRTRP-Net can help clinicians accurately develop personalized first-time treatment regimens for BM patients and improve their survival.

Abstract 6173: The development and evaluation of a convolutional neural network for melanoma detection in whole slide images

Abstract The current melanoma staging system is predictive of 74% of the variance in survival, with prognostic biomarkers subject to high levels of inter-observer variation. The application of convolutional neural networks (CNNs) to whole slide images (WSIs), may reveal new insights into tumor morphology and therefore patient prognosis. Melanoma morphology appears to be of greater significance than in other solid tumors, with Breslow thickness remaining the strongest prognostic indicator. Other biomarkers based on tumor morphology have been generated and although some have been found to be prognostically superior to Breslow thickness, none have been integrated into clinical workflows. This may in part be explained by their demands on pathologist time. Therefore, this work outlines the development and evaluation of a CNN for invasive cutaneous melanoma detection in WSIs, which may be used for prognostic biomarker generation. 1,157 WSIs containing cutaneous melanoma from five datasets (three from the University of Leeds, one from the Melanoma Institute Australia, as well as The Cancer Genome Atlas) have been used in the initial development and evaluation of the CNN. A custom-designed 2-class tumor segmentation network with a fully convolutional architecture was trained using annotations. The CNN was evaluated using various methodologies, including comparison at per-pixel and per-tumor levels as compared to manual annotation, as well as variation across 3 scanning platforms (Leica Aperio AT2 (Milton Keynes, UK), Roche Ventana DP600 (Arizona, US) and Hamamatsu NanoZoomer S360 (Hamamatsu City, Japan). The CNN detected and located invasive melanoma tissue of no specific type with an average per-pixel sensitivity and specificity of 97.6% and 99.9% respectively across the 5 test sets. There were no statistical differences between tumor dimensions generated by the CNN as compared to manual annotation. Similarly, there were no statistically significant differences between CNN generated tumor dimensions across three scanning platforms. We have developed and performed initial evaluation of a CNN which appears to accurately detect invasive cutaneous melanoma of no specific type in WSIs for objective evaluation of tumor morphology. Future work should interrogate these data further for its propensity to predict survival outcomes. Citation Format: Emily L. Clarke, Derek Magee, Julia Newton-Bishop, William Merchant, Marlous Hall, Robert Insall, Nigel Maher, Richard Scolyer, Grace Farnworth, Anisah Ali, Sally O'Shea, Darren Treanor. The development and evaluation of a convolutional neural network for melanoma detection in whole slide images [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 6173.

DGCBG-Net: A dual-branch network with global cross-modal interaction and boundary guidance for tumor segmentation in PET/CT images

Background and objectivesAutomatic tumor segmentation plays a crucial role in cancer diagnosis and treatment planning. Computed tomography (CT) and positron emission tomography (PET) are extensively employed for their complementary medical information. However, existing methods ignore bilateral cross-modal interaction of global features during feature extraction, and they underutilize multi-stage tumor boundary features. MethodsTo address these limitations, we propose a dual-branch tumor segmentation network based on global cross-modal interaction and boundary guidance in PET/CT images (DGCBG-Net). DGCBG-Net consists of 1) a global cross-modal interaction module that extracts global contextual information from PET/CT images and promotes bilateral cross-modal interaction of global feature; 2) a shared multi-path downsampling module that learns complementary features from PET/CT modalities to mitigate the impact of misleading features and decrease the loss of discriminative features during downsampling; 3) a boundary prior-guided branch that extracts potential boundary features from CT images at multiple stages, assisting the semantic segmentation branch in improving the accuracy of tumor boundary segmentation. ResultsExtensive experiments are conducted on STS and Hecktor 2022 datasets to evaluate the proposed method. The average Dice scores of our DGCB-Net on the two datasets are 80.33% and 79.29%, with average IOU scores of 67.64% and 70.18%. DGCB-Net outperformed the current state-of-the-art methods with a 1.77% higher Dice score and a 2.12% higher IOU score. ConclusionsExtensive experimental results demonstrate that DGCBG-Net outperforms existing segmentation methods, and is competitive to state-of-arts.

MPEDA-Net: A lightweight brain tumor segmentation network using multi-perspective extraction and dense attention

Malignant brain tumors are highly deadly, necessitating the quickly precise segmentation of tumor regions. Previously, clinicians manually classified brain tumor regions utilizing magnetic resonance imaging (MRI). A new trend is the use of computer vision processing to assist clinicians in clinical analysis. Even though numerous recent methodologies based on CNN have been presented, there remains a lack of high performance when evaluating regions in MRI images. Furthermore, there is still a possibility for development in terms of parameter number and computational complexity. To collect feature information and improve the relevance of contextual feature extraction, a multi-perspective extraction (MPE) module is proposed. MPE consists of three different convolutional kernels and special operations. In addition, a dense attention (DA) module is used to provide each point with an appropriate level of attention while fusing features. The effectiveness of these two modules has been proved through ablation experiments. The proposed MPEDA-Net achieves dice of 82.52%, 93.07%, and 87.67% on the R-BraTS2021 (reconstructed-BraTS2021) in the ET, WT, and TC respectively. In addition, the BraTS2018 and BraTS2019 experiments illustrate that the dice of ET, WT, and TC reach 82.44%, 91.38%, 88.27%, and 83.73%, 91.87%, 88.71%, respectively. The more effective segmentation performance shows that MPEDA-Net can significantly enhance brain tumor segmentation accuracy, exceeding several existing methods. The MPEDA-Net code is already available on GitHub: https://github.com/luohaohaoluo/MPEDANet-pytorch.

Multi-modal brain tumor segmentation via disentangled representation learning and region-aware contrastive learning

Brain tumors are threatening the life and health of people in the world. Automatic brain tumor segmentation using multiple MR images is challenging in medical image analysis. It is known that accurate segmentation relies on effective feature learning. Existing methods address the multi-modal MR brain tumor segmentation by explicitly learning a shared feature representation. However, these methods fail to capture the relationship between MR modalities and the feature correlation between different target tumor regions. In this paper, I propose a multi-modal brain tumor segmentation network via disentangled representation learning and region-aware contrastive learning. Specifically, a feature fusion module is first designed to learn the valuable multi-modal feature representation. Subsequently, a novel disentangled representation learning is proposed to decouple the fused feature representation into multiple factors corresponding to the target tumor regions. Furthermore, contrastive learning is presented to help the network extract tumor region-related feature representations. Finally, the segmentation results are obtained using the segmentation decoders. Quantitative and qualitative experiments conducted on the public datasets, BraTS 2018 and BraTS 2019, justify the importance of the proposed strategies, and the proposed approach can achieve better performance than other state-of-the-art approaches. In addition, the proposed strategies can be extended to other deep neural networks.

ELTS-Net: An enhanced liver tumor segmentation network with augmented receptive field and global contextual information

The liver is one of the organs with the highest incidence rate in the human body, and late-stage liver cancer is basically incurable. Therefore, early diagnosis and lesion location of liver cancer are of important clinical value. This study proposes an enhanced network architecture ELTS-Net based on the 3D U-Net model, to address the limitations of conventional image segmentation methods and the underutilization of image spatial features by the 2D U-Net network structure. ELTS-Net expands upon the original network by incorporating dilated convolutions to increase the receptive field of the convolutional kernel. Additionally, an attention residual module, comprising an attention mechanism and residual connections, replaces the original convolutional module, serving as the primary components of the encoder and decoder. This design enables the network to capture contextual information globally in both channel and spatial dimensions. Furthermore, deep supervision modules are integrated between different levels of the decoder network, providing additional feedback from deeper intermediate layers. This constrains the network weights to the target regions and optimizing segmentation results. Evaluation on the LiTS2017 dataset shows improvements in evaluation metrics for liver and tumor segmentation tasks compared to the baseline 3D U-Net model, achieving 95.2% liver segmentation accuracy and 71.9% tumor segmentation accuracy, with accuracy improvements of 0.9% and 3.1% respectively. The experimental results validate the superior segmentation performance of ELTS-Net compared to other comparison models, offering valuable guidance for clinical diagnosis and treatment.

Diffusion MRI anomaly detection in glioma patients

Diffusion-MRI (dMRI) measures molecular diffusion, which allows to characterize microstructural properties of the human brain. Gliomas strongly alter these microstructural properties. Delineation of brain tumors currently mainly relies on conventional MRI-techniques, which are, however, known to underestimate tumor volumes in diffusely infiltrating glioma. We hypothesized that dMRI is well suited for tumor delineation, and developed two different deep-learning approaches. The first diffusion-anomaly detection architecture is a denoising autoencoder, the second consists of a reconstruction and a discrimination network. Each model was exclusively trained on non-annotated dMRI of healthy subjects, and then applied on glioma patients’ data. To validate these models, a state-of-the-art supervised tumor segmentation network was modified to generate groundtruth tumor volumes based on structural MRI. Compared to groundtruth segmentations, a dice score of 0.67 ± 0.2 was obtained. Further inspecting mismatches between diffusion-anomalous regions and groundtruth segmentations revealed, that these colocalized with lesions delineated only later on in structural MRI follow-up data, which were not visible at the initial time of recording. Anomaly-detection methods are suitable for tumor delineation in dMRI acquisitions, and may further enhance brain-imaging analysis by detection of occult tumor infiltration in glioma patients, which could improve prognostication of disease evolution and tumor treatment strategies.

NIMG-30. DEEP LEARNING SEGMENTATION OF INFILTRATIVE AND ENHANCING CELLULAR TUMOR IN POST-TREATMENT GLIOBLASTOMA PATIENTS WITH MULTI-SHELL DIFFUSION MRI

Abstract Multimodal MRI is used for the evaluation of tumor burden in glioblastoma (GBM) patients after surgical resection, radiation, and chemotherapy. However, it can be challenging to identify non-enhancing infiltrative tumor and differentiate recurrent enhancing tumor from post-treatment changes. Restriction spectrum imaging (RSI), an advanced multishell diffusion technique, has shown promise in distinguishing cellular tumor from edema and enhancing post-treatment changes including pseudoprogression and radiation necrosis. We developed a convolutional network (nnUNet) that incorporates traditional multimodal MRI with DSC perfusion and RSI to detect and quantify enhancing and nonenhancing cellular tumor in post-treatment glioblastoma patients. This cellular tumor segmentation network was evaluated in a cohort of 186 post-treatment GBM patients (55.5 +/- 13.7 years, 158 male) with 243 timepoints using 5-fold cross validation. Cellular tumor volumes were then used to predict overall survival (OS) and progression free survival (PFS) in 94 patients with imaging within 3 months of initial surgery. Accuracy for segmenting cellular tumor as measured by the Dice-similarity coefficient was 0.77 (IQR 0.55-0.87) and the AUC of the ROC curve for detecting residual/recurrent tumor compared to post-treatment changes was 0.86. The predicted cellular tumor volumes were an independent hazard over age, sex, GTR status and conventional enhancing residual tumor for both OS (p=0.0003) and PFS (p=0.0004). The tumor segmentation network was validated in an external dataset of 22 GBM patients (55 timepoints) with a median Dice of 0.73 and AUC of 0.91. Survival prediction was validated in a different external dataset of 70 post-treatment GBM patients, with cellular tumor volumes being an independent predictor of OS (p=0.03). In conclusion, deep learning models incorporating advanced imaging can accurately segment enhancing and nonenhancing cellular tumor, which can help distinguish recurrent/residual tumor from post-treatment changes and predict OS and PFS.

LSAM: L2-norm self-attention and latent space feature interaction for automatic 3D multi-modal head and neck tumor segmentation

Objective. Head and neck (H&N) cancers are prevalent globally, and early and accurate detection is absolutely crucial for timely and effective treatment. However, the segmentation of H&N tumors is challenging due to the similar density of the tumors and surrounding tissues in CT images. While positron emission computed tomography (PET) images provide information about the metabolic activity of the tissue and can distinguish between lesion regions and normal tissue. But they are limited by their low spatial resolution. To fully leverage the complementary information from PET and CT images, we propose a novel and innovative multi-modal tumor segmentation method specifically designed for H&N tumor segmentation. Approach. The proposed novel and innovative multi-modal tumor segmentation network (LSAM) consists of two key learning modules, namely L2-Norm self-attention and latent space feature interaction, which exploit the high sensitivity of PET images and the anatomical information of CT images. These two advanced modules contribute to a powerful 3D segmentation network based on a U-shaped structure. The well-designed segmentation method can integrate complementary features from different modalities at multiple scales, thereby improving the feature interaction between modalities. Main results. We evaluated the proposed method on the public HECKTOR PET-CT dataset, and the experimental results demonstrate that the proposed method convincingly outperforms existing H&N tumor segmentation methods in terms of key evaluation metrics, including DSC (0.8457), Jaccard (0.7756), RVD (0.0938), and HD95 (11.75). Significance. The innovative Self-Attention mechanism based on L2-Norm offers scalability and is effective in reducing the impact of outliers on the performance of the model. And the novel method for multi-scale feature interaction based on Latent Space utilizes the learning process in the encoder phase to achieve the best complementary effects among different modalities.

RMAU-Net: Breast Tumor Segmentation Network Based on Residual Depthwise Separable Convolution and Multiscale Channel Attention Gates

Breast cancer is one of the most common female diseases, posing a great threat to women’s health, and breast ultrasound imaging is a common method for breast cancer diagnosis. In recent years, U-Net and its variants have dominated the medical image segmentation field with their excellent performance. However, the existing U-type segmentation networks have the following problems: (1) the design of the feature extractor is complicated, and the calculation difficulty is increased; (2) the skip connection operation simply combines the features of the encoder and the decoder, without considering both spatial and channel dimensions; (3) during the downsampling phase, the pooling operation results in the loss of feature information. To address the above deficiencies, this paper proposes a breast tumor segmentation network, RMAU-Net, that combines residual depthwise separable convolution and a multi-scale channel attention gate. Specifically, we designed the RDw block, which has a simple structure and a larger sensory field, to overcome the localization problem of convolutional operations. Meanwhile, the MCAG module is designed to correct the low-level features in both spatial and channel dimensions and assist the high-level features to recover the up-sampling and pinpoint non-regular breast tumor features. In addition, this paper used the Patch Merging operation instead of the pooling method to prevent the loss of breast ultrasound image information. Experiments were conducted on two breast ultrasound datasets, Dataset B and BUSI, and the results show that the method in this paper has superior segmentation performance and better generalization.

SPA-UNet: A liver tumor segmentation network based on fused multi-scale features.

Liver tumor segmentation is a critical part in the diagnosis and treatment of liver cancer. While U-shaped convolutional neural networks (UNets) have made significant strides in medical image segmentation, challenges remain in accurately segmenting tumor boundaries and detecting small tumors, resulting in low segmentation accuracy. To improve the segmentation accuracy of liver tumors, this work proposes space pyramid attention (SPA)-UNet, a novel image segmentation network with an encoder-decoder architecture. SPA-UNet consists of four modules: (1) Spatial pyramid convolution block (SPCB), extracting multi-scale features by fusing three sets of dilated convolutions with different rates. (2) Spatial pyramid pooling block (SPPB), performing downsampling to reduce image size. (3) Upsample module, integrating dense positional and semantic information. (4) Residual attention block (RA-Block), enabling precise tumor localization. The encoder incorporates 5 SPCBs and 4 SPPBs to capture contextual information. The decoder consists of the Upsample module and RA-Block, and finally a segmentation head outputs segmented images of liver and liver tumor. Experiments using the liver tumor segmentation dataset demonstrate that SPA-UNet surpasses the traditional UNet model, achieving a 1.0 and 2.0% improvement in intersection over union indicators for liver and tumors, respectively, along with increased recall rates by 1.2 and 1.8%. These advancements provide a dependable foundation for liver cancer diagnosis and treatment.

Comparison of Image Normalization Methods for Multi-Site Deep Learning

In this study, we evaluate the influence of normalization on the performance of deep learning networks for tumor segmentation and the prediction of the pathological response of locally advanced rectal cancer to neoadjuvant chemoradiotherapy. The techniques were applied to a multicenter and multimodal magnet resonance imaging data set consisting of 201 patients recorded at six centers. We implemented and investigated six different normalization methods (setting the mean and standard deviation, histogram matching, percentiles, combining percentiles and histogram matching, fixed window and an auto-encoder with adversarial loss using the imaging parameters) and evaluated their impact on four deep learning tasks: tumor segmentation, prediction of treatment outcome, and prediction of sex and age. The latter two tasks were implemented as a reference test. We trained a modified U-Net with different normalization methods in multiple configurations: on all images, images from all centers except one, and images from a single center. Our results show that normalization only plays a minor role in segmentation, with a difference in Dice of less than 0.02 between the best and worst performing networks. For the prediction of sex and treatment outcomes, the percentile method combined with histogram matching works best for all scenarios. The biggest difference in performance, depending on the normalization method, occurs for classification. In conclusion, normalization is especially important for small data sets or for generalizing to different data distributions. The deep learning method was superior to the classical methods only in a minority of cases, probably due to the limited amount of training data.

Prediction of brain tumor recurrence location based on multi-modal fusion and nonlinear correlation learning.

Brain tumor is one of the leading causes of cancer death. The high-grade brain tumors are easier to recurrent even after standard treatment. Therefore, developing a method to predict brain tumor recurrence location plays an important role in the treatment planning and it can potentially prolong patient's survival time. There is still little work to deal with this issue. In this paper, we present a deep learning-based brain tumor recurrence location prediction network. Since the dataset is usually small, we propose to use transfer learning to improve the prediction. We first train a multi-modal brain tumor segmentation network on the public dataset BraTS 2021. Then, the pre-trained encoder is transferred to our private dataset for extracting the rich semantic features. Following that, a multi-scale multi-channel feature fusion model and a nonlinear correlation learning module are developed to learn the effective features. The correlation between multi-channel features is modeled by a nonlinear equation. To measure the similarity between the distributions of original features of one modality and the estimated correlated features of another modality, we propose to use Kullback-Leibler divergence. Based on this divergence, a correlation loss function is designed to maximize the similarity between the two feature distributions. Finally, two decoders are constructed to jointly segment the present brain tumor and predict its future tumor recurrence location. To the best of our knowledge, this is the first work that can segment the present tumor and at the same time predict future tumor recurrence location, making the treatment planning more efficient and precise. The experimental results demonstrated the effectiveness of our proposed method to predict the brain tumor recurrence location from the limited dataset.

Brain Tumor Segmentation Network with Multi-View Ensemble Discrimination and Kernel-Sharing Dilated Convolution

Accurate segmentation of brain tumors from magnetic resonance 3D images (MRI) is critical for clinical decisions and surgical planning. Radiologists usually separate and analyze brain tumors by combining images of axial, coronal, and sagittal views. However, traditional convolutional neural network (CNN) models tend to use information from only a single view or one by one. Moreover, the existing models adopt a multi-branch structure with different-size convolution kernels in parallel to adapt to various tumor sizes. However, the difference in the convolution kernels' parameters cannot precisely characterize the feature similarity of tumor lesion regions with various sizes, connectivity, and convexity. To address the above problems, we propose a hierarchical multi-view convolution method that decouples the standard 3D convolution into axial, coronal, and sagittal views to provide complementary-view features. Then, every pixel is classified by ensembling the discriminant results from the three views. Moreover, we propose a multi-branch kernel-sharing mechanism with a dilated rate to obtain parameter-consistent convolution kernels with different receptive fields. We use the BraTS2018 and BraTS2020 datasets for comparison experiments. The average Dice coefficients of the proposed network on the BraTS2020 dataset can reach 78.16%, 89.52%, and 83.05% for the enhancing tumor (ET), whole tumor (WT), and tumor core (TC), respectively, while the number of parameters is only 0.5 M. Compared with the baseline network for brain tumor segmentation, the accuracy was improved by 1.74%, 0.5%, and 2.19%, respectively.

A deep learning based four-fold approach to classify brain MRI: BTSCNet

Incorrect diagnosis of brain tumor types prevent appropriate response to medical assistance and reduces patients' chances of survival. Examining MRI images of the patient's brain manually is one traditional method for distinguishing brain tumors, but it is time consuming and prone to human errors. Thus, an automated and new deep learning based four-fold method, Brain Tumor Segmentation and Classification Network (BTSCNet), is proposed in this article which will be helpful for physicians to classify three types of brain tumors (meningioma, glioma, and pituitary tumor) from T1-weighted contrast-enhanced MRI (CE-MRI) images properly. The proposed four folds includes: segmentation of brain tumor region using Brain Tumor Segmentation Network (BTSNet), ROI selection using morphological operation, feature extraction using multi-region gray level co-occurrence matrix (MR-GLCM), andclassification using Sliding Window Euclidean distance (SWED) measure. The lack of annotated training samples is the main challenge in deep-learning-based brain MRIimage classification. Thus, while training the proposed system, the scale, orientation and flip of the input image is randomly changed in the first fold of BTSCNetso that the network can be trained with varied or augmentedimages with respect toscale, orientation and flip.Four performance measures are used to measure the efficiency of the proposed method. The output of the proposed method is tested on a public database where the correct classification rates obtained are96.6% (meningioma), 98.1% (glioma), and 95.3% (pituitary tumor), when considering MR-Contrast feature, MR-Correlation and MR-Homogeneity feature respectively, which indicates that the efficiency of the proposed method.

MSRA-Net: Tumor segmentation network based on Multi-scale Residual Attention

Automatic Medical segmentation of medical images is an important part in the field of computer medical diagnosis, among which tumor segmentation is an important branch of medical image segmentation. Accurate automatic segmentation method is very important in medical diagnosis and treatment. Positron emission computed tomography (PET) and X-ray computed tomography (CT) images are widely used in medical image segmentation to help doctors accurately locate information such as tumor location and shape, providing metabolic and anatomical information, respectively. At present, PET/CT images have not been effectively combined in the research of medical image segmentation, and the complementary semantic information between the superficial and deep layers of neural network has not been ensured. To solve the above problems, this paper proposed a Multi-scale Residual Attention network (MSRA-Net) for tumor segmentation of PET/CT. We first use an attention-fusion based approach to automatically learn the tumor-related areas of PET images and weaken the irrelevant area. Then, the segmentation results of PET branch are processed to optimize the segmentation results of CT branch by using attention mechanism. The proposed neural network (MSRA-Net) can effectively fuse PET image and CT image, which can improve the precision of tumor segmentation by using complementary information of the multi-modal image, and reduce the uncertainty of single modal image segmentation. Proposed model uses multi-scale attention mechanism and residual module, which fuse multi-scale features to form complementary features of different scales. We compare with state-of-the-art medical image segmentation methods. The experiment showed that the Dice coefficient of the proposed network in soft tissue sarcoma and lymphoma datasets increased by 8.5% and 6.1% respectively compared with UNet, showing a significant improvement.