Explainability based Panoptic brain tumor segmentation using a hybrid PA-NET with GCNN-ResNet50

Abstract

Automatic segmentation is a difficult task due to the enormous amount of information offered by the Magnetic Resonance Imaging (MRI) and the variation in tumor’s location, the shape and size of the tumor. An Explainable Deep Learning Architecture for brain tumor segmentation, which offers significant insights into the decision-making process is presented. Panoptic segmentation is presented in this study, which analyzes the method with explainable deep learning and takes uncertainty into account. The main idea is to eliminate the uncertainties of the image, increase tumor identification accuracy, and apply the modified Grad-CAM method to create an explainable deep learning network that could boost confidence in medical professionals. The suggested strategy includes:(1) hybrid deep learning models for segmenting brain tumors while taking uncertainties into account, considering both the semantic and instance labels; (2) Panoptic segmentation using hybrid PA-NET with GCNN-ResNet50 for brain tumor identification considering uncertainty to improve accuracy; and (3) Explainability is examined using the modified Grad-CAM approach ensuring the model’s decisions are not only precise but also clear and understandable. Several tests performed on brain tumor segmentation datasets, BraTS 2021 and BraTS 2019 revealed that the suggested hybrid approach considerably increases tumor segmentation accuracy and achieves the highest performance. The suggested method can be used to identify actual brain tumors with competitive segmentation accuracy and trustworthy outcomes for physicians with visual explanations. Healthcare practitioners can gain an enhanced understanding of model’s decision-making through suggested framework, which builds confidence, allows for more informed clinical judgments, and aids in precise segmentation.