Shallow SegNet with bilinear interpolation and weighted cross-entropy loss for Semantic segmentation of brain tissue

A novel method for tackling the problem of imbalanced data in medical image segmentation is proposed in this work. In Balanced Cross Entropy (BCE) loss, which is a type of weighted CE loss, the weight assigned to each class is the inverse of the class frequency. These balancing weights are expected to equalize the effect of each class on the overall loss and prevent the model from being biased towards the majority class. But, as it has been shown in previous studies, this method degrades the performance by a large margin. Therefore, Balanced CE is not a popular loss in medical segmentation tasks, and usually a region-based loss, like the Dice loss, is used to address the class imbalance problem. Existing weighting CE loss approaches, like Balanced CE, are usually adapted from classification tasks. In this study we propose a weighting method which is designed for segmentation and takes advantage of spatial and geometrical properties of the labels. In the proposed method, the weighting of cross entropy loss for each class is based on a dilated area of each class mask, and balancing weights are assigned to each class together with its surrounding pixels. The goal of this study is to show that the performance of Balanced CE loss can be greatly improved my modifying its weighting strategy. Experiments on different datasets show that the proposed Dilated Balanced CE (DBCE) loss outperforms the Balanced CE loss by a large margin and produces superior results compared to CE loss, and its performance is similar or better compared to the performance of the combination of Dice and CE loss. Also, the performance of different combination losses can be improved by replacing the CE loss with the proposed DBCE loss. This study shows that a weighted cross entropy loss with the right weighing strategy can be as effective as a region-based loss in handling the class imbalance problem.

A neural network is a mathematical model that is able to perform a task automatically or semi-automatically after learning the human knowledge that we provided. Moreover, a Convolutional Neural Network (CNN) is a type of neural network that has shown to efficiently learn tasks related to the area of image analysis, such as image segmentation, whose main purpose is to find regions or separable objects within an image. A more specific type of segmentation, called semantic segmentation, guarantees that each region has a semantic meaning by giving it a label or class. Since CNNs can automate the task of image semantic segmentation, they have been very useful for the medical area, applying them to the segmentation of organs or abnormalities (tumors). This work aims to improve the task of binary semantic segmentation of volumetric medical images acquired by Magnetic Resonance Imaging (MRI) using a preexisting Three-Dimensional Convolutional Neural Network (3D CNN) architecture. We propose a formulation of a loss function for training this 3D CNN, for improving pixel-wise segmentation results. This loss function is formulated based on the idea of adapting a similarity coefficient, used for measuring the spatial overlap between the prediction and ground truth, and then using it to train the network. As contribution, the developed approach achieved good performance in a context where the pixel classes are imbalanced. We show how the choice of the loss function for training can affect the final quality of the segmentation. We validate our proposal over two medical image semantic segmentation datasets and show comparisons in performance between the proposed loss function and other pre-existing loss functions used for binary semantic segmentation.

A neural network is a mathematical model that is able to perform a task automatically or semi-automatically after learning the human knowledge that we provided. Moreover, a Convolutional Neural Network (CNN) is a type of neural network that has shown to efficiently learn tasks related to the area of image analysis, such as image segmentation, whose main purpose is to find regions or separable objects within an image. A more specific type of segmentation, called semantic segmentation, guarantees that each region has a semantic meaning by giving it a label or class. Since CNNs can automate the task of image semantic segmentation, they have been very useful for the medical area, applying them to the segmentation of organs or abnormalities (tumors). This work aims to improve the task of binary semantic segmentation of volumetric medical images acquired by Magnetic Resonance Imaging (MRI) using a pre-existing Three-Dimensional Convolutional Neural Network (3D CNN) architecture. We propose a formulation of a loss function for training this 3D CNN, for improving pixel-wise segmentation results. This loss function is formulated based on the idea of adapting a similarity coefficient, used for measuring the spatial overlap between the prediction and ground truth, and then using it to train the network. As contribution, the developed approach achieved good performance in a context where the pixel classes are imbalanced. We show how the choice of the loss function for training can affect the nal quality of the segmentation. We validate our proposal over two medical image semantic segmentation datasets and show comparisons in performance between the proposed loss function and other pre-existing loss functions used for binary semantic segmentation.

Semantic image segmentation (SiS) plays a fundamental role in a broad variety of computer vision applications, providing key information for the global understanding of an image. This survey is an effort to summarize two decades of research in the field of SiS, where we propose a literature review of solutions starting from early historical methods followed by an overview of more recent deep learning methods including the latest trend of using transformers. We complement the review by discussing particular cases of the weak supervision and side machine learning techniques that can be used to improve the semantic segmentation such as curriculum, incremental or self-supervised learning. State-of-the-art SiS models rely on a large amount of annotated samples, which are more expensive to obtain than labels for tasks such as image classification. Since unlabeled data is instead significantly cheaper to obtain, it is not surprising that Unsupervised Domain Adaptation (UDA) reached a broad success within the semantic segmentation community. Therefore, a second core contribution of this monograph is to summarize five years of a rapidly growing field, Domain Adaptation for Semantic Image Segmentation (DASiS) which embraces the importance of semantic segmentation itself and a critical need of adapting segmentation models to new environments. In addition to providing a comprehensive survey on DASiS techniques, we unveil also newer trends such as multi-domain learning, domain generalization, domain incremental learning, test-time adaptation and source-free domain adaptation. Finally, we conclude this survey by describing datasets and benchmarks most widely used in SiS and DASiS and briefly discuss related tasks such as instance and panoptic image segmentation, as well as applications such as medical image segmentation. We hope that this monograph will provide researchers across academia and industry with a comprehensive reference guide and will help them in fostering new research directions in the field.

Need for medical imaging has immensely increased to diagnose functionality of organs and tissues. Application of magnetic resonance imaging (MRI) or computerized tomography scan (CT scan) has automated the process of disease prediction which otherwise would need manual intervention of experienced physicians. Critical transplants and tumor diagnosis of kidney, heart and liver could be very challenging without the application of high-resolution medical imaging. Semantic image segmentation helps to identify features within the image and label them to demarcate the background from organs and tumors formed on them. Current proposed model is built by applying image segmentation using deep learning models by training 3D image information obtained from KITS-19 using CNN U-Net architecture to segregate essential tumor details from the kidney background images. Considering the evaluation of medical condition based on imagery is very challenging due to lack of pre-processing methods to handle unbalanced nature of class distribution which is common in medical imaging. In the current research, we apply state-of-the-art convolution neural network and deep learning models to predict the tumor in MRI imagery. We performed evaluation of a subset of 60 CT scans with slice thickness values obtained from KiTS19 dataset. We also compared the image segmentation model with BRATS’2013 challenging dataset also to validate the prediction of tumor in brain MRI scan images. Performance evaluation of kidney and tumor features obtained through threefold cross validation comparing with ground truth gave us dice coefficient score of 0.96 while tumor segmentation prediction was close to 0.80. We believe the performance can be increased through higher GPU memory requirements, which was a limitation to our existing GPU hardware. Nevertheless, the possibility of applying improved deep learning model based on existing results and can be a promising step forward in medical imaging segmentation.

Deep learning techniques have rapidly become important as a preferred method for evaluating medical image segmentation. This survey analyses different contributions in the deep learning medical field, including the major common issues published in recent years, and also discusses the fundamentals of deep learning concepts applicable to medical image segmentation. The study of deep learning can be applied to image categorization, object recognition, segmentation, registration, and other tasks. First, the basic ideas of deep learning techniques, applications, and frameworks are introduced. Deep learning techniques that operate the ideal applications are briefly explained. This paper indicates that there is a previous experience with different techniques in the class of medical image segmentation. Deep learning has been designed to describe and respond to various challenges in the field of medical image analysis such as low accuracy of image classification, low segmentation resolution, and poor image enhancement. Aiming to solve these present issues and improve the evolution of medical image segmentation challenges, we provide suggestions for future research.

Holistic decomposition convolution for effective semantic segmentation of medical volume images.

Neural architecture search (NAS) has significant progress in improving the accuracy of image classification. Recently, some works attempt to extend NAS to image segmentation which shows preliminary feasibility. However, all of them focus on searching architecture for semantic segmentation in natural scenes. In this paper, we design three types of primitive operation set on search space to automatically find two cell architecture DownSC and UpSC for semantic image segmentation especially medical image segmentation. Inspired by the U-net architecture and its variants successfully applied to various medical image segmentation, we propose NAS-Unet which is stacked by the same number of DownSC and UpSC on a U-like backbone network. The architectures of DownSC and UpSC updated simultaneously by a differential architecture strategy during the search stage. We demonstrate the good segmentation results of the proposed method on Promise12, Chaos, and ultrasound nerve datasets, which collected by magnetic resonance imaging, computed tomography, and ultrasound, respectively. Without any pretraining, our architecture searched on PASCAL VOC2012, attains better performances and much fewer parameters (about 0.8M) than U-net and one of its variants when evaluated on the above three types of medical image datasets.

Brain tumor segmentation in medical image analysis is a challenging task. Deep learning techniques have recently shown promise in resolving a variety of computer vision problems, such as semantic segmentation and image classification. Brain MRI (magnetic resonance imaging) requires precise brain image segmentation for effective, rapid diagnosis and treatment planning. However, it is quite difficult to manually segment the brain image rapidly and accurately in low-quality, noisy images. This paper proposes a U-Net and combined attention mechanism-based method. This research enhances the segmentation of images of tumors in the brain using modified U-net. Traditional U-net segmentation techniques are still widely used in the medical field, but they have a number of limitations when dealing with small targets and fuzzier boundaries. To address this issue, we made the following modifications to U-net: We propose attention mechanisms to assist the network in concentrating on important regions. The multiscale feature fusion strategy improves the efficacy of network segmentation at various scales. Cross-entropy loss function and data augmentation improve the performance of the network further. Our method was validated using the Brats2019 dataset. The experimental results demonstrate that our proposed methodology exhibits superior speed and efficiency compared to existing techniques in the context of brain image segmentation. The dice coefficients for the multiple branch TS-U-net model were 0.876, 0.868, and 0.814 in the tumor subregions of WT, TC, and ET, respectively. This exemplifies the feasibility and potential of our methodology for the segmentation of medical images.

Semantic segmentation is a significant research topic for decades and has been employed in several applications. In recent years, semantic segmentation has been focused on different deep learning approaches in the area of computer vision, which has aimed for getting superior efficiency while analyzing the aerial and remote-sensing images. The main aim of this review is to provide a clear algorithmic categorization and analysis of the diverse contribution of semantic segmentation of aerial images and expects to give the comprehensive details associated with the recent developments. In addition, the emerged deep learning methods demonstrated much improved performance measures on several public datasets and incredible efforts have been dedicated to advancing pixel-level accuracy. Hence, the analysis on diverse datasets of each contribution is studied, and also, the best performance measures achieved by the existing semantic segmentation models are evaluated. Thus, this survey can facilitate researchers in understanding the development of semantic segmentation in a shorter time, simplify understanding of its latest advancements, research gaps, and challenges to be used as a reference for developing the new semantic image segmentation models in the future.

We present a new model-based framework for coupled segmentation and de-noising of medical images. The segmentation and de-noising steps are coupled through a discrete formulation of the total variation de-noising problem in a restricted setting such that each pixel in the image has its de-noised intensity level selected from a drastically reduced set of intensities. By creating such a reduced set of intensity levels, in which each intensity level represent the intensity across a region to be segmented, the intensity value for each de-noised pixel will be forced to assume a value in this limited set; by associating all pixels with the same de-noised value as a single region, image segmentation is naturally achieved. We derive two formulations corresponding to two noise models: additive white Gaussian and multiplicative Rayleigh. We furthermore show that the proposed framework enables globally optimal foreground/background segmentation of images with Rayleigh distributed noise.

To increase the accuracy of medical image analysis using supervised learning-based AI technology, a large amount of accurately labeled training data is required. However, the supervised learning approach may not be applicable to real-world medical imaging due to the lack of labeled data, the privacy of patients, and the cost of specialized knowledge. To handle these issues, we utilized Kronecker-factored decomposition, which enhances both computational efficiency and stability of the learning process. We combined this approach with a model-agnostic meta-learning framework for the parameter optimization. Based on this method, we present a bidirectional meta-Kronecker factored optimizer (BM-KFO) framework to quickly optimize semantic segmentation tasks using just a few magnetic resonance imaging (MRI) images as input. This model-agnostic approach can be implemented without altering network components and is capable of learning the learning process and meta-initial points while training on previously unseen data. We also incorporated a combination of average Hausdorff distance loss (AHD-loss) and cross-entropy loss into our objective function to specifically target the morphology of organs or lesions in medical images. Through evaluation of the proposed method on the abdominal MRI dataset, we obtained an average performance of 78.07% in setting 1 and 79.85% in setting 2. Our experiments demonstrate that BM-KFO with AHD-loss is suitable for general medical image segmentation applications and achieves superior performance compared to the baseline method in few-shot learning tasks. In order to replicate the proposed method, we have shared our code on GitHub. The corresponding URL can be found: https://github.com/YeongjoonKim/BMKFO.git.

Accurate segmentation and analysis of thyroid nodules in ultrasound (US) images are essential for the diagnosis and management of thyroid conditions, including cancer. Despite advancements in medical imaging, achieving accurate and efficient segmentation remains a significant challenge due to the complexity and variability of US images. Recently, deep learning (DL) techniques, such as convolutional neural networks (CNNs) and vision transformers (ViTs), have emerged as powerful tools for computer-aided diagnosis (CAD). This review highlights recent advancements in thyroid US image segmentation, focusing on state-of-the-art DL techniques such as contrastive learning, consistency learning, and knowledge-driven DL. We explore various approaches to improve segmentation accuracy, including multi-task learning, self-supervised learning, and methods that minimize reliance on the availability of large, annotated datasets. Additionally, we examine the clinical significance of these methods in differentiating between benign and malignant nodules, as well as their potential for integration into clinically adopted, fully automated CAD systems. By addressing the latest developments and ongoing challenges, this review serves as a comprehensive reference for future research and clinical implementation of thyroid US diagnostics.

Medical image segmentation using deep semantic-based methods: A review of techniques, applications and emerging trends

The various methods which have been adopted in processing and segmentation of medical images using deep learning and machine learning techniques are examined and analyzed in this article. Medical images analysis and their methods have been swiftly evolved into deep learning techniques and especially in convolutional neural networks. Deep learning concepts that may be used for the image classification, detection, and logging of medical related pictures and objects have been examined. Medical applications include: research and investigations into neuro, retinal and pulmonary, digital pathologies, breast, heart and musculoskeletal diseases and their corresponding analysis. Deep learning has already been used to accurately diagnose diseases and classify image samples, and it has the potential to revolutionise the entire landscape of healthcare. These uses are only expected to expand in the future. The most recent developments, including a critical analysis of the current problems have been summarized, and made plans for additional research in medical imaging.