Region-wise loss for biomedical image segmentation

Segmentation of breast lesions on dynamic contrast enhanced (DCE) magnetic resonance imaging (MRI) is the first step in lesion diagnosis in a computer-aided diagnosis framework. Because manual segmentation of such lesions is both time consuming and highly susceptible to human error and issues of reproducibility, an automated lesion segmentation method is highly desirable. Traditional automated image segmentation methods such as boundary-based active contour (AC) models require a strong gradient at the lesion boundary. Even when region-based terms are introduced to an AC model, grayscale image intensities often do not allow for clear definition of foreground and background region statistics. Thus, there is a need to find alternative image representations that might provide (1) strong gradients at the margin of the object of interest (OOI); and (2) larger separation between intensity distributions and region statistics for the foreground and background, which are necessary to halt evolution of the AC model upon reaching the border of the OOI. In this paper, the authors introduce a spectral embedding (SE) based AC (SEAC) for lesion segmentation on breast DCE-MRI. SE, a nonlinear dimensionality reduction scheme, is applied to the DCE time series in a voxelwise fashion to reduce several time point images to a single parametric image where every voxel is characterized by the three dominant eigenvectors. This parametric eigenvector image (PrEIm) representation allows for better capture of image region statistics and stronger gradients for use with a hybrid AC model, which is driven by both boundary and region information. They compare SEAC to ACs that employ fuzzy c-means (FCM) and principal component analysis (PCA) as alternative image representations. Segmentation performance was evaluated by boundary and region metrics as well as comparing lesion classification using morphological features from SEAC, PCA+AC, and FCM+AC. On a cohort of 50 breast DCE-MRI studies, PrEIm yielded overall better region and boundary-based statistics compared to the original DCE-MR image, FCM, and PCA based image representations. Additionally, SEAC outperformed a hybrid AC applied to both PCA and FCM image representations. Mean dice similarity coefficient (DSC) for SEAC was significantly better (DSC = 0.74 ± 0.21) than FCM+AC (DSC = 0.50 ± 0.32) and similar to PCA+AC (DSC = 0.73 ± 0.22). Boundary-based metrics of mean absolute difference and Hausdorff distance followed the same trends. Of the automated segmentation methods, breast lesion classification based on morphologic features derived from SEAC segmentation using a support vector machine classifier also performed better (AUC = 0.67 ± 0.05; p < 0.05) than FCM+AC (AUC = 0.50 ± 0.07), and PCA+AC (AUC = 0.49 ± 0.07). In this work, we presented SEAC, an accurate, general purpose AC segmentation tool that could be applied to any imaging domain that employs time series data. SE allows for projection of time series data into a PrEIm representation so that every voxel is characterized by the dominant eigenvectors, capturing the global and local time-intensity curve similarities in the data. This PrEIm allows for the calculation of strong tensor gradients and better region statistics than the original image intensities or alternative image representations such as PCA and FCM. The PrEIm also allows for building a more accurate hybrid AC scheme.

Brain tumor (BT) segmentation from brain magnetic resonance imaging (MRI) plays an important role in diagnosis and treatment planning for patients. In this study, we proposed a new approach for brain tumor segmentation based on deep neural networks. The paper proposes to use Attention U-Net architecture which can handle the shape variety with the attention gate for brain tumor segmentation from MRI images. Especially, instead of using cross-entropy loss function, dice coefficient loss function or both, we propose to utilize a new loss function based on activate contour loss that is known to overcome the limitation of pixel-wise fitting of the segmentation map on the loss functions used before, to train the network. We evaluated and compared our approach and other approaches on a dataset of nearly 4000 brain MRI scans. Experiments demonstrate that the proposed method outperforms the state-of-the-art methods in terms of Dice coefficient and Jaccard indexes.KeywordsBrain tumor segmentationActivate contour modelAttention U-NetAttention gateU-Net

Reliable performance of PET segmentation algorithms on clinically relevant tasks is required for their clinical translation. However, these algorithms are typically evaluated using figures of merit (FoMs) that are not explicitly designed to correlate with clinical task performance. Such FoMs include the Dice similarity coefficient (DSC), the Jaccard similarity coefficient (JSC), and the Hausdorff distance (HD). The objective of this study was to investigate whether evaluating PET segmentation algorithms using these task-agnostic FoMs yields interpretations consistent with evaluation on clinically relevant quantitative tasks. Methods: We conducted a retrospective study to assess the concordance in the evaluation of segmentation algorithms using the DSC, JSC, and HD and on the tasks of estimating the metabolic tumor volume (MTV) and total lesion glycolysis (TLG) of primary tumors from PET images of patients with non-small cell lung cancer. The PET images were collected from the American College of Radiology Imaging Network 6668/Radiation Therapy Oncology Group 0235 multicenter clinical trial data. The study was conducted in 2 contexts: (1) evaluating conventional segmentation algorithms, namely those based on thresholding (SUVmax40% and SUVmax50%), boundary detection (Snakes), and stochastic modeling (Markov random field-Gaussian mixture model); (2) evaluating the impact of network depth and loss function on the performance of a state-of-the-art U-net-based segmentation algorithm. Results: Evaluation of conventional segmentation algorithms based on the DSC, JSC, and HD showed that SUVmax40% significantly outperformed SUVmax50%. However, SUVmax40% yielded lower accuracy on the tasks of estimating MTV and TLG, with a 51% and 54% increase, respectively, in the ensemble normalized bias. Similarly, the Markov random field-Gaussian mixture model significantly outperformed Snakes on the basis of the task-agnostic FoMs but yielded a 24% increased bias in estimated MTV. For the U-net-based algorithm, our evaluation showed that although the network depth did not significantly alter the DSC, JSC, and HD values, a deeper network yielded substantially higher accuracy in the estimated MTV and TLG, with a decreased bias of 91% and 87%, respectively. Additionally, whereas there was no significant difference in the DSC, JSC, and HD values for different loss functions, up to a 73% and 58% difference in the bias of the estimated MTV and TLG, respectively, existed. Conclusion: Evaluation of PET segmentation algorithms using task-agnostic FoMs could yield findings discordant with evaluation on clinically relevant quantitative tasks. This study emphasizes the need for objective task-based evaluation of image segmentation algorithms for quantitative PET.

In CT-based medical image segmentation, the choice of loss function profoundly impacts the training efficacy of deep neural networks. Traditional loss functions like cross entropy (CE), Dice, Boundary, and TopK each have unique strengths and limitations, often introducing biases when used individually. This study aims to enhance segmentation accuracy by optimizing ensemble loss functions, thereby addressing the biases and limitations of single loss functions and their linearcombinations. We implemented a comprehensive evaluation of loss function combinations by integrating CE, Dice, Boundary, and TopK loss functions through both loss-level linear combination and model-level ensemble methods. Our approach utilized two state-of-the-art 3D segmentation architectures, Attention U-Net (AttUNet) and SwinUNETR, to test the impact of these methods. The study was conducted on two large CT dataset cohorts: an institutional dataset containing pelvic organ segmentations, and a public dataset consisting of multiple organ segmentations. All the models were trained from scratch with different loss settings, and performance was evaluated using Dice similarity coefficient (DSC), Hausdorff distance (HD), and average surface distance (ASD). In the ensemble approach, both static averaging and learnable dynamic weighting strategies were employed to combine the outputs of models trained with different lossfunctions. Extensive experiments revealed the following: (1) the linear combination of loss functions achieved results comparable to those of single loss-driven methods; (2) compared to the best non-ensemble methods, ensemble-based approaches resulted in a 2%-7% increase in DSC scores, along with notable reductions in HD (e.g., a 19.1% reduction for rectum segmentation using SwinUNETR) and ASD (e.g., a 49.0% reduction for prostate segmentation using AttUNet); (3) the learnable ensemble approach with optimized weights produced finer details in predicted masks, as confirmed by qualitative analyses; and (4) the learnable ensemble consistently outperforms the static ensemble across most metrics (DSC, HD, ASD) for both AttUNet and SwinUNETR architectures. Our findings support the efficacy of using ensemble models with optimized weights to improve segmentation accuracy, highlighting the potential for broader applications in automated medical imageanalysis.

Active contours or snakes are widely used for segmentation and tracking. Recently a new active contour model was proposed, combining edge and region information. The method has a convex energy function, thus becoming invariant to the initialization of the active contour. This method is promising, but has no regularization term. Therefore segmentation results of this method are highly dependent of the quality of the images. We propose a new active contour model which also uses region and edge information, but which has an extra regularization term. This work provides an efficient optimization scheme based on Split Bregman for the proposed active contour method. It is experimentally shown that the proposed method has significant better results in the presence of noise and clutter.

The loss function plays an important role in deep learning models as it determines the model convergence behavior and performance. In semantic segmentation, many methods utilize pixel-wise (e.g. cross-entropy) and region-wise (e.g. dice) losses while boundary-wise loss is underexplored. It is known that one of the key aims of semantic segmentation is to precisely delineate objects' boundaries. Hence, it is essential to design a loss function that measures the errors around objects' boundaries. Fuzzy rough sets are constituted by the fuzzy equivalence relation, which is commonly used to measure the difference between two sets. In this paper, the lower approximation of fuzzy rough sets is proposed to construct the boundary-wise loss in deep learning models for the first time. The experiments with various segmentation models and datasets have verified that the proposed fuzzy rough sets loss is superior to other boundary-wise losses in terms of segmentation accuracy and time complexity. Compared with the commonly used pixel-wise and region-wise losses, the proposed boundary-wise loss performs similarly in dice coefficient, pixel-wise accuracy, but has a better performance in Hausdorff distance and symmetric surface distance. It indicates that the proposed loss provides a better guidance for segmentation models in producing more accurate shapes of the target objects. Code is available online at Github: https://github.com/qiaolin1992/Boundary-Loss.

This study aims to develop a framework for automated mandibular canal segmentation in cone beam computed tomography (CBCT) scans. The dataset, source code, and trained models are publicly accessible, allowing for reproducibility and further development by the research community. A total of 236 CBCT scans were collected from the Stomatology Hospital of the Shantou University Medical College, and the mandibular canals in these scans were manually annotated with fine granularity. A custom-designed 3D U-Net, named ManCan_ResU-Net, along with two commonly used 3D U-Net models, was employed as candidate models. The soft Dice Similarity Coefficient (DSC) loss was used as the loss function. During inference, a post-processing step involving connected components analysis and removal of small disconnected objects was applied to refine the segmentation results. Model performance was evaluated using following metrics: voxel accuracy (ACC), sensitivity (SEN), specificity (SPE), DSC, Hausdorff distance (HD), 95th percentile Hausdorff distance (HD95), average surface distance (ASD), and average symmetric surface distance (ASSD). The MCSTU dataset, which contains a development dataset (218 CBCT images) and an independent test dataset (18 CBCT images) with fine-grained annotations, has been made publicly available. The validation loss of ManCan_ResU-Net was lower than those of two commonly used models. Incorporating post-processing significantly improved model performance, particularly by reducing the HD metric. On the hold-out test dataset, the ManCan_ResU-Net model achieved ACC, SEN, SPE, DSC, HD, HD95, ASD, ASSD with 95% confidence interval of 1 (1–1), 0.86 (0.83–0.87), 1 (1–1), 0.85 (0.83–0.86), 10.1 (8.67–13.6), 1.8 (1.6–2.2), 0.69 (0.58–0.85), and 0.72 (0.6–0.83), respectively. On the test dataset, the ManCan_ResU-Net model obtained ACC, SEN, SPE, DSC, HD, HD95, ASD, ASSD with 95% confidence interval of 1 (1–1), 0.93 (0.91–0.95), 1 (1–1), 0.80 (0.79–0.81), 21.3 (11.7–53.9), 2.59 (2.33–3), 1 (0.96–1.21), and 0.92 (0.861–1), respectively. Both the code and trained models are publicly available. The proposed segmentation framework achieved strong performance on both the hold-out and independent test datasets. In the future, after further validation of the model’s generalization ability, it may be applied in real clinical settings for oral surgery planning.

The Hausdorff Distance (HD) is widely used in evaluating medical image segmentation methods. However, the existing segmentation methods do not attempt to reduce HD directly. In this paper, we present novel loss functions for training convolutional neural network (CNN)-based segmentation methods with the goal of reducing HD directly. We propose three methods to estimate HD from the segmentation probability map produced by a CNN. One method makes use of the distance transform of the segmentation boundary. Another method is based on applying morphological erosion on the difference between the true and estimated segmentation maps. The third method works by applying circular/spherical convolution kernels of different radii on the segmentation probability maps. Based on these three methods for estimating HD, we suggest three loss functions that can be used for training to reduce HD. We use these loss functions to train CNNs for segmentation of the prostate, liver, and pancreas in ultrasound, magnetic resonance, and computed tomography images and compare the results with commonly-used loss functions. Our results show that the proposed loss functions can lead to approximately 18-45% reduction in HD without degrading other segmentation performance criteria such as the Dice similarity coefficient. The proposed loss functions can be used for training medical image segmentation methods in order to reduce the large segmentation errors.

The aim of this study was to find a new loss function to automatically segment temporal lobes on localized CT images for radiotherapy with more accuracy and a solution to dealing with the classification of class-imbalanced samples in temporal lobe segmentation. Localized CT images for radiotherapy of 70 patients with nasopharyngeal carcinoma were selected. Radiation oncologists sketched mask maps. The dataset was randomly divided into the training set (n = 49), the validation set (n = 7), and the test set (n = 14). The training set was expanded by rotation, flipping, zooming, and shearing, and the models were evaluated using Dice similarity coefficient (DSC), Jaccard similarity coefficient (JSC), positive predictive value (PPV), sensitivity (SE), and Hausdorff distance (HD). This study presented an improved loss function, focal generalized Dice-binary cross-entropy loss (FGD-BCEL), and compared it with four other loss functions, Dice loss (DL), generalized Dice loss (GDL), Tversky loss (TL), and focal Tversky loss (FTL), using the U-Net model framework. With the U-Net model based on FGD-BCEL, the DSC, JSC, PPV, SE, and HD were 0.87 ± 0.11, 0.78 ± 0.11, 0.90 ± 0.10, 0.87 ± 0.13, and 4.11 ± 0.75, respectively. Except for the SE, all the other evaluation metric values of the temporal lobes segmented by the FGD-BCEL-based U-Net model were improved compared to the DL, GDL, TL, and FTL loss function-based U-Net models. Moreover, the FGD-BCEL-based U-Net model was morphologically more similar to the mask maps. The over- and under-segmentation was lessened, and it effectively segmented the tiny structures in the upper and lower poles of the temporal lobe with a limited number of samples. For the segmentation of the temporal lobe on localized CT images for radiotherapy, the U-Net model based on the FGD-BCEL can meet the basic clinical requirements and effectively reduce the over- and under-segmentation compared with the U-Net models based on the other four loss functions. However, there still exists some over- and under-segmentation in the results, and further improvement is needed.

OAR sparing 3D radiotherapy planning supported by fMRI brain mapping investigations

Dosimetric Advantages of Online Adaptative Radiotherapy for Cervical Cancer on 1.5T MR-Linac

Gray matter segmentation of the spinal cord with active contours in MR images

Objective: This study aims to develop a publicly accessible dataset for mandibular canal segmentation in cone beam computed tomography (CBCT) scans and to propose a framework for automated mandibular canal segmentation. Methods: A total of 236 CBCT scans were collected from the Stomatology Hospital of the Shantou University Medical College, and the mandibular canals in these files were finely annotated. A custom designed 3D UNet, named MyResUNet, along with two commonly used UNet models, were used as candidate models. Soft dice similarity coefficient (DSC) loss was used as the loss function. A post-processing step involving connected components analysis and removing small objects was applied during inference. Model performance was assessed using voxel accuracy (ACC), sensitivity (SEN), specificity (SPE),DSC(1), Hausdorff distance (HD), the 95th percentile Hausdorff distance (HD95), average surface distance (ASD), and average symmetric surface distance (ASSD). Results: The MCSTU dataset, which contains a development dataset and an independent test dataset comprising 218 and 18 CBCT images with fine-grained annotations, respectively, has been made publicly available. The validation loss of MyResUNet was lower than that of two commonly used models. The inclusion of post-processing significantly enhanced the performance, especially by reducing the HD metric. On the hold-out test dataset, the MyResUNet model achieved ACC, SEN, SPE, DSC, HD, HD95, ASD, ASSD with 95% confidence interval of 1 (1-1), 0.86 (0.83-0.87), 1 (1-1), 0.85 (0.83-0.86), 10.1 (8.67-13.6), 1.8 (1.6-2.2), 0.69 (0.58-0.85), and 0.72 (0.6-0.83), respectively. On the test dataset, the MyResUNet model obtained ACC, SEN, SPE, DICE, HD, HD95, ASD, ASSD(2, 3)with 95% confidence interval of 1 (1-1), 0.93 (0.91-0.95), 1 (1-1), 0.80 (0.79-0.81), 21.3 (11.7-53.9), 2.59 (2.33-3), 1 (0.96-1.21), and 0.92 (0.861-1), respectively. Both the code and trained models are publicly available. Conclusion: The proposed segmentation framework achieved strong performance on both the hold-out and independent test datasets. In the future, after further validation of the model’s generalization ability, it may be applied in real clinical settings for oral surgery planning.

Background: Manual delineation of volumes for prostate radiotherapy treatment is a time-consuming task for radiation oncologists and is also prone to variability.Deep learning-based auto-segmentation methods showed promising results with accurate and high-fidelity contours. Objective: The objective of this study was to evaluate the feasibility of a Computed Tomography (CT)-based deep learning auto-segmentation algorithm for multi-organ delineation in prostate radiotherapy.Material and Methods: In this single-institution retrospective study, a total of 118 patients with prostate cancer were included. We applied 3D nnU-net deep convolutional neural network architecture,a self-adapting ensemble method for simultaneous fast and reproducible multi-organ auto-contouring. The dataset was randomly divided into training and test sets from 95 and 23 patients,respectively. Intensity-modulated radiotherapy plans were generated for both manual and automatic delineations using identical optimization settings. Contours were assessed in terms of the DiceSimilarity Coefficient (DSC), and average Hausdorff Distance (HD). Dose distributions were additionally evaluated using parameters derived from Dose-Volume Histograms (DVH). Results: On the test set, 3D nnU-net achieved the best performance in the bladder (DSC:0.97, HD:4.13), right femur head (DSC:0.96, HD:3.58), left femur head (DSC:0.96, HD:3.95), rectum (DSC:0.9, HD:10.04), prostate (DSC:0.82, HD:3.68), lymph nodes (DSC:0.77, HD:15.5), and seminal vesicles (DSC:0.69, HD:10.95). DVH parameters of targets and Organ at Risks (OARs) were significantly different except for lymph nodes and femoral heads between treatment plans based on manual and automatic contours. Conclusion: The 3D nnU-net architecture can be successfully used for multi-organ segmentation in the male pelvic area.

Active contours driven by local and global intensity fitting energies based on local entropy