Deep Learning Segmentation Methods Research Articles

Semantic Segmentation with High-Resolution Sentinel-1 SAR Data

The world’s high-resolution images are supplied by a radar system named Synthetic Aperture Radar (SAR). Semantic SAR image segmentation proposes a computer-based solution to make segmentation tasks easier. When conducting scientific research, accessing freely available datasets and images with low noise levels is rare. However, SAR images can be accessed for free. We propose a novel process for labeling Sentinel-1 SAR radar images, which the European Space Agency (ESA) provides free of charge. This process involves denoising the images and using an automatically created dataset with pioneering deep neural networks to augment the results of the semantic segmentation task. In order to exhibit the power of our denoising process, we match the results of our newly created dataset with speckled noise and noise-free versions. Thus, we attained a mean intersection over union (mIoU) of 70.60% and overall pixel accuracy (PA) of 92.23 with the HRNet model. These deep learning segmentation methods were also assessed with the McNemar test. Our experiments on the newly created Sentinel-1 dataset establish that combining our pipeline with deep neural networks results in recognizable improvements in challenging semantic segmentation accuracy and mIoU values.

A review of deep learning segmentation methods for carotid artery ultrasound images

The carotid artery is a critical blood vessel that supplies blood to the brain, and its health and function are essential for preventing cardiovascular diseases such as stroke. Ultrasound imaging is commonly used to diagnose the carotid artery and monitor its health, but traditional methods have limitations in terms of accuracy and efficiency. In recent years, deep learning segmentation methods have been developed to improve the diagnosis of the carotid artery, which have shown great potential for improving the accuracy and efficiency of cardiovascular diagnosis. In this paper, we aim to review and summarize the recent research on deep learning segmentation methods for the carotid artery ultrasound images. Specifically, we focus on techniques for the segmentation of the intima-media, plaque, and lumen sites, which are important for clinical diagnosis. Through our analysis of the literature, we seek to identify the key trends and challenges in this field, and to provide insights into the opportunities and challenges for future research and development in this area.

AAU-Net: An Adaptive Attention U-Net for Breast Lesions Segmentation in Ultrasound Images.

Various deep learning methods have been proposed to segment breast lesions from ultrasound images. However, similar intensity distributions, variable tumor morphologies and blurred boundaries present challenges for breast lesions segmentation, especially for malignant tumors with irregular shapes. Considering the complexity of ultrasound images, we develop an adaptive attention U-net (AAU-net) to segment breast lesions automatically and stably from ultrasound images. Specifically, we introduce a hybrid adaptive attention module (HAAM), which mainly consists of a channel self-attention block and a spatial self-attention block, to replace the traditional convolution operation. Compared with the conventional convolution operation, the design of the hybrid adaptive attention module can help us capture more features under different receptive fields. Different from existing attention mechanisms, the HAAM module can guide the network to adaptively select more robust representation in channel and space dimensions to cope with more complex breast lesions segmentation. Extensive experiments with several state-of-the-art deep learning segmentation methods on three public breast ultrasound datasets show that our method has better performance on breast lesions segmentation. Furthermore, robustness analysis and external experiments demonstrate that our proposed AAU-net has better generalization performance in the breast lesion segmentation. Moreover, the HAAM module can be flexibly applied to existing network frameworks. The source code is available on https://github.com/CGPxy/AAU-net.

MDCF_Net: A Multi-dimensional hybrid network for liver and tumor segmentation from CT

The segmentation of the liver and liver tumors is critical in the diagnosis of liver cancer, and the high mortality rate of liver cancer has made it one of the most popular areas for segmentation research. Some deep learning segmentation methods outperformed traditional methods in terms of segmentation results. However, they are unable to obtain satisfactory segmentation results due to blurred original image boundaries, the presence of noise, very small lesion sites, and other factors. In this paper, we propose MDCF_Net, which has dual encoding branches composed of CNN and CnnFormer and can fully utilize multi-dimensional image features. First, it extracts both intra-slice and inter-slice information and improves the accuracy of the network output by symmetrically using multi-dimensional fusion layers. In the meantime, we propose a novel feature map stacking approach that focuses on the correlation of adjacent channels of two feature maps, improving the network's ability to perceive 3D features. Furthermore, the two coding branches collaborate to obtain both texture and edge features, and the network segmentation performance is further improved. Extensive experiments were carried out on the public datasets LiTS to determine the optimal slice thickness for this task. The superiority of the segmentation performance of our proposed MDCF_Net was confirmed by comparison with other leading methods on two public datasets, the LiTS and the 3DIRCADb.

Artificial intelligence aided diagnosis of pulmonary nodules segmentation and feature extraction

To develop a high-accuracy low-dose computed tomography (LDCT) lung nodule diagnosis system by combining artificial intelligence (AI) technology with the Lung CT Screening Reporting and Data System (Lung-RADS), which can be used in the future AI-aided diagnosis of pulmonary nodules. The study comprised the following steps: (1) the best deep-learning segmentation method for pulmonary nodules was compared and selected objectively; (2) the Image Biomarker Standardization Initiative (IBSI) was used for feature extraction and to determine the best feature reduction method; and (3) a principal component analysis (PCA) and three machine learning methods were used to analyse the extracted features, and the best method was determined. The Lung Nodule Analysis 16 dataset was applied to train and test the established system in this study. The competition performance metric (CPM) score of the nodule segmentation reached 0.83, the accuracy of nodule classification was 92%, the kappa coefficient with the ground truth was 0.68, and the overall diagnostic accuracy (calculated by the nodules) was 0.75. This paper summarises a more efficient AI-assisted diagnosis process of pulmonary nodules, and has better performance compared with the previous literature. In addition, this method will be validated in a future external clinical study.

Segmentation of the aorta in systolic phase from 4D flow MRI: multi-atlas vs. deep learning.

In the management of the aortic aneurysm, 4D flow magnetic resonance Imaging provides valuable information for the computation of new biomarkers using computational fluid dynamics (CFD). However, accurate segmentation of the aorta is required. Thus, our objective is to evaluate the performance of two automatic segmentation methods on the calculation of aortic wall pressure. Automatic segmentation of the aorta was performed with methods based on deep learning and multi-atlas using the systolic phase in the 4D flow MRI magnitude image of 36 patients. Using mesh morphing, isotopological meshes were generated, and CFD was performed to calculate the aortic wall pressure. Node-to-node comparisons of the pressure results were made to identify the most robust automatic method respect to the pressures obtained with a manually segmented model. Deep learning approach presented the best segmentation performance with a mean Dice similarity coefficient and a mean Hausdorff distance (HD) equal to 0.92+/- 0.02 and 21.02+/- 24.20mm, respectively. At the global level HD is affected by the performance in the abdominal aorta. Locally, this distance decreases to 9.41+/- 3.45 and 5.82+/- 6.23 for the ascending and descending thoracic aorta, respectively. Moreover, with respect to the pressures from the manual segmentations, the differences in the pressures computed from deep learning were lower than those computed from multi-atlas method. To reduce biases in the calculation of aortic wall pressure, accurate segmentation is needed, particularly in regions with high blood flow velocities. Thus, the deep learning segmen-tation method should be preferred.

Towards automatic cartilage quantification in clinical trials – Continuing from the 2019 IWOAI knee segmentation challenge

ObjectiveTo evaluate whether the deep learning (DL) segmentation methods from the six teams that participated in the IWOAI 2019 Knee Cartilage Segmentation Challenge are appropriate for quantifying cartilage loss in longitudinal clinical trials. DesignWe included 556 subjects from the Osteoarthritis Initiative study with manually read cartilage volume scores for the baseline and 1-year visits. The teams used their methods originally trained for the IWOAI 2019 challenge to segment the 1130 knee MRIs. These scans were anonymized and the teams were blinded to any subject or visit identifiers. Two teams also submitted updated methods. The resulting 9,040 segmentations are available online.The segmentations included tibial, femoral, and patellar compartments. In post-processing, we extracted medial and lateral tibial compartments and geometrically defined central medial and lateral femoral sub-compartments. The primary study outcome was the sensitivity to measure cartilage loss as defined by the standardized response mean (SRM). ResultsFor the tibial compartments, several of the DL segmentation methods had SRMs similar to the gold standard manual method. The highest DL SRM was for the lateral tibial compartment at 0.38 (the gold standard had 0.34). For the femoral compartments, the gold standard had higher SRMs than the automatic methods at 0.31/0.30 for medial/lateral compartments. ConclusionThe lower SRMs for the DL methods in the femoral compartments at 0.2 were possibly due to the simple sub-compartment extraction done during post-processing. The study demonstrated that state-of-the-art DL segmentation methods may be used in standardized longitudinal single-scanner clinical trials for well-defined cartilage compartments.

CFPNet-M: A light-weight encoder-decoder based network for multimodal biomedical image real-time segmentation

—Deep learning techniques are proving instrumental in identifying, classifying, and quantifying patterns in medical images. Segmentation is one of the important applications in medical image analysis. The U-Net has become the predominant deep-learning approach to medical image segmentation tasks. Existing U-Net based models have limitations in several respects, however, including: the requirement for millions of parameters in the U-Net, which consumes considerable computational resources and memory; the lack of global information; and incomplete segmentation in difficult cases. To remove some of those limitations, we built on our previous work and applied two modifications to improve the U-Net model: 1) we designed and added the dilated channel-wise CNN module and 2) we simplified the U-shape network. We then proposed a novel light-weight architecture, the Channel-wise Feature Pyramid Network for Medicine (CFPNet-M). To evaluate our method, we selected five datasets from different imaging modalities: thermography, electron microscopy, endoscopy, dermoscopy, and digital retinal images. We compared its performance with several models having a variety of complexities. We used the Tanimoto similarity instead of the Jaccard index for gray-level image comparisons. The CFPNet-M achieves segmentation results on all five medical datasets that are comparable to existing methods, yet require only 8.8 MB memory, and just 0.65 million parameters, which is about 2% of U-Net. Unlike other deep-learning segmentation methods, this new approach is suitable for real-time application: its inference speed can reach 80 frames per second when implemented on a single RTX 2070Ti GPU with an input image size of 256 × 192 pixels.

Semantic Segmentation and Inpainting of Dust with the S-Dust Dataset

In agriculture the performance of camera-based vision systems is strongly affected by environmental factors, such as rain, fog, and dust. This paper presents the first open dataset for training and evaluating image processing algorithms that remove swirling dust from images. It includes images from agricultural fields during tillage, which raises dust that is manually labeled in the images. However, dust is not a classical object that can be delineated based on edges. Moreover, the dust density is not uniformly distributed but varies and is concentrated in the area behind the implement. Therefore, different segmentation approaches were investigated. First, traditional and deep-learning methods for dust segmentation in images were compared. For the neural network, a Unet with a pretrained VGG encoder was chosen. The results show that the network cannot distinguish from the background in areas with low dust density. In contrast, the traditional segmentation method based on dark channel prior (DCP) outperforms it. Subsequently, various inpainting methods to remove dust were investigated. One observation was that dust removal from images cannot be solved with current inpainting methods. The dataset is available at *.*www.agriscapes-dataset.com

COVID-19 Lung CT image segmentation using localization and enhancement methods with U-Net

Segmentation of pneumonia lesions from Lung CT images has become vital for diagnosing the disease and evaluating the severity of the patients during the COVID-19 pandemic. Several AI-based systems have been proposed for this task. However, some low-contrast abnormal zones in CT images make the task challenging. The researchers investigated image preprocessing techniques to accomplish this problem and to enable more accurate segmentation by the AI-based systems. This study proposes a COVID-19 Lung-CT segmentation system based on histogram-based non-parametric region localization and enhancement (LE) methods prior to the U-Net architecture. The COVID-19-infected lung CT images were initially processed by the LE method, and the infected regions were detected and enhanced to provide more discriminative features to the deep learning segmentation methods. The U-Net is trained using the enhanced images to segment the regions affected by COVID-19. The proposed system achieved 97.75%, 0.85, and 0.74 accuracy, dice score, and Jaccard index, respectively. The comparison results suggested that the use of LE methods as a preprocessing step in CT Lung images significantly improved the feature extraction and segmentation abilities of the U-Net model by a 0.21 dice score. The results might lead to implementing the LE method in segmenting varied medical images.

XANet: An Efficient Remote Sensing Image Segmentation Model Using Element-Wise Attention Enhancement and Multi-Scale Attention Fusion

Massive and diverse remote sensing data provide opportunities for data-driven tasks in the real world, but also present challenges in terms of data processing and analysis, especially pixel-level image interpretation. However, the existing shallow-learning and deep-learning segmentation methods, bounded by their technical bottlenecks, cannot properly balance accuracy and efficiency, and are thus hardly scalable to the practice scenarios of remote sensing in a successful way. Instead of following the time-consuming deep stacks of local operations as most state-of-the-art segmentation networks, we propose a novel segmentation model with the encoder–decoder structure, dubbed XANet, which leverages the more computationally economical attention mechanism to boost performance. Two novel attention modules in XANet are proposed to strengthen the encoder and decoder, respectively, namely the Attention Recalibration Module (ARM) and Attention Fusion Module (AFM). Unlike current attention modules, which only focus on elevating the feature representation power, and regard the spatial and channel enhancement of a feature map as two independent steps, ARM gathers element-wise semantic descriptors coupling spatial and channel information to directly generate a 3D attention map for feature enhancement, and AFM innovatively utilizes the cross-attention mechanism for the sufficient spatial and channel fusion of multi-scale features. Extensive experiments were conducted on ISPRS and GID datasets to comprehensively analyze XANet and explore the effects of ARM and AFM. Furthermore, the results demonstrate that XANet surpasses other state-of-the-art segmentation methods in both model performance and efficiency, as ARM yields a superior improvement versus existing attention modules with a competitive computational overhead, and AFM achieves the complementary advantages of multi-level features under the sufficient consideration of efficiency.

Cell segmentation and representation with shape priors

Cell segmentation is a fundamental problem of computational biology, for which convolutional neural networks yield the best results nowadays. This field is expanding rapidly, and in the recent years, shape-constrained segmentation models emerged as strong competitors to traditional, pixel-based segmentation methods for instance segmentation. These methods predict the parameters of the underlying shape model, so choosing the right shape representation is critical for the success of the segmentation. In this study, we introduce two new representation-based deep learning segmentation methods after a quantitative comparison of the most important shape descriptors in the literature. Our networks are based on Fourier coefficients and statistical shape models, both of which have proven to be reliable tools for cell shape modelling. Our results indicate that the methods are competitive alternatives to the most widely used baseline deep learning algorithms, especially when the number of parameters for the underlying shape model are low or the cells to be segmented have irregular morphologies.

Fully automatic volume measurement of the adrenal gland on CT using deep learning to classify adrenal hyperplasia.

To develop a fully automated deep learning model for adrenal segmentation and to evaluate its performance in classifying adrenal hyperplasia. This retrospective study evaluated automated adrenal segmentation in 308 abdominal CT scans from 48 patients with adrenal hyperplasia and 260 patients with normal glands from 2010 to 2021 (mean age, 42 years; 156 women). The dataset was split into training, validation, and test sets at a ratio of 6:2:2. Contrast-enhanced CT images and manually drawn adrenal gland masks were used to develop a U-Net-based segmentation model. Predicted adrenal volumes were obtained by fivefold splitting of the dataset without overlapping the test set. Adrenal volumes and anthropometric parameters (height, weight, and sex) were utilized to develop an algorithm to classify adrenal hyperplasia, using multilayer perceptron, support vector classification, a random forest classifier, and a decision tree classifier. To measure the performance of the developed model, the dice coefficient and intraclass correlation coefficient (ICC) were used for segmentation, and area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity were used for classification. The model for segmenting adrenal glands achieved a Dice coefficient of 0.7009 for 308 cases and an ICC of 0.91 (95% CI, 0.90-0.93) for adrenal volume. The models for classifying hyperplasia had the following results: AUC, 0.98-0.99; accuracy, 0.948-0.961; sensitivity, 0.750-0.813; and specificity, 0.973-1.000. The proposed segmentation algorithm can accurately segment the adrenal glands on CT scans and may help clinicians identify possible cases of adrenal hyperplasia. • A deep learning segmentation method can accurately segment the adrenal gland, which is a small organ, on CT scans. • The machine learning algorithm to classify adrenal hyperplasia using adrenal volume and anthropometric parameters (height, weight, and sex) showed good performance. • The proposed segmentation algorithm may help clinicians identify possible cases of adrenal hyperplasia.

The multimodal MRI brain tumor segmentation based on AD-Net

Multimodal glioma images provide different features of tumor boundaries based on magnetic resonance imaging (MRI), where multimodal features are often challenging to extract for deep learning segmentation methods. Disturbance between features of different modes is an important factor restricting multimodal learning. To efficiently extract multimodal features, we propose an automatic weighted dilated convolutional network (AD-Net) to learn multimodal brain tumor features through channel feature separation learning. Specifically, the auto-weight dilated convolutional unit (AD unit) utilizes dual-scale convolutional feature maps to acquire channel separation features. We employ two learnable parameters to fuse dual-scale convolutional feature maps in encoding layers, and the two learnable parameters are automatically adjusted with the back propagation of the gradient. We adopt the Jensen–Shannon divergence to constrain the distribution of its feature map, which in turn regularizes the weights of the entire down-sampling. In addition, we use the training technique of deep supervision to achieve fast fitting. Our proposed method got dice scores of 0.90, 0.80, and 0.76 for the whole tumor (WT), the tumor core (TC), and the enhancing tumor (ET) on the BraTS20 dataset. The experimental results showed good performance with the AD-Net network.

A preliminary deep learning study on automatic segmentation of contrast-enhanced bolus in videofluorography of swallowing

Although videofluorography (VFG) is an effective tool for evaluating swallowing functions, its accurate evaluation requires considerable time and effort. This study aimed to create a deep learning model for automated bolus segmentation on VFG images of patients with healthy swallowing and dysphagia using the artificial intelligence deep learning segmentation method, and to assess the performance of the method. VFG images of 72 swallowing of 12 patients were continuously converted into 15 static images per second. In total, 3910 images were arbitrarily assigned to the training, validation, test 1, and test 2 datasets. In the training and validation datasets, images of colored bolus areas were prepared, along with original images. Using a U-Net neural network, a trained model was created after 500 epochs of training. The test datasets were applied to the trained model, and the performances of automatic segmentation (Jaccard index, Sørensen–Dice coefficient, and sensitivity) were calculated. All performance values for the segmentation of the test 1 and 2 datasets were high, exceeding 0.9. Using an artificial intelligence deep learning segmentation method, we automatically segmented the bolus areas on VFG images; our method exhibited high performance. This model also allowed assessment of aspiration and laryngeal invasion.

A Deep Learning Segmentation Pipeline for Cardiac T1 Mapping Using MRI Relaxation-based Synthetic Contrast Augmentation.

To design and evaluate an automated deep learning method for segmentation and analysis of cardiac MRI T1 maps with use of synthetic T1-weighted images for MRI relaxation-based contrast augmentation. This retrospective study included MRI scans acquired between 2016 and 2019 from 100 patients (mean age ± SD, 55 years ± 13; 72 men) across various clinical abnormalities with use of a modified Look-Locker inversion recovery, or MOLLI, sequence to quantify native T1 (T1native), postcontrast T1 (T1post), and extracellular volume (ECV). Data were divided into training (n = 60) and internal (n = 40) test subsets. "Synthetic" T1-weighted images were generated from the T1 exponential inversion-recovery signal model at a range of optimal inversion times, yielding high blood-myocardium contrast, and were used for contrast-based image augmentation during training and testing of a convolutional neural network for myocardial segmentation. Automated segmentation, T1, and ECV were compared with experts with use of Dice similarity coefficients (DSCs), correlation coefficients, and Bland-Altman analysis. An external test dataset (n = 147) was used to assess model generalization. Internal testing showed high myocardial DSC relative to experts (0.81 ± 0.08), which was similar to interobserver DSC (0.81 ± 0.08). Automated segmental measurements strongly correlated with experts (T1native, R = 0.87; T1post, R = 0.91; ECV, R = 0.92), which were similar to interobserver correlation (T1native, R = 0.86; T1post, R = 0.94; ECV, R = 0.95). External testing showed strong DSC (0.80 ± 0.09) and T1native correlation (R = 0.88) between automatic and expert analysis. This deep learning method leveraging synthetic contrast augmentation may provide accurate automated T1 and ECV analysis for cardiac MRI data acquired across different abnormalities, centers, scanners, and T1 sequences.Keywords: MRI, Cardiac, Tissue Characterization, Segmentation, Convolutional Neural Network, Deep Learning Algorithms, Machine Learning Algorithms, Supervised Learning Supplemental material is available for this article. © RSNA, 2022.

Influence of contrast and texture based image modifications on the performance and attention shift of U-Net models for brain tissue segmentation.

Contrast and texture modifications applied during training or test-time have recently shown promising results to enhance the generalization performance of deep learning segmentation methods in medical image analysis. However, a deeper understanding of this phenomenon has not been investigated. In this study, we investigated this phenomenon using a controlled experimental setting, using datasets from the Human Connectome Project and a large set of simulated MR protocols, in order to mitigate data confounders and investigate possible explanations as to why model performance changes when applying different levels of contrast and texture-based modifications. Our experiments confirm previous findings regarding the improved performance of models subjected to contrast and texture modifications employed during training and/or testing time, but further show the interplay when these operations are combined, as well as the regimes of model improvement/worsening across scanning parameters. Furthermore, our findings demonstrate a spatial attention shift phenomenon of trained models, occurring for different levels of model performance, and varying in relation to the type of applied image modification.

Experimental large-scale jet flames’ geometrical features extraction for risk management using infrared images and deep learning segmentation methods

Jet fires are relatively small and have the least severe effects among the diverse fire accidents that can occur in industrial plants; however, they are usually involved in a process known as the domino effect, that leads to more severe events, such as explosions or the initiation of another fire, making the analysis of such fires an important part of risk analysis. This research work explores the application of deep learning models in an alternative approach that uses the semantic segmentation of jet fires flames to extract the flame’s main geometrical attributes, relevant for fire risk assessments. A comparison is made between traditional image processing methods and some state-of-the-art deep learning models. It is found that the best approach is a deep learning architecture known as UNet, along with its two improvements, Attention UNet and UNet++. The models are then used to segment a group of vertical jet flames of varying pipe outlet diameters to extract their main geometrical characteristics. Attention UNet obtained the best general performance in the approximation of both height and area of the flames, while also showing a statistically significant difference between it and UNet++. UNet obtained the best overall performance for the approximation of the lift-off distances; however, there is not enough data to prove a statistically significant difference between Attention UNet and UNet++. The only instance where UNet++ outperformed the other models, was while obtaining the lift-off distances of the jet flames with 0.01275 m pipe outlet diameter. In general, the explored models show good agreement between the experimental and predicted values for relatively large turbulent propane jet flames, released in sonic and subsonic regimes; thus, making these radiation zones segmentation models, a suitable approach for different jet flame risk management scenarios.

Two fully automated data-driven 3D whole-breast segmentation strategies in MRI for MR-based breast density using image registration and U-Net with a focus on reproducibility

Presence of higher breast density (BD) and persistence over time are risk factors for breast cancer. A quantitatively accurate and highly reproducible BD measure that relies on precise and reproducible whole-breast segmentation is desirable. In this study, we aimed to develop a highly reproducible and accurate whole-breast segmentation algorithm for the generation of reproducible BD measures. Three datasets of volunteers from two clinical trials were included. Breast MR images were acquired on 3 T Siemens Biograph mMR, Prisma, and Skyra using 3D Cartesian six-echo GRE sequences with a fat-water separation technique. Two whole-breast segmentation strategies, utilizing image registration and 3D U-Net, were developed. Manual segmentation was performed. A task-based analysis was performed: a previously developed MR-based BD measure, MagDensity, was calculated and assessed using automated and manual segmentation. The mean squared error (MSE) and intraclass correlation coefficient (ICC) between MagDensity were evaluated using the manual segmentation as a reference. The test-retest reproducibility of MagDensity derived from different breast segmentation methods was assessed using the difference between the test and retest measures (Δ2-1), MSE, and ICC. The results showed that MagDensity derived by the registration and deep learning segmentation methods exhibited high concordance with manual segmentation, with ICCs of 0.986 (95%CI: 0.974-0.993) and 0.983 (95%CI: 0.961-0.992), respectively. For test-retest analysis, MagDensity derived using the registration algorithm achieved the smallest MSE of 0.370 and highest ICC of 0.993 (95%CI: 0.982-0.997) when compared to other segmentation methods. In conclusion, the proposed registration and deep learning whole-breast segmentation methods are accurate and reliable for estimating BD. Both methods outperformed a previously developed algorithm and manual segmentation in the test-retest assessment, with the registration exhibiting superior performance for highly reproducible BD measurements.

Deep active learning for suggestive segmentation of biomedical image stacks via optimisation of Dice scores and traced boundary length.

Manual segmentation of stacks of 2D biomedical images (e.g., histology) is a time-consuming task which can be sped up with semi-automated techniques. In this article, we present a suggestive deep active learning framework that seeks to minimise the annotation effort required to achieve a certain level of accuracy when labelling such a stack. The framework suggests, at every iteration, a specific region of interest (ROI) in one of the images for manual delineation. Using a deep segmentation neural network and a mixed cross-entropy loss function, we propose a principled strategy to estimate class probabilities for the whole stack, conditioned on heterogeneous partial segmentations of the 2D images, as well as on weak supervision in the form of image indices that bound each ROI. Using the estimated probabilities, we propose a novel active learning criterion based on predictions for the estimated segmentation performance and delineation effort, measured with average Dice scores and total delineated boundary length, respectively, rather than common surrogates such as entropy. The query strategy suggests the ROI that is expected to maximise the ratio between performance and effort, while considering the adjacency of structures that may have already been labelled - which decrease the length of the boundary to trace. We provide quantitative results on synthetically deformed MRI scans and real histological data, showing that our framework can reduce labelling effort by up to 60-70% without compromising accuracy.