Test-time Augmentation Research Articles

SkiNet: A deep learning framework for skin lesion diagnosis with uncertainty estimation and explainability

Skin cancer is considered to be the most common human malignancy. Around 5 million new cases of skin cancer are recorded in the United States annually. Early identification and evaluation of skin lesions are of great clinical significance, but the disproportionate dermatologist-patient ratio poses a significant problem in most developing nations. Therefore a novel deep architecture, named as SkiNet, is proposed to provide faster screening solution and assistance to newly trained physicians in the process of clinical diagnosis of skin cancer. The main motive behind SkiNet's design and development is to provide a white box solution, addressing a critical problem of trust and interpretability which is crucial for the wider adoption of Computer-aided diagnosis systems by medical practitioners. The proposed SkiNet is a two-stage pipeline wherein the lesion segmentation is followed by the lesion classification. Monte Carlo dropout and test time augmentation techniques have been employed in the proposed method to estimate epistemic and aleatoric uncertainty. A novel segmentation model named Bayesian MultiResUNet is used to estimate the uncertainty on the predicted segmentation map. Saliency-based methods like XRAI, Grad-CAM and Guided Backprop are explored to provide post-hoc explanations of the deep learning models. The ISIC-2018 dataset is used to perform the experimentation and ablation studies. The results establish the robustness of the proposed model on the traditional benchmarks while addressing the black-box nature of such models to alleviate the skepticism of medical practitioners by incorporating transparency and confidence to the model's prediction.

Glacier extraction based on high-spatial-resolution remote-sensing images using a deep-learning approach with attention mechanism

Abstract. The accurate and rapid extraction of glacier boundaries plays an important role in the study of glacier inventory, glacier change and glacier movement. With the successive launches of high-resolution remote-sensing satellites and the increasing abundance of available remote-sensing data, great opportunities and challenges now exist. In this study, we improved the DeepLab V3+ as Attention DeepLab V3+ and designed a complete solution based on the improved network to automatically extract glacier outlines from Gaofen-6 panchromatic and multispectral (PMS) images with a spatial resolution of 2 m. In the solution, test-time augmentation (TTA) was adopted to increase model robustness, and the convolutional block attention module (CBAM) was added into the atrous spatial pyramid pooling (ASPP) structure in DeepLab V3+ to enhance the weight of the target pixels and reduce the impact of superfluous features. The results show that the improved model effectively increases the robustness of the model, enhances the weight of target image elements and reduces the influence of non-target elements. Compared with deep-learning models, such as full convolutional network (FCN), U-Net and DeepLab V3+, the improved model performs better in the test dataset. Moreover, our method achieves superior performance for glacier boundary extraction in parts of the Tanggula Mountains, the Kunlun Mountains and the Qilian Mountains based on Gaofen-6 PMS images. It could distinguish glaciers from terminal moraine lakes, thin snow and clouds, thus demonstrating excellent performance and great potential for rapid and precise extraction of glacier boundaries.

Recognition of terminal buds of densely-planted Chinese fir seedlings using improved YOLOv5 by integrating attention mechanism.

Accurate and timely information on the number of densely-planted Chinese fir seedlings is essential for their scientific cultivation and intelligent management. However, in the later stage of cultivation, the overlapping of lateral branches among individuals is too severe to identify the entire individual in the UAV image. At the same time, in the high-density planting nursery, the terminal bud of each seedling has a distinctive characteristic of growing upward, which can be used as an identification feature. Still, due to the small size and dense distribution of the terminal buds, the existing recognition algorithm will have a significant error. Therefore, in this study, we proposed a model based on the improved network structure of the latest YOLOv5 algorithm for identifying the terminal bud of Chinese fir seedlings. Firstly, the micro-scale prediction head was added to the original prediction head to enhance the model’s ability to perceive small-sized terminal buds. Secondly, a multi-attention mechanism module composed of Convolutional Block Attention Module (CBAM) and Efficient Channel Attention (ECA) was integrated into the neck of the network to enhance further the model’s ability to focus on key target objects in complex backgrounds. Finally, the methods including data augmentation, Test Time Augmentation (TTA) and Weighted Boxes Fusion (WBF) were used to improve the robustness and generalization of the model for the identification of terminal buds in different growth states. The results showed that, compared with the standard version of YOLOv5, the recognition accuracy of the improved YOLOv5 was significantly increased, with a precision of 95.55%, a recall of 95.84%, an F1-Score of 96.54%, and an mAP of 94.63%. Under the same experimental conditions, compared with other current mainstream algorithms (YOLOv3, Faster R-CNN, and PP-YOLO), the average precision and F1-Score of the improved YOLOv5 also increased by 9.51-28.19 percentage points and 15.92-32.94 percentage points, respectively. Overall, The improved YOLOv5 algorithm integrated with the attention network can accurately identify the terminal buds of densely-planted Chinese fir seedlings in UAV images and provide technical support for large-scale and automated counting and precision cultivation of Chinese fir seedlings.

Research and Development of Deep Learning Algorithms for the Classification of Pneumonia Type and Detection of Ground-Glass Loci on Radiological Images

Pneumonia is a highly dangerous state that poses serious risks to the health of a patient. In contrast to common pneumonia, lung disease COVID-19 causes a large number of lethal outcomes. The pneumonia of a viral etiology caused by the RNA virus SARS-CoV-2 is visually hardly distinguishable from the bacterial pneumonia or inflammation caused by other viral infections. Now, COVID-19 can be diagnosed using PCR tests or X-rays of the thoracic cage. However, the results of a molecular study take a long time to prepare. In contrast, the radiological images of the thoracic cage can be obtained immediately after the radiological study. Although there exist guiding principles which help radiologists to differentiate COVID-19 from other types of infections, their assessments differ. In addition, doctors who are not radiologists can be assisted in better locating the disease, for instance, by a bounding box. Development of precise computer methods based on artificial intelligence can help medical workers in quickly determining the type of pneumonia and detecting the loci of inflammation. In this study a package of methods is developed to determine the type of pneumonia and detect the ground-glass loci using the appropriate architectures of neural networks, loss functions, augmentations at the training data generation stage, test time augmentation, and computer vision model ensembles. This task is successfully solved in the SIIM-FISABIO-RSNA COVID-19 Detection competition [17] and the proposed algorithm is in the top 10% of the best solutions.

IterNet++: An improved model for retinal image segmentation by curvelet enhancing, guided filtering, offline hard‐sample mining, and test‐time augmenting

In clinical medicine, the segmentation of blood vessels in retinal images is essential for subsequent analysis in clinical diagnosis. However, retinal images are often noisy and their vascular structure is relatively tiny, which poses significant challenges for vessel segmentation. To improve the performance of vessel segmentation, an improved model IterNet++ based on the architecture of IterNet is proposed. First, curvelet signal analysis is applied to enhance retinal images. Second, residual convolution (ResConv) blocks and guided filters are introduced to utilise the encoder features of previous iterations in the model to reduce overfitting. Third, offline hard-sample mining is used to improve segmentation performance by utilising training samples with low segmentation accuracy as many possible on a few-sample training set. In addition, a test-time augmentation method is applied to testing samples in test dataset during inference. Extensive experiments show that this model achieves Dice scores of 0.8313, 0.8277, and 0.8372 on DRIVE, CHASE-DB1, and STARE datasets, respectively, demonstrating the best performance compared with IterNet and other baseline models.

Ensemble learning and test-time augmentation for the segmentation of mineralized cartilage versus bone in high-resolution microCT images

High-resolution non-destructive 3D microCT imaging allows the visualization and structural characterization of mineralized cartilage and bone. Deriving statistically relevant quantitative structural information about these tissues, however, requires automated segmentation procedures, mainly because manual contouring is user-biased and time-consuming. Despite the increased spatial resolution in microCT 3D volumes, automatic segmentation of mineralized cartilage versus bone remains non-trivial since they have similar grayscale values. Our work investigates how reliable 2D segmentation masks can be predicted automatically based on a (set of) convolutional neural network(s) trained with a limited number of manually annotated samples. To do that, we compared different strategies to select the 2D samples to annotate and considered ensemble learning and test-time augmentation (TTA) to mitigate the limited accuracy and robustness resulting from the small number of annotated training samples. We show that, for a fixed amount of annotated image samples, 2D microCT slices to annotate should preferably be selected in distinct 3D volumes, at regular intervals, rather than being grouped in adjacent slices of a same 3D volume. Two main lessons are drawn regarding the use of ensembles or TTA instead of a single model. First, ensemble learning is shown to improve segmentation accuracy and to reduce the mean and standard deviation of the absolute errors in cartilage characteristics obtained with different initializations of the neural network training process. In contrast, TTA appears to be unable to improve the model’s robustness to unlucky initializations. Second, both TTA and ensembling improved the model’s confidence in its predictions and segmentation failure detection.

Cascade multiscale residual attention CNNs with adaptive ROI for automatic brain tumor segmentation

A brain tumor is one of the fatal cancer types which causes abnormal growth of brain cells. Earlier diagnosis of a brain tumor can play a vital role in its treatment; however, manual segmentation of the brain tumors from MRI images is a laborious and time-consuming task. Therefore, several automatic segmentation techniques have been proposed, but high inter-and intra-tumor variations in shape and texture make accurate segmentation of enhanced and core tumor regions challenging. The demand for a highly accurate and robust segmentation method persists; therefore, in this paper, we propose a novel fully automatic technique for brain tumor regions segmentation by using multiscale residual attention-UNet (MRA-UNet). MRA-UNet uses three consecutive slices as input to preserve the sequential information and employs multiscale learning in a cascade fashion, enabling it to exploit the adaptive region of interest scheme to segment enhanced and core tumor regions accurately. The study also investigates the application of postprocessing techniques (i.e., conditional random field and test time augmentation), which further helps to improve the overall performance. The proposed method has been rigorously evaluated on the most extensive publicly available datasets, i.e., BraTS2017, BraTS2019, and BraTS2020. Our approach achieved state-of-the-art results on the BraTS2020 dataset with the average dice score of 90.18%, 87.22%, and 86.74% for the whole tumor, tumor core, and enhanced tumor regions, respectively. A significant improvement has been demonstrated for the core and enhancing tumor region segmentation showing the proposed method’s evident effectiveness.

P-171 Sensitivity analysis of an embryo grading artificial intelligence model to different focal planes

Abstract Study question What is the sensitivity of an embryo-grading artificial intelligence (AI) model to different focal planes and how do we obtain consistent scores across focal planes? Summary answer Test-time augmentation and ensemble modeling reduce sensitivity of the AI model to different focal planes while maintaining performance. What is known already When prioritizing embryos for transfer, embryologists assess the 3D morphological features under a microscope, by zooming up and down, and assign a score that reflects the embryo quality. In comparison, some AI-based embryo grading models typically take one 2D focal plane of an embryo and output a score based on that focal plane. AI models such as convolutional neural networks (CNNs) are known to be sensitive to perturbations in its input. In order to reduce sensitivity and generalization error and thus improve predictive performance, techniques such as ensemble learning and test-time augmentation can be used. Study design, size, duration Historical, de-identified images of blastocyst-stage embryos were collected from 11 IVF clinics in the United States for cycles between 2015-2020. 5,100 blastocysts were matched to pregnancy outcomes as determined by fetal heartbeat. 2,900 blastocysts were matched to aneuploid PGT-A results and added to the negative training group to reduce selection bias. Data was split to 70% for training and 30% for testing. A set of 10 embryos were used for focal plane sensitivity. Participants/materials, setting, methods A single model (ResNet18), a three-model (ResNet18), and a six-model (ResNet18 and EfficientNet-b1) ensemble with and without test-time augmentation were trained to rank embryos according to their likelihood of reaching clinical pregnancy. Test-time augmentation involved taking the average scores from 4 flipped and rotated copies of the original input image. Manual grades were mapped to numeric scores for comparison. The AUC was used to evaluate the ability of the models to rank embryos. Main results and the role of chance Focal plane sensitivity was calculated as the range, or difference between the maximum and minimum score, for an embryo at different focal planes. Between 12 and 100 focal plane images were available for each of the 10 embryos. On average, the focal plane range was 0.26 for the single model, 0.22 for the single model with test-time augmentation, 0.14 for a 3-model ensemble with test-time augmentation, and 0.11 for a 6-model ensemble with test-time augmentation. Test-time augmentation on the single model reduced the range by 17%; whereas ensemble modeling with test-time augmentation reduced the range by 46% for the 3-model ensemble and 60% for the 6-model ensemble. Reduction in range did not compromise performance. The AUC for the test set for all embryos was 0.73 for the single model, 0.74 for the single model with test-time augmentation, 0.75 for the three-model ensemble with test-time augmentation and 0.74 for the six-model ensemble with test-time augmentation. All models outperformed manual grading, which was estimated to have an AUC of 0.67 for all embryos. Limitations, reasons for caution Our analysis on focal plane sensitivity was limited to a small sample size of 10 embryos, so more samples will be needed to confirm our findings. Wider implications of the findings Test-time augmentation and ensemble techniques can be used to reduce sensitivity while maintaining model performance. By reducing sensitivity to different focal planes, an AI model can produce one reliable score for a single embryo as is done currently in practice with manual grading. Trial registration number not applicable

Learning test-time augmentation for content-based image retrieval

Off-the-shelf convolutional neural network features achieve outstanding results in many image retrieval tasks. However, their invariance to target data is pre-defined by the network architecture and training data. Existing image retrieval approaches require fine-tuning or modification of pre-trained networks to adapt to variations unique to the target data. In contrast, our method enhances the invariance of off-the-shelf features by aggregating features extracted from images augmented at test-time, with augmentations guided by a policy learned through reinforcement learning. The learned policy assigns different magnitudes and weights to the selected transformations, which are selected from a list of image transformations. Policies are evaluated using a metric learning protocol to learn the optimal policy. The model converges quickly and the cost of each policy iteration is minimal as we propose an off-line caching technique to greatly reduce the computational cost of extracting features from augmented images. Experimental results on large trademark retrieval (METU trademark dataset) and landmark retrieval (ROxford5k and RParis6k scene datasets) tasks show that the learned ensemble of transformations is highly effective for improving performance, and is practical, and transferable.

MP-Net: Deep learning-based segmentation for fluorescence microscopy images of microplastics isolated from clams.

Environmental monitoring of microplastics (MP) contamination has become an area of great research interest, given potential hazards associated with human ingestion of MP. In this context, determination of MP concentration is essential. However, cheap, rapid, and accurate quantification of MP remains a challenge to this date. This study proposes a deep learning-based image segmentation method that properly distinguishes fluorescent MP from other elements in a given microscopy image. A total of nine different deep learning models, six of which are based on U-Net, were investigated. These models were trained using at least 20,000 patches sampled from 99 fluorescence microscopy images of MP and their corresponding binary masks. MP-Net, which is derived from U-Net, was found to be the best performing model, exhibiting the highest mean F1-score (0.736) and mean IoU value (0.617). Test-time augmentation (using brightness, contrast, and HSV) was applied to MP-Net for robust learning. However, compared to the results obtained without augmentation, no clear improvement in predictive performance could be observed. Recovery assessment for both spiked and real images showed that, compared to already existing tools for MP quantification, the MP quantities predicted by MP-Net are those closest to the ground truth. This observation suggests that MP-Net allows creating masks that more accurately reflect the quantitative presence of fluorescent MP in microscopy images. Finally, MAP (Microplastics Annotation Package) is introduced, an integrated software environment for automated MP quantification, offering support for MP-Net, already existing MP analysis tools like MP-VAT, manual annotation, and model fine-tuning.

CAU-Net: A Deep Learning Method for Deep Gray Matter Nuclei Segmentation

The abnormal iron deposition of the deep gray matter nuclei is related to many neurological diseases. With the quantitative susceptibility mapping (QSM) technique, it is possible to quantitatively measure the brain iron content in vivo. To assess the magnetic susceptibility of the deep gray matter nuclei in the QSM, it is mandatory to segment the nuclei of interest first, and many automatic methods have been proposed in the literature. This study proposed a contrast attention U-Net for nuclei segmentation and evaluated its performance on two datasets acquired using different sequences with different parameters from different MRI devices. Experimental results revealed that our proposed method was superior on both datasets over other commonly adopted network structures. The impacts of training and inference strategies were also discussed, which showed that adopting test time augmentation during the inference stage can impose an obvious improvement. At the training stage, our results indicated that sufficient data augmentation, deep supervision, and nonuniform patch sampling contributed significantly to improving the segmentation accuracy, which indicated that appropriate choices of training and inference strategies were at least as important as designing more advanced network structures.

Deep learning-based accurate grapevine inflorescence and flower quantification in unstructured vineyard images acquired using a mobile sensing platform

Early grapevine yield forecasting at satisfactory accuracy is among the major trends in precision viticulture research. Conventionally, yield is estimated manually through extrapolation right before harvest, which is mostly inaccurate, requires considerable labor as well as resources, and is often destructive. The number of flowers per vine is one of the main determinants of grapevine yield and can be used as an early indicator for viticulture yield forecasting. In the present study, a non-invasive, automated image analysis framework was proposed for quantifying flowers in unstructured RGB images of grapevine canopies. Images were automatically acquired under field conditions at night, using a mobile sensing platform equipped with artificial illumination. Due to the small shape and dense distribution of individual flowers (a few hundred per inflorescence) and the similar hue of all plant organs in the fore- and background, efficient flower quantification in images is challenging. To overcome this, we adopted a two-step segmentation approach in our algorithm. First, image regions containing inflorescences were recognized and extracted by segmenting each instance of inflorescences in the image using a mask region-based convolutional neural network (Mask R-CNN), followed by test-time augmentation post-processing to achieve high accuracy. Finally, individual flowers in the extracted inflorescences were detected and quantified using another Mask R-CNN model preceded by a contrast enhancement pre-processing operation. For efficient segmentation and quantification of individual flowers, a high-resolution full image was split down into smaller patches and processed in multiple iterations during inference. The algorithm yielded significant performances, with F1 score values of 0.943 and 0.903 for inflorescence segmentation and single flower detection tasks, respectively, against a test set of 75 images from three different cultivars. A determination coefficient (R2) of 0.98 and a normalized root mean square error of 12.24% were obtained in the test set between the automatic flower number quantification and manual counts. In conclusion, the proposed algorithm constitutes a promising approach for automatically predicting yield potential in the early stages of grapevine development, and it can be used for objective monitoring and optimal management of commercial vineyards.

One-Shot Learning-Based Animal Video Segmentation

Deep learning-based video segmentation methods can offer a good performance after being trained on the large-scale pixel labeled datasets. However, a pixel-wise manual labeling of animal images is challenging and time consuming due to irregular contours and motion blur. To achieve desirable tradeoffs between the accuracy and speed, a novel one-shot learning-based approach is proposed in this article to segment animal video with only one labeled frame. The proposed approach consists of the following three main modules: guidance frame selection utilizes “BubbleNet” to choose one frame for manual labeling, which can leverage the fine-tuning effects of the only labeled frame; Xception-based fully convolutional network localizes dense prediction using depthwise separable convolutions based on one single labeled frame; and postprocessing is used to remove outliers and sharpen object contours, which consists of two submodules—test time augmentation and conditional random field. Extensive experiments have been conducted on the DAVIS 2016 animal dataset. Our proposed video segmentation approach achieved mean intersection-over-union score of 89.5% on the DAVIS 2016 animal dataset with less run time, and outperformed the state-of-art methods (OSVOS and OSMN). The proposed one-shot learning-based approach achieves real-time and automatic segmentation of animals with only one labeled video frame. This can be potentially used further as a baseline for intelligent perception-based monitoring of animals and other domain-specific applications. <xref ref-type="fn" rid="fn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">¹</xref> <fn id="fn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><label>¹</label> The source code, datasets, and pre-trained weights for this work are publicly [Online]. Available: <uri>https://github.com/tengfeixue-victor/One-Shot-Animal-Video-Segmentation</uri>. </fn>

Generalisation effects of predictive uncertainty estimation in deep learning for digital pathology

Deep learning (DL) has shown great potential in digital pathology applications. The robustness of a diagnostic DL-based solution is essential for safe clinical deployment. In this work we evaluate if adding uncertainty estimates for DL predictions in digital pathology could result in increased value for the clinical applications, by boosting the general predictive performance or by detecting mispredictions. We compare the effectiveness of model-integrated methods (MC dropout and Deep ensembles) with a model-agnostic approach (Test time augmentation, TTA). Moreover, four uncertainty metrics are compared. Our experiments focus on two domain shift scenarios: a shift to a different medical center and to an underrepresented subtype of cancer. Our results show that uncertainty estimates increase reliability by reducing a model’s sensitivity to classification threshold selection as well as by detecting between 70 and 90% of the mispredictions done by the model. Overall, the deep ensembles method achieved the best performance closely followed by TTA.

Informative and Reliable Tract Segmentation for Preoperative Planning.

Identifying white matter (WM) tracts to locate eloquent areas for preoperative surgical planning is a challenging task. Manual WM tract annotations are often used but they are time-consuming, suffer from inter- and intra-rater variability, and noise intrinsic to diffusion MRI may make manual interpretation difficult. As a result, in clinical practice direct electrical stimulation is necessary to precisely locate WM tracts during surgery. A measure of WM tract segmentation unreliability could be important to guide surgical planning and operations. In this study, we use deep learning to perform reliable tract segmentation in combination with uncertainty quantification to measure segmentation unreliability. We use a 3D U-Net to segment white matter tracts. We then estimate model and data uncertainty using test time dropout and test time augmentation, respectively. We use a volume-based calibration approach to compute representative predicted probabilities from the estimated uncertainties. In our findings, we obtain a Dice of ≈0.82 which is comparable to the state-of-the-art for multi-label segmentation and Hausdorff distance <10mm. We demonstrate a high positive correlation between volume variance and segmentation errors, which indicates a good measure of reliability for tract segmentation ad uncertainty estimation. Finally, we show that calibrated predicted volumes are more likely to encompass the ground truth segmentation volume than uncalibrated predicted volumes. This study is a step toward more informed and reliable WM tract segmentation for clinical decision-making.

Polar transform network for prostate ultrasound segmentation with uncertainty estimation.

Automatic and accurate prostate ultrasound segmentation is a long-standing and challenging problem due to the severe noise and ambiguous/missing prostate boundaries. In this work, we propose a novel polar transform network (PTN) to handle this problem from a fundamentally new perspective, where the prostate is represented and segmented in the polar coordinate space rather than the original image grid space. This new representation gives a prostate volume, especially the most challenging apex and base sub-areas, much denser samples than the background and thus facilitate the learning of discriminative features for accurate prostate segmentation. Moreover, in the polar representation, the prostate surface can be efficiently parameterized using a 2D surface radius map with respect to a centroid coordinate, which allows the proposed PTN to obtain superior accuracy compared with its counterparts using convolutional neural networks while having significantly fewer (18%∼41%) trainable parameters. We also equip our PTN with a novel strategy of centroid perturbed test-time augmentation (CPTTA), which is designed to further improve the segmentation accuracy and quantitatively assess the model uncertainty at the same time. The uncertainty estimation function provides valuable feedback to clinicians when manual modifications or approvals are required for the segmentation, substantially improving the clinical significance of our work. We conduct a three-fold cross validation on a clinical dataset consisting of 315 transrectal ultrasound (TRUS) images to comprehensively evaluate the performance of the proposed method. The experimental results show that our proposed PTN with CPTTA outperforms the state-of-the-art methods with statistical significance on most of the metrics while exhibiting a much smaller model size. Source code of the proposed PTN is released at https://github.com/DIAL-RPI/PTN.

Endoscopy Artefact Detection by Deep Transfer Learning of Baseline Models.

To visualise the tumours inside the body on a screen, a long and thin tube is inserted with a light source and a camera at the tip to obtain video frames inside organs in endoscopy. However, multiple artefacts exist in these video frames that cause difficulty during the diagnosis of cancers. In this research, deep learning was applied to detect eight kinds of artefacts: specularity, bubbles, saturation, contrast, blood, instrument, blur, and imaging artefacts. Based on transfer learning with pre-trained parameters and fine-tuning, two state-of-the-art methods were applied for detection: faster region-based convolutional neural networks (Faster R-CNN) and EfficientDet. Experiments were implemented on the grand challenge dataset, Endoscopy Artefact Detection and Segmentation (EAD2020). To validate our approach in this study, we used phase I of 2,200 frames and phase II of 331 frames in the original training dataset with ground-truth annotations as training and testing dataset, respectively. Among the tested methods, EfficientDet-D2 achieves a score of 0.2008 (mAPd[Formula: see text]0.6+mIoUd[Formula: see text]0.4) on the dataset that is better than three other baselines: Faster-RCNN, YOLOv3, and RetinaNet, and competitive to the best non-baseline result scored 0.25123 on the leaderboard although our testing was on phase II of 331 frames instead of the original 200 testing frames. Without extra improvement techniques beyond basic neural networks such as test-time augmentation, we showed that a simple baseline could achieve state-of-the-art performance in detecting artefacts in endoscopy. In conclusion, we proposed the combination of EfficientDet-D2 with suitable data augmentation and pre-trained parameters during fine-tuning training to detect the artefacts in endoscopy.

Can uncertainty estimation predict segmentation performance in ultrasound bone imaging?

Segmenting bone surfaces in ultrasound (US) is a fundamental step in US-based computer-assisted orthopaedic surgeries. Neural network-based segmentation techniques are a natural choice for this, given promising results in related tasks. However, to gain widespread use, we must be able to know how much to trust segmentation networks during clinical deployment when ground-truth data is unavailable. We investigated alternative ways to measure the uncertainty of trained networks by implementing a baseline U-Net trained on a large dataset, together with three uncertainty estimation modifications: Monte Carlo dropout, test time augmentation, and ensemble learning. We measured the segmentation performance, calibration quality, and the ability to predict segmentation performance on test data. We further investigated the effect of data quality on these measures. Overall, we found that ensemble learning with binary cross-entropy (BCE) loss achieved the best segmentation performance (mean Dice: 0.75-0.78 and RMS distance: 0.62-0.86mm) and the lowest calibration errors (mean: 0.22-0.28%). In contrast to previous studies of area or volumetric segmentation, we found that the resulting uncertainty measures are not reliable proxies for surface segmentation performance. Our experiments indicate that a significant performance and confidence calibration boost can be achieved with ensemble learning and BCE loss, as tested on 13,687 US images containing various anatomies and imaging parameters. However, these techniques do not allow us to reliably predict future segmentation performance. The results of this study can be used to improve the calibration and performance of US segmentation networks.

Melanoma segmentation using deep learning with test-time augmentations and conditional random fields

In a computer-aided diagnostic (CAD) system for skin lesion segmentation, variations in shape and size of the skin lesion makes the segmentation task more challenging. Lesion segmentation is an initial step in CAD schemes as it leads to low error rates in quantification of the structure, boundary, and scale of the skin lesion. Subjective clinical assessment of the skin lesion segmentation results provided by current state-of-the-art deep learning segmentation techniques does not offer the required results as per the inter-observer agreement of expert dermatologists. This study proposes a novel deep learning-based, fully automated approach to skin lesion segmentation, including sophisticated pre and postprocessing approaches. We use three deep learning models, including UNet, deep residual U-Net (ResUNet), and improved ResUNet (ResUNet++). The preprocessing phase combines morphological filters with an inpainting algorithm to eliminate unnecessary hair structures from the dermoscopic images. Finally, we used test time augmentation (TTA) and conditional random field (CRF) in the postprocessing stage to improve segmentation accuracy. The proposed method was trained and evaluated on ISIC-2016 and ISIC-2017 skin lesion datasets. It achieved an average Jaccard Index of 85.96% and 80.05% for ISIC-2016 and ISIC-2017 datasets, when trained individually. When trained on combined dataset (ISIC-2016 and ISIC-2017), the proposed method achieved an average Jaccard Index of 80.73% and 90.02% on ISIC-2017 and ISIC-2016 testing datasets. The proposed methodological framework can be used to design a fully automated computer-aided skin lesion diagnostic system due to its high scalability and robustness.

Breast cancer diagnosis based on detecting lymph node metastases using deep learning

Introduction: Automated detection of metastatic breast cancer from whole slide images of lymph nodes utilizing a deep convolutional neural network was proposed in this study. Methods: The dataset is taken from the PatchCamelyon subset, which contains 220,025 images divided into training, validation, and testing sets at a ratio of 60:20:20. The pretrained ResNet50 model was utilized, and transfer learning was subsequently applied to adjust the weights of the model. To elevate the model performance, the evaluation metrics were assessed by the accuracy score, confusion matrix, receiver operating characteristic (ROC) curve, and area under the ROC curve (AUC) score. Results: As a result, the proposed algorithm obtained high performance, with scores over 95% in all the evaluation methods, especially the AUC score, which achieved 0.989. Moreover, the model is validated in a testing set with the test-time augmentation (TTA) technique to enhance prediction quality and reduce generalization error. Conclusion: Overall, the proposed model achieves high accuracy when applying transfer learning. The results prove that the trained Resnet50 model can extract useful information from small cells in histopathologic images for breast cancer detection.