Test-time augmentation for deep learning-based cell segmentation on microscopy images

Nikita Moshkov,Nikita Moshkov,Nikita Moshkov,Botond Mathe,Reka Hollandi,Peter Horvath,Peter Horvath,Attila Kertesz-Farkas

doi:10.1038/s41598-020-61808-3

Abstract

Recent advancements in deep learning have revolutionized the way microscopy images of cells are processed. Deep learning network architectures have a large number of parameters, thus, in order to reach high accuracy, they require a massive amount of annotated data. A common way of improving accuracy builds on the artificial increase of the training set by using different augmentation techniques. A less common way relies on test-time augmentation (TTA) which yields transformed versions of the image for prediction and the results are merged. In this paper we describe how we have incorporated the test-time argumentation prediction method into two major segmentation approaches utilized in the single-cell analysis of microscopy images. These approaches are semantic segmentation based on the U-Net, and instance segmentation based on the Mask R-CNN models. Our findings show that even if only simple test-time augmentations (such as rotation or flipping and proper merging methods) are applied, TTA can significantly improve prediction accuracy. We have utilized images of tissue and cell cultures from the Data Science Bowl (DSB) 2018 nuclei segmentation competition and other sources. Additionally, boosting the highest-scoring method of the DSB with TTA, we could further improve prediction accuracy, and our method has reached an ever-best score at the DSB.

Highlights

Recent advancements in deep learning have revolutionized the way microscopy images of cells are processed
We have evaluated the performance of time augmentation (TTA) on two datasets, named ‘Fluorescent’ and ‘Tissue’ datasets, described in the “Dataset acquisition and description” section in detail
Each of them was split in 3 different ways to have approximately 5%, 15% and 30% as a test set. By using such versatile data collected from different sources and representing a wide variety of experimental conditions, as well as by the test set splits, we aimed to present the truly general performance of TTA, and demonstrate how robustly it works

Summary

Introduction

Recent advancements in deep learning have revolutionized the way microscopy images of cells are processed. In this paper we describe how we have incorporated the test-time argumentation prediction method into two major segmentation approaches utilized in the single-cell analysis of microscopy images. Deep learning approaches for object segmentation require a large, and often pixel-wise annotated dataset for training This task relies on high-quality samples and domain experts to accurately annotate images. Data augmentation has become the de facto technique in deep learning, especially in the case of heterogeneous or small datasets, to improve the accuracy of cell-based analysis. Another option of improving performance relies on augmenting both the training and the test datasets, performing the prediction both on the original and on the augmented versions of the image, followed by merging the predictions. Its disadvantage is increased prediction time, as it is run on the original image, but on all of its augmentations as well

Objectives

Methods

Results

Conclusion