Systematic Clinical Evaluation of a Deep Learning Method for Medical Image Segmentation: Radiosurgery Application.

Boris Shirokikh,Valery Kostjuchenko,Alexandra Dalechina,Andrey Golanov,Mikhail Galkin,Amayak Durgaryan,Mikhail Belyaev,Alexey Shevtsov,Egor Krivov

doi:10.1109/jbhi.2022.3153394

Abstract

We systematically evaluate a Deep Learning model in a 3D medical image segmentation task. With our model, we address the flaws of manual segmentation: high inter-rater contouring variability and time consumption of the contouring process. The main extension over the existing evaluations is the careful and detailed analysis that could be further generalized on other medical image segmentation tasks. Firstly, we analyze the changes in the inter-rater detection agreement. We show that the model reduces the number of detection disagreements by [Formula: see text] [Formula: see text]. Secondly, we show that the model improves the inter-rater contouring agreement from [Formula: see text] to [Formula: see text] surface Dice Score [Formula: see text]. Thirdly, we show that the model accelerates the delineation process between [Formula: see text] and [Formula: see text] times [Formula: see text]. Finally, we design the setup of the clinical experiment to either exclude or estimate the evaluation biases; thus, preserving the significance of the results. Besides the clinical evaluation, we also share intuitions and practical ideas for building an efficient DL-based model for 3D medical image segmentation.

Highlights

Advances in Deep Learning (DL) provide a solid foundation for automating medical image segmentation tasks [1]
We extend our previous work on clinical evaluation of a DL model for the brain metastases segmentation [16]
IV, we report the increase of sDSC and concordance index (CCI) when the raters switch the technique from manual contouring (MC) to assisted contouring (AC) along with the corresponding P-values

Summary

Introduction

Advances in Deep Learning (DL) provide a solid foundation for automating medical image segmentation tasks [1]. Segmentation algorithms are successfully used to compute clinically relevant anatomical or disease characteristics. These algorithms, Convolutional Neural Networks (CNN) in particular, achieve near human-level performance in various tasks: a brain tumor, lung cancer, organ-at-risk, or liver tumor segmentation; and continue to develop. The challenges and benchmark datasets, e.g., Brain Tumor Segmentation challenge [2], mainly accelerate this development where the algorithms improve over each other in terms of the computer imaging metrics. The results have been obtained under the support of the Russian Foundation for Basic Research grant 18-29-26030.

Objectives

Methods

Findings

Discussion

Conclusion