RIL-Contour: a Medical Imaging Dataset Annotation Tool for and with Deep Learning

Kenneth A Philbrick,Arunnit Boonrod,Bradley J Erickson,Zeynettin Akkus,Atefeh Zeinoddini,Panagiotis Korfiatis,Timothy L Kline,Petro Kostandy,Naoki Takahashi,Tomas Sakinis,Alexander D Weston

doi:10.1007/s10278-019-00232-0

Abstract

Deep-learning algorithms typically fall within the domain of supervised artificial intelligence and are designed to “learn” from annotated data. Deep-learning models require large, diverse training datasets for optimal model convergence. The effort to curate these datasets is widely regarded as a barrier to the development of deep-learning systems. We developed RIL-Contour to accelerate medical image annotation for and with deep-learning. A major goal driving the development of the software was to create an environment which enables clinically oriented users to utilize deep-learning models to rapidly annotate medical imaging. RIL-Contour supports using fully automated deep-learning methods, semi-automated methods, and manual methods to annotate medical imaging with voxel and/or text annotations. To reduce annotation error, RIL-Contour promotes the standardization of image annotations across a dataset. RIL-Contour accelerates medical imaging annotation through the process of annotation by iterative deep learning (AID). The underlying concept of AID is to iteratively annotate, train, and utilize deep-learning models during the process of dataset annotation and model development. To enable this, RIL-Contour supports workflows in which multiple-image analysts annotate medical images, radiologists approve the annotations, and data scientists utilize these annotations to train deep-learning models. To automate the feedback loop between data scientists and image analysts, RIL-Contour provides mechanisms to enable data scientists to push deep newly trained deep-learning models to other users of the software. RIL-Contour and the AID methodology accelerate dataset annotation and model development by facilitating rapid collaboration between analysts, radiologists, and engineers.

Highlights

Deep-learning algorithms typically fall within the domain of supervised artificial intelligence and are designed to Blearn^ from annotated data [1]
The ImageNet dataset used to train powerful general-purpose deep-learning image classifiers contains millions of unique images each annotated to describe the objects contained within the image [2]
The effort required to curate these training datasets is widely regarded as a major barrier to the development of deep-learning systems

Summary

Introduction

Deep-learning algorithms typically fall within the domain of supervised artificial intelligence and are designed to Blearn^ from annotated data [1]. RIL-Contour provides a mechanism to manage the association between imaging data and annotation metadata for datasets stored on the file system or within a MIRMAID content management system These interfaces are designed to minimize workflow complexity and empower the data analyst to focus on data annotation and review and not on the management of imaging and metadata. RIL-Contour is designed to facilitate AID by (1) enabling deep-learning models to be applied to annotation images from within the software, (2) by providing mechanism from within the software to edit deep-learning derived annotations, (3) by providing a mechanism to export data to promote rapid model training, (4) by supporting concurrent workflows, and (5) by providing mechanisms which automate the sharing of deep-learning models between users of the software. A focus of future development efforts is to add support in RIL-Contour to export annotated datasets in the DICOM format to facilitate the utilization of RIL-Contour annotated datasets in other software packages

Conclusion

Findings

13. Fischl B