A crowdsourcing framework for retinal image semantic annotation and report documentation with deep learning enhancement

Abstract

To propose and implement a crowdsourcing framework for retinal image annotations to improve the annotation efficiency. In this study, open-source Bluelight was taken as backbone of the front end for online manual retinal image annotation for image semantic annotation and report documents, and based on that intelligent annotation and classification with deep learning (DL) was supplemented. For DL modules, we trained Mask-RCNN model to explicitly label the area of optic disc and macula. Furthermore, we trained Inception V3 model to classify diabetic retinopathy (DR) and normal retina. Then, we used Flask as the backend serving DL models. Finally, the implementation of interoperable annotation reports documentation and retrieval were conducted based on Lucene. The crowdsourcing framework was specially designed for professional doctors and computer researchers who have the ability to annotate. It efficiently and quickly completed the annotation of the retinal image and the macular area, and at the same time classified DR. Under this Browser/Server architecture, the tool achieved good cross-platform performance. In particular, the framework could provide annotation report documents to facilitate the optimization of subsequent DL models. Such crowdsourcing framework and reports documentation for retina semantic annotation could improve the effect of annotation and classification and worth further improvement and clinical validation.