Deep Learning for Automated Sorting of Retinal Photographs

Abstract

Though the domain of big data and artificial intelligence in health care continues to evolve, there is a lack of systemic methods to improve data quality and streamline the preparation process. To address this, we aimed to develop an automated sorting system (RetiSort) that accurately labels the type and laterality of retinal photographs. Cross-sectional study. RetiSort was developed with retinal photographs from the Singapore Epidemiology of Eye Diseases (SEED) study. The development of RetiSort was composed of 3 steps: 2 deep-learning (DL) algorithms and 1 rule-based classifier. For step 1, a DL algorithm was developed to locate the optic disc, the "landmark feature." For step 2, based on the location of the optic disc derived from step 1, a rule-based classifier was developed to sort retinal photographs into 3 types: macular-centered, optic disc-centered, or related to other fields. Step 2 concurrently distinguished laterality (i.e., the left or right eye) of macular-centered photographs. For step 3, an additional DL algorithm was developed to differentiate the laterality of disc-centered photographs. Via the 3 steps, RetiSort sorted and labeled retinal images into (1) right macular-centered, (2) left macular-centered, (3) right optic disc-centered, (4) left optic disc-centered, and (5) images relating to other fields. Subsequently, the accuracy of RetiSort was evaluated on 5000 randomly selected retinal images from SEED as well as on 3 publicly available image databases (DIARETDB0, HEI-MED, and Drishti-GS). The main outcome measure was the accuracy for sorting of retinal photographs. RetiSort mislabeled 48 out of 5000 retinal images from SEED, representing an overall accuracy of 99.0% (95% confidence interval [CI], 98.7-99.3). In external tests, RetiSort mislabeled 1, 0, and 2 images, respectively, from DIARETDB0, HEI-MED, and Drishti-GS, representing an accuracy of 99.2% (95% CI, 95.8-99.9), 100%, and 98.0% (95% CI, 93.1-99.8), respectively. Saliency maps consistently showed that the DL algorithm in step 3 required pixels in the central left lateral border and optic disc of optic disc-centered retinal photographs to differentiate the laterality. RetiSort is a highly accurate automated sorting system. It can aid in data preparation and has practical applications in DL research that uses retinal photographs.