Validation of Soft Labels in Developing Deep Learning Algorithms for Detecting Lesions of Myopic Maculopathy From Optical Coherence Tomographic Images.

Abstract

It is common for physicians to be uncertain when examining some images. Models trained with human uncertainty could be a help for physicians in diagnosing pathologic myopia. This is a hospital-based study that included 9176 images from 1327 patients that were collected between October 2015 and March 2019. All collected images were graded by 21 myopia specialists according to the presence of myopic neovascularization (MNV), myopic traction maculopathy (MTM), and dome-shaped macula (DSM). Hard labels were made by the rule of major wins, while soft labels were possibilities calculated by whole grading results from the different graders. The area under the curve (AUC) of the receiver operating characteristics curve, the area under precision-recall (AUPR) curve, F-score, and least square errors were used to evaluate the performance of the models. The AUC values of models trained by soft labels in MNV, MTM, and DSM models were 0.985, 0.946, and 0.978; and the AUPR values were 0.908, 0.876, and 0.653 respectively. However, 0.56% of MNV "negative" cases were answered as "positive" with high certainty by the hard label model, whereas no case was graded with extreme errors by the soft label model. The same results were found for the MTM (0.95% vs none) and DSM (0.43% vs 0.09%) models. The predicted possibilities from the models trained by soft labels were close to the results made by myopia specialists. These findings could inspire the novel use of deep learning models in the medical field.