Retinal Photograph-Based Deep Learning System for Detection of Hyperthyroidism: A Multicenter, Diagnostic Study

Abstract

Background: Screening for hyperthyroidism using gold-standard diagnostic criteria in the general population is not cost-effective, leading to a relatively high rate of undiagnosed and untreated patients. This study aimed to establish a deep learning-based system to detect hyperthyroidism based on retinal photographs. Methods: The multicenter, observational study included retinal photographs taken from participants in two hospitals and 24 health care centers throughout China. We first trained two models to identify hyperthyroidism: in model #1, the non-hyperthyroidism individuals were randomly selected, while in model #2, the non-hyperthyroidism group was matched for age and gender with the hyperthyroidism group. After internal validation, we selected the better model for further evaluation using external validation datasets. Results: The study included 22,940 retinal photographs of 11,409 participants for the model development, and 3862 retinal photographs (1870 participants) which were obtained from two hospitals and four medical centers as the external validation datasets. Model #1 achieved a higher area under the receiver operator curve (AUC) than model #2 (0.907, 95% CI: 0.894-0.918 versus 0.850, 95% CI: 0.832-0.866) in the internal validation so that model #1 was used for further evaluation. In external datasets, model #1 reached AUCs ranging from 0.816 (95% CI 0.789-0.846) to 0.849 (95% CI 0.824-0.874) and achieved accuracies between 0.735 (95% CI 0.700-0.773) and 0.796 (95% CI 0.765-0.824). Heatmaps showed a focus of the DL-algorism on large fundus vessels and the optic nerve head. Conclusions: Retinal fundus photographs may serve for DL systems for a cost-effective and non-invasive method to detect hyperthyroidism. Funding Information: This study was supported by the National Natural Science Foundation of China (82071005); The Special Fund of the Pediatric Medical Coordinated Development Center of Beijing Hospitals Authority (XTCX201824); the Research Foundation of Beijing Friendship Hospital, Capital Medical University (yyqdkt2018-33); the Capital Health Research and Development of Special (2020-1-2052); Science & Technology Project of Beijing Municipal Science & Technology Commission (Z201100005520045, Z181100001818003); the Beijing Municipal Administration of Hospitals’ Ascent Plan (DFL20150201) Declaration of Interests: Authors Lie Ju, Xin Wang, Xin Zhao, Chao He, Yu Zhong Chen, were employed by the company Beijing Eaglevision Technology Co., Ltd, China. Authors Zhao Hui Wang and Jian Xiong Gao were employed by the company iKang Guobin Healthcare Group Co., Ltd, Beijing, China. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. Ethics Approval Statement: The Medical Ethics Committee of the Beijing Tongren Hospital, and the Ethics Committee the iKang Corporation approved the study protocol fulfilling the requirements published in the Helsinki declaration. For patients whose fundus images were stored in the retrospective databases at each participating hospital, informed consent was waived by the institutional review boards.