Referable Diabetic Retinopathy Research Articles

Comparison of 21 artificial intelligence algorithms in automated diabetic retinopathy screening using handheld fundus camera

Background Diabetic retinopathy (DR) is a common complication of diabetes and may lead to irreversible visual loss. Efficient screening and improved treatment of both diabetes and DR have amended visual prognosis for DR. The number of patients with diabetes is increasing and telemedicine, mobile handheld devices and automated solutions may alleviate the burden for healthcare. We compared the performance of 21 artificial intelligence (AI) algorithms for referable DR screening in datasets taken by handheld Optomed Aurora fundus camera in a real-world setting. Patients and methods Prospective study of 156 patients (312 eyes) attending DR screening and follow-up. Both papilla- and macula-centred 50° fundus images were taken from each eye. DR was graded by experienced ophthalmologists and 21 AI algorithms. Results Most eyes, 183 out of 312 (58.7%), had no DR and mild NPDR was noted in 21 (6.7%) of the eyes. Moderate NPDR was detected in 66 (21.2%) of the eyes, severe NPDR in 1 (0.3%), and PDR in 41 (13.1%) composing a group of 34.6% of eyes with referable DR. The AI algorithms achieved a mean agreement of 79.4% for referable DR, but the results varied from 49.4% to 92.3%. The mean sensitivity for referable DR was 77.5% (95% CI 69.1–85.8) and specificity 80.6% (95% CI 72.1–89.2). The rate for images ungradable by AI varied from 0% to 28.2% (mean 1.9%). Nineteen out of 21 (90.5%) AI algorithms resulted in grading for DR at least in 98% of the images. Conclusions Fundus images captured with Optomed Aurora were suitable for DR screening. The performance of the AI algorithms varied considerably emphasizing the need for external validation of screening algorithms in real-world settings before their clinical application.

Distribution shift detection for the postmarket surveillance of medical AI algorithms: a retrospective simulation study

Distribution shifts remain a problem for the safe application of regulated medical AI systems, and may impact their real-world performance if undetected. Postmarket shifts can occur for example if algorithms developed on data from various acquisition settings and a heterogeneous population are predominantly applied in hospitals with lower quality data acquisition or other centre-specific acquisition factors, or where some ethnicities are over-represented. Therefore, distribution shift detection could be important for monitoring AI-based medical products during postmarket surveillance. We implemented and evaluated three deep-learning based shift detection techniques (classifier-based, deep kernel, and multiple univariate kolmogorov-smirnov tests) on simulated shifts in a dataset of 130’486 retinal images. We trained a deep learning classifier for diabetic retinopathy grading. We then simulated population shifts by changing the prevalence of patients’ sex, ethnicity, and co-morbidities, and example acquisition shifts by changes in image quality. We observed classification subgroup performance disparities w.r.t. image quality, patient sex, ethnicity and co-morbidity presence. The sensitivity at detecting referable diabetic retinopathy ranged from 0.50 to 0.79 for different ethnicities. This motivates the need for detecting shifts after deployment. Classifier-based tests performed best overall, with perfect detection rates for quality and co-morbidity subgroup shifts at a sample size of 1000. It was the only method to detect shifts in patient sex, but required large sample sizes (>30′000\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$> 30^{\\prime} 000$$\\end{document}). All methods identified easier-to-detect out-of-distribution shifts with small (≤300) sample sizes. We conclude that effective tools exist for detecting clinically relevant distribution shifts. In particular classifier-based tests can be easily implemented components in the post-market surveillance strategy of medical device manufacturers.

Automated machine learning model for fundus image classification by health-care professionals with no coding experience

To assess the feasibility of code-free deep learning (CFDL) platforms in the prediction of binary outcomes from fundus images in ophthalmology, evaluating two distinct online-based platforms (Google Vertex and Amazon Rekognition), and two distinct datasets. Two publicly available datasets, Messidor-2 and BRSET, were utilized for model development. The Messidor-2 consists of fundus photographs from diabetic patients and the BRSET is a multi-label dataset. The CFDL platforms were used to create deep learning models, with no preprocessing of the images, by a single ophthalmologist without coding expertise. The performance metrics employed to evaluate the models were F1 score, area under curve (AUC), precision and recall. The performance metrics for referable diabetic retinopathy and macular edema were above 0.9 for both tasks and CFDL. The Google Vertex models demonstrated superior performance compared to the Amazon models, with the BRSET dataset achieving the highest accuracy (AUC of 0.994). Multi-classification tasks using only BRSET achieved similar overall performance between platforms, achieving AUC of 0.994 for laterality, 0.942 for age grouping, 0.779 for genetic sex identification, 0.857 for optic, and 0.837 for normality with Google Vertex. The study demonstrates the feasibility of using automated machine learning platforms for predicting binary outcomes from fundus images in ophthalmology. It highlights the high accuracy achieved by the models in some tasks and the potential of CFDL as an entry-friendly platform for ophthalmologists to familiarize themselves with machine learning concepts.

Comparative Analysis of Vision Transformers and Conventional Convolutional Neural Networks in Detecting Referable Diabetic Retinopathy

ObjectiveVision Transformers (ViTs) have shown promising performance in various classification tasks previously dominated by Convolutional Neural Networks (CNNs). However, the performance of ViTs in referable Diabetic Retinopathy (DR) detection is relatively underexplored. In this study, using retinal photographs, we evaluated the comparative performances of ViTs and CNNs on detection of referable DR. DesignRetrospective study ParticipantsA total of 48,269 retinal images from the open-source Kaggle DR detection dataset, the Messidor-1 dataset and the Singapore Epidemiology of Eye Diseases (SEED) study were included. MethodUsing 41,614 retinal photographs from the Kaggle dataset, we developed five CNN (VGG19, ResNet50, InceptionV3, DenseNet201 and EfficientNetV2S) and four ViTs models (VAN_small, CrossViT_small, ViT_small and SWIN_tiny) for the detection of referable DR. We defined the presence of referable DR as eyes with moderate or worse DR. The comparative performance of all nine models was evaluated in the Kaggle internal test dataset (with 1,045 study eyes), and in two external test sets, the SEED study (5,455 study eyes) and the Messidor-1 (1,200 study eyes). Main Outcome MeasuresArea Under Receiver Operative Curve (AUC), Specificity and Sensitvity. ResultsAmong all models, the SWIN transformer displayed the highest area under the curve (AUC) of 95.7% on the internal test set, significantly outperforming the CNN models (all P<0.001). The same observation was confirmed in the external test sets, with the SWIN transformer achieving AUC of 97.3% in SEED and 96.3% in Messidor-1. When specificity level was fixed at 80% for the internal test, the SWIN transformer achieved the highest sensitivity of 94.4%, significantly better than all the CNN models (sensitivity levels ranging between 76.3% and 83.8%; all P <0.001). This trend was also consistently observed in both external test sets. ConclusionOur findings demonstrate that ViTs provide superior performance over CNNs in detecting referable DR from retinal photographs. These results point to the potential of utilizing ViT models to improve and optimize retinal photo-based deep learning for referable DR detection.

Reliability of Clinical Grading of Diabetic Retinopathy Compared to Grading of Ultra-widefield Images

Purpose: Evaluate the reliability of clinical grading of diabetic retinopathy (DR) severity compared to grading on ultra-widefield pseudocolor fundus (UWF-CF) and ultra-widefield fluorescein angiography (UWF-FA) images and their relative detection of sight-threatening DR (STDR) and referable DR. Methods: We analyzed 184 diabetic eyes. UWF-CF and UWF-FA images were graded based on the International Clinical Diabetic Retinopathy (ICDR) severity scale. Agreement between clinical and UWF-based severity grading was evaluated using Cohen’s kappa coefficient. The rate of STDR and referable DR was evaluated for each grading method. Results: We found moderate agreement between clinical grading and UWF-CF (k=.456, p &lt;.001) and between UWF-CF and UWF-FA (k= .443, p &lt;.001). Agreement between clinical grading and UWF-FA was fair (k=.397, p &lt;.001). UWF-based grading identified a higher DR grade in 56 eyes (30%) on UWF-CF and 85 eyes (46.2%) on UWF-FA. Compared to clinical grading, UWF-FA detected a higher rate of STDR (44%; 81/184 vs 22.3%; 41/184), while UWF-CF detected more referable eyes (58.1%;107/184 vs 45.65%;84/184). Conclusion: UWF-CF is a valuable tool for identifying referable eyes and can be a useful, non-invasive adjunct to clinical grading. Our results suggest that UWF-FA is particularly useful for detecting unsuspected STDR in eyes with clinically referable DR.

Referable Diabetic Retinopathy Prediction Algorithm Applied to a Population of 120,389 Type 2 Diabetics over 11 Years Follow-Up.

(1) Background: Although DR screening is effective, one of its most significant problems is a lack of attendance. The aim of the present study was to demonstrate the effectiveness of our algorithm in predicting the development of any type of DR and referable DR. (2) Methods: A retrospective study with an 11-year follow-up of a population of 120,389 T2DM patients was undertaken. (3) Results: Applying the results of the algorithm showed an AUC of 0.93 (95% CI, 0.92-0.94) for any DR and 0.90 (95% CI, 0.89-0.91) for referable DR. Therefore, we achieved a promising level of agreement when applying our algorithm. (4) Conclusions: The algorithm is useful for predicting which patients may develop referable forms of DR and also any type of DR. This would allow a personalized screening plan to be drawn up for each patient.

Vision transformer with masked autoencoders for referable diabetic retinopathy classification based on large-size retina image.

Computer-aided diagnosis systems based on deep learning algorithms have shown potential applications in rapid diagnosis of diabetic retinopathy (DR). Due to the superior performance of Transformer over convolutional neural networks (CNN) on natural images, we attempted to develop a new model to classify referable DR based on a limited number of large-size retinal images by using Transformer. Vision Transformer (ViT) with Masked Autoencoders (MAE) was applied in this study to improve the classification performance of referable DR. We collected over 100,000 publicly fundus retinal images larger than 224×224, and then pre-trained ViT on these retinal images using MAE. The pre-trained ViT was applied to classify referable DR, the performance was also compared with that of ViT pre-trained using ImageNet. The improvement in model classification performance by pre-training with over 100,000 retinal images using MAE is superior to that pre-trained with ImageNet. The accuracy, area under curve (AUC), highest sensitivity and highest specificity of the present model are 93.42%, 0.9853, 0.973 and 0.9539, respectively. This study shows that MAE can provide more flexibility to the input image and substantially reduce the number of images required. Meanwhile, the pretraining dataset scale in this study is much smaller than ImageNet, and the pre-trained weights from ImageNet are not required also.

Design, recruitment and baseline characteristics of the LENS trial.

Findings from cardiovascular outcome trials suggest that treatment with fenofibrate may reduce the progression of diabetic retinopathy. However, no dedicated large-scale randomised trials have yet investigated this hypothesis. LENS is a streamlined randomised double-masked placebo-controlled trial, based in Scotland, assessing whether treatment with fenofibrate (145 mg tablet daily or, in the context of impaired renal function, on alternate days) in people with early retinopathy reduces progression to referable diabetic retinopathy (defined in NHS Scotland's Diabetic Eye Screening grading scheme as referable background or proliferative retinopathy, or referable maculopathy in either eye) or treatment with retinal laser, intravitreal injections or vitrectomy. Adults with diabetes mellitus and non-referable retinopathy (mild background retinopathy in both eyes or observable background retinopathy in one/both eyes at the most recent NHS retinal screening assessment; or observable maculopathy in one/both eyes in the previous 3 years) were eligible. Potential participants were identified from routinely collected healthcare data and followed up using regular contact from the research team and linkage to national electronic morbidity, mortality, biochemistry and retinal screening records. Study treatment was mailed to participants. Between 18 September 2018 and 27 July 2021, 1151 participants were randomised. Their mean age was 61 (SD 12) years, 312 (27%) were female and 305 (26%) had type 1 diabetes. 96% had bilateral mild background retinopathy and 10% had observable maculopathy. LENS will provide a robust evaluation of the efficacy of treating people at risk of progression of diabetic retinopathy with fenofibrate. Results are anticipated in mid-2024. NCT03439345; ISRCTN15073006; EuDRACT 2016-002656-24.

Diabetic retinopathy screening using MII Ret Cam assisted smartphone-based fundus imaging

BackgroundDiabetes Mellitus is a tremendous health implication and is proposed to be the seventh leading cause of death in 2030 by WHO. Diabetic retinopathy is one of the top complications of diabetes leading to preventable blindness. This investigation highlights the importance of timely screening and counseling using MII Ret cCam-assisted smartphone-based fundus imaging. Our study is a boon for bed-ridden and seriously ill patients who could not be evaluated on a fundus camera. MethodsThis is an observational prospective study conducted at the Ophthalmology unit, wards, and intensive care unit (ICU) of a single tertiary teaching hospital between 1 May to 31 November 2022. A total of 268 subjects with diabetes were included. The fundus of all the diabetic patients during the study period were examined by a single-trained ophthalmologist. Those diagnosed with diabetic retinopathy were taken for acquisition of fundus picture with MII Ret Cam. Images were captured using Apple iPhone 5s and 20D mounted in MII Ret Cam device. ResultsOut of 268 subjects (536 eyes) screened during the study period,86 patients (154 eyes) were found to have DR. Maleto Female ratio was 1.4:1. The mean age was 56.37 ± 15.26 years and the mean duration of diabetes was 10.60 ± 6.97 years. Grading of DR revealed proliferative diabetic retinopathy (PDR) to be the topmost (n=84 eyes, 54.5%), followed by mild non-proliferative diabetic retinopathy (NPDR) in 26 eyes (16%) and moderate NPDR in 24 eyes (15%) while severe NPDR was seen in 10 eyes (6.5%) and very severe NPDR was found in 8 eyes (5.2%) and diabetic papillopathy in 2 eyes. Grading of vitreous hemorrhage revealed Grade 1 in one eye (1.8%); Grade 2 in 19 eyes (34.54%); Grade 3 in 33 eyes (60%) and Grade 4 in 2 eyes (3.63%). Grading of macular edema according to the distribution of hard exudates as Grade 1(No Clinically significant macular edema) in 33 eyes (34.37%) and Grade 2 (Clinically significant macular edema) in 63 eyes (65.62%). The duration of diabetes had a direct association with the occurrence of DR, 13patients (15.1%) had a duration of diabetes less than five years; 31patients(36%) had a duration of diabetes between 6 years to 10 years; 28 patients (32.6%) had a duration between 11 years to 20 years. 14 patients (16.3%) had a duration of more than twenty years. The Mean duration of diabetes in Group I was 11.40+/- 7.09 years and was 9.10+/- 6.48 in Group II. The mean HbA1c in Group I is 7.44+/- 1.24 and 6.54 +/- 1.09 in Group II. The mean fasting blood sugar in Group I was 130.87 +/- 29.57 and 106.62+/- 29.85 in Group II. The mean of random blood sugar was 202.15 +/- 60.84 in Group I and 155.13 +/- 55.95 in Group II. Results are suggestive of the association of DR occurrence with fasting, random blood sugar, duration of diabetes, and HbA1c. ConclusionsDiabetic Retinopathy Screening using MII Ret Cam assisted Smartphone-based fundus imaging may be a useful device to screen for referable and vision-threatening diabetic retinopathy. MII Ret Cam is portable equipment and is easily accessible to counsel patients in resource-limited countries.

Prediction of retinopathy progression using deep learning on retinal images within the Scottish screening programme

Background/aimsNational guidelines of many countries set screening intervals for diabetic retinopathy (DR) based on grading of the last screening retinal images. We explore the potential of deep learning (DL) on...

Accuracy of the EyeArt artificial intelligence system in detecting referable diabetic retinopathy in patients with diabetes in Hong Kong

Accuracy of the EyeArt artificial intelligence system in detecting referable diabetic retinopathy in patients with diabetes in Hong Kong

Artificial intelligence-supported diabetic retinopathy screening in Tanzania: rationale and design of a randomised controlled trial

IntroductionGlobally, diabetic retinopathy (DR) is a major cause of blindness. Sub-Saharan Africa is projected to see the largest proportionate increase in the number of people living with diabetes over the...

Association between remnant cholesterol levels and incident diabetic retinopathy in middle-aged and older patients with diabetes

Objective: To investigate the association between remnant cholesterol (RC) and the risk of diabetic retinopathy (DR) in middle-aged and older individuals with diabetes. Methods: Based on the Shanghai Nicheng Cohort Study database, the data of 1 255 individuals with diabetes aged 55-70 years at baseline (2013-2014) with complete fundus photographs and serum cholesterol data in Nicheng, Shanghai, were analyzed. Multinomial logistic regression models were used to evaluate risk ratios (RRs) and their 95% confidence intervals (CIs) between baseline RC level and incident DR. Results: The median age of the subjects was 61.9 years, and 60.4% were women. After a 4.6-year follow-up, 79 (6.3%) patients developed DR, including 50 (4.0%) mild non-proliferative DR and 29 (2.3%) referable DR (RDR). Multivariable logistic regression showed that each mmol/L increase of RC was associated with a 40% higher risk of RDR (RR=1.40, 95%CI 1.03-1.90). Compared with the lowest tertile of RC (<0.63 mmol/L), the risk of RDR in the highest tertile (≥0.85 mmol/L) increased by 4.59 times (RR=5.59, 95%CI 1.51-20.73). Conclusion: The RC level may help identify individuals at high risk of incident RDR in middle-aged and older Chinese adults with diabetes.

Real-world evaluation of smartphone-based artificial intelligence to screen for diabetic retinopathy in Dominica: a clinical validation study

ObjectiveSeveral artificial intelligence (AI) systems for diabetic retinopathy screening have been validated but there is limited evidence on their performance in real-world settings. This study aimed to assess the performance...

A Hybrid CNN Model for Deep Feature Extraction for Diabetic Retinopathy Detection and Gradation

Diabetic retinopathy (DR) is a complication of prolonged diabetic mellitus causing damage to the micro-vascular system in retina. Computer-aided detection of DR can significantly reduce time, effort, and cost in early diagnosis and thereby restrict further complications. In this paper, we present a deep learning based framework for automated DR stage classification from fundus retinal images.We design and develop a custom build Convolutional Neural Network (CNN) model named Model-IVS to extract meaningful and discriminative features from the retinal fundus images. The proposed hybrid-CNN combines the multi-scale pyramidal feature processing concept of inception model with the convolutional pipeline based feature extraction concept of VGG-Nets and adopts multi-scale feature concatenation based short-cut connections of DenseNet with learnable convolutions. For the binary classification tasks, we employ Model-IVS as a feature extractor and use Support Vector Machine (SVM) classifier on the reduced features obtained through principal component analysis. The proposed hybrid model is end-to-end trained incrementally on a class-balanced dataset curated from the train set of EyePACS dataset. The cross-dataset generalization of the proposed hybrid CNN based DR classification framework is evaluated on APTOS, MESSIDOR-1 and MESSIDOR-2 datasets. Experimental results demonstrate that our model has outperformed all the existing state-of-the-art methods for DR severity grading task on APTOS dataset. The proposed Model-IVS has achieved best accuracy, AUC-ROC, and kappa score of 84.33%, 0.978 and 0.899, respectively in DR severity grading task. In binary classification tasks like DR screening and referable DR predictions, our Model-IVS feature extraction network with SVM classifier on PCA features has also achieved comparable performance results w.r.t. the state-of-the-art methods. In DR screening task, we recorded an accuracy of 97.84%, 87.42%, and 89.67% on APTOS, MESSIDOR-1, and MESSIDOR-2 datasets, respectively. In referable DR prediction, we achieved AUC-ROC score of 0.987, 0.928, and 0.873 on APTOS, MESSIDOR-1, and MESSIDOR-2 datasets, respectively.

Clinician-Driven AI: Code-Free Self-Training on Public Data for Diabetic Retinopathy Referral

Democratizing artificial intelligence (AI) enables model development by clinicians with a lack of coding expertise, powerful computing resources, and large, well-labeled data sets. To determine whether resource-constrained clinicians can use self-training via automated machine learning (ML) and public data sets to design high-performing diabetic retinopathy classification models. This diagnostic quality improvement study was conducted from January 1, 2021, to December 31, 2021. A self-training method without coding was used on 2 public data sets with retinal images from patients in France (Messidor-2 [n = 1748]) and the UK and US (EyePACS [n = 58 689]) and externally validated on 1 data set with retinal images from patients of a private Egyptian medical retina clinic (Egypt [n = 210]). An AI model was trained to classify referable diabetic retinopathy as an exemplar use case. Messidor-2 images were assigned adjudicated labels available on Kaggle; 4 images were deemed ungradable and excluded, leaving 1744 images. A total of 300 images randomly selected from the EyePACS data set were independently relabeled by 3 blinded retina specialists using the International Classification of Diabetic Retinopathy protocol for diabetic retinopathy grade and diabetic macular edema presence; 19 images were deemed ungradable, leaving 281 images. Data analysis was performed from February 1 to February 28, 2021. Using public data sets, a teacher model was trained with labeled images using supervised learning. Next, the resulting predictions, termed pseudolabels, were used on an unlabeled public data set. Finally, a student model was trained with the existing labeled images and the additional pseudolabeled images. The analyzed metrics for the models included the area under the receiver operating characteristic curve (AUROC), accuracy, sensitivity, specificity, and F1 score. The Fisher exact test was performed, and 2-tailed P values were calculated for failure case analysis. For the internal validation data sets, AUROC values for performance ranged from 0.886 to 0.939 for the teacher model and from 0.916 to 0.951 for the student model. For external validation of automated ML model performance, AUROC values and accuracy were 0.964 and 93.3% for the teacher model, 0.950 and 96.7% for the student model, and 0.890 and 94.3% for the manually coded bespoke model, respectively. These findings suggest that self-training using automated ML is an effective method to increase both model performance and generalizability while decreasing the need for costly expert labeling. This approach advances the democratization of AI by enabling clinicians without coding expertise or access to large, well-labeled private data sets to develop their own AI models.

Diagnostic test accuracy of artificial intelligence in screening for referable diabetic retinopathy in real-world settings: A systematic review and meta-analysis.

Retrospective studies on artificial intelligence (AI) in screening for diabetic retinopathy (DR) have shown promising results in addressing the mismatch between the capacity to implement DR screening and increasing DR incidence. This review sought to evaluate the diagnostic test accuracy (DTA) of AI in screening for referable diabetic retinopathy (RDR) in real-world settings. We searched CENTRAL, PubMed, CINAHL, Scopus, and Web of Science on 9 February 2023. We included prospective DTA studies assessing AI against trained human graders (HGs) in screening for RDR in patients with diabetes. Two reviewers independently extracted data and assessed methodological quality against QUADAS-2 criteria. We used the hierarchical summary receiver operating characteristics (HSROC) model to pool estimates of sensitivity and specificity and, forest plots and SROC plots to visually examine heterogeneity in accuracy estimates. From our initial search results of 3899 studies, we included 15 studies comprising 17 datasets. Meta-analyses revealed a sensitivity of 95.33% (95%CI: 90.60-100%) and specificity of 92.01% (95%CI: 87.61-96.42%) for patient-level analysis (10 datasets, N = 45,785) while, for the eye-level analysis, sensitivity was 91.24% (95%CI: 79.15-100%) and specificity, 93.90% (95%CI: 90.63-97.16%) (7 datasets, N = 15,390). Subgroup analyses did not provide variations in the diagnostic accuracy of country classification and DR classification criteria. However, a moderate increase was observed in diagnostic accuracy in the primary-level healthcare settings: sensitivity of 99.35% (95%CI: 96.85-100%), specificity of 93.72% (95%CI: 88.83-98.61%) and, a minimal decrease in the tertiary-level healthcare settings: sensitivity of 94.71% (95%CI: 89.00-100%), specificity of 90.88% (95%CI: 83.22-98.53%). Sensitivity analyses did not show any variations in studies that included diabetic macular edema in the RDR definition, nor studies with ≥3 HGs. This review provides evidence, for the first time from prospective studies, for the effectiveness of AI in screening for RDR in real-world settings. The results may serve to strengthen existing guidelines to improve current practices.

Feasibility and acceptance of artificial intelligence-based diabetic retinopathy screening in Rwanda

BackgroundEvidence on the practical application of artificial intelligence (AI)-based diabetic retinopathy (DR) screening is needed.MethodsConsented participants were screened for DR using retinal imaging with AI interpretation from March 2021 to...

Artificial intelligence to detect referable diabetic retinopathy in a Chinese population in Hong Kong

Artificial intelligence to detect referable diabetic retinopathy in a Chinese population in Hong Kong

Variability in Grading Diabetic Retinopathy Using Retinal Photography and Its Comparison with an Automated Deep Learning Diabetic Retinopathy Screening Software.

Diabetic retinopathy (DR) screening using colour retinal photographs is cost-effective and time-efficient. In real-world clinical settings, DR severity is frequently graded by individuals of different expertise levels. We aim to determine the agreement in DR severity grading between human graders of varying expertise and an automated deep learning DR screening software (ADLS). Using the International Clinical DR Disease Severity Scale, two hundred macula-centred fundus photographs were graded by retinal specialists, ophthalmology residents, family medicine physicians, medical students, and the ADLS. Based on referral urgency, referral grading was divided into no referral, non-urgent referral, and urgent referral to an ophthalmologist. Inter-observer and intra-group variations were analysed using Gwet's agreement coefficient, and the performance of ADLS was evaluated using sensitivity and specificity. The agreement coefficient for inter-observer and intra-group variability ranged from fair to very good, and moderate to good, respectively. The ADLS showed a high area under curve of 0.879, 0.714, and 0.836 for non-referable DR, non-urgent referable DR, and urgent referable DR, respectively, with varying sensitivity and specificity values. Inter-observer and intra-group agreements among human graders vary widely, but ADLS is a reliable and reasonably sensitive tool for mass screening to detect referable DR and urgent referable DR.