External testing of an established glaucoma screening AI on a glaucoma clinic population from Finland

Abstract

Purpose: To assess the external performance of an existing deep learning regression model for glaucoma detection (G‐RISK) in a glaucoma clinic population from Finland.Methods: Colour fundus images were retrospectively collected from the Tampere University Hospital (Tays), Finland. Eyes that had a visit in 2019 or 2020 were included based on the availability of ground truth labels. Ground truth labels for glaucoma were obtained through clinical evaluation of the optic nerve head (ONH) from colour fundus photos, clinical evaluation of retinal nerve fibre layer (RNFL) defects from fundus photos imaged with a cobalt filter, and clinical evaluation of the SITA Fast visual field (VF) test. Glaucoma diagnosis at Tays followed the ‘2 out of 3 rule’ recommended by the Finnish Evidence‐Based Guideline for Glaucoma, which requires at least 2 positive indications of glaucoma tests for referral [1]. Preprocessed colour fundus images were used as input to a previously trained deep learning model (G‐RISK) [2]. Performance was evaluated using the area under the receiver operating characteristic curve (AUC), accompanied by 95% confidence intervals. Patient aggregated results were obtained by comparing the maximum of both prediction and ground truth label.Results: Over 3000 patients had at least one ground truth available for a visit in 2019 or 2020. Clinical evaluation of structural features (ONH, RNFL) triggered more glaucomatous symptoms than the SITA Fast VF test. When thresholding the AI prediction against a single ground truth type, the best AUC was obtained on the ONH label (0.90 [0.89–0.91]), followed by RNFL (0.84 [0.83–0.86]) and VF (0.83 [0.81–0.84]). When implementing the 2 out of 3 rule, AUC reached 0.88 [0.82–0.90].Conclusion: The model performed best when using the ONH label and incorporating the ‘2 out of 3 rule’, reaching an AUC of 0.88. These findings suggest that the G‐RISK model could be a valuable tool for screening in clinical practice.