Predicting sex from retinal fundus photographs using automated deep learning

Edward Korot,Nikolas Pontikos,Livia Faes,Josef Huemer,Pearse A Keane,Siegfried K Wagner,Alastair K Denniston,Xiaoxuan Liu,Anthony Khawaja,Konstantinos Balaskas

doi:10.1038/s41598-021-89743-x

Abstract

Deep learning may transform health care, but model development has largely been dependent on availability of advanced technical expertise. Herein we present the development of a deep learning model by clinicians without coding, which predicts reported sex from retinal fundus photographs. A model was trained on 84,743 retinal fundus photos from the UK Biobank dataset. External validation was performed on 252 fundus photos from a tertiary ophthalmic referral center. For internal validation, the area under the receiver operating characteristic curve (AUROC) of the code free deep learning (CFDL) model was 0.93. Sensitivity, specificity, positive predictive value (PPV) and accuracy (ACC) were 88.8%, 83.6%, 87.3% and 86.5%, and for external validation were 83.9%, 72.2%, 78.2% and 78.6% respectively. Clinicians are currently unaware of distinct retinal feature variations between males and females, highlighting the importance of model explainability for this task. The model performed significantly worse when foveal pathology was present in the external validation dataset, ACC: 69.4%, compared to 85.4% in healthy eyes, suggesting the fovea is a salient region for model performance OR (95% CI): 0.36 (0.19, 0.70) p = 0.0022. Automated machine learning (AutoML) may enable clinician-driven automated discovery of novel insights and disease biomarkers.

Highlights

As age and gender, the latter with an area under the curve (AUC) of 0.97
We previously reported on the ability of physicians to create automated machine learning (AutoML) models for medical image analysis[24]
To evaluate reproducibility and address varying performance of deep learning algorithms involving random seed initiation, we retrained the model to identical specifications, and found similar performance with an AUC of 0.93

Summary

Introduction

As age and gender, the latter with an area under the curve (AUC) of 0.97. Here, the physiologic cause and effect relationships are not readily apparent to domain experts[21]. Predicting gender from fundus photos, previously inconceivable to those who spent their careers looking at retinas, withstood external validation on an independent dataset of patients with different baseline demographics[23]. Not likely to be clinically useful, this finding hints at the future potential of deep learning for the discovery of novel associations through unbiased modelling of high-dimensional data. We previously reported on the ability of physicians to create automated machine learning (AutoML) models for medical image analysis[24]. Since that proof of concept, AutoML platforms have advanced significantly, with multiple employing code free deep learning (CFDL). We demonstrate AutoML as a tool for automated discovery of novel insights by performing sex classification from retinal fundus photos, and comparing its performance to the bespoke deep learning model by Poplin et al[22]

Objectives

Methods

Results

Discussion

Conclusion